Causal Priors and Their Influence on Judgements of Causality in Visualized Data

Results from the first study in this paper of participant-rated causal relationships for 56 concept pairs curated from open-source datasets. Participants were asked questions in the form "How much will an increase in X cause an increase in Y?" for each X→Y concept pair. In (a), the Y-axis represents the participant-reported scores for concept causal relations (1 to 5: none to high). Each concept pair is placed in order by mean causal relation along the X-axis, showing 95% confidence intervals. The light blue horizontal band represents the mean score across all concept pairs ± one standard deviation (SD). The vertical dashed lines delineate concept pairs that we refer to as having either low causal priors (<mean-SD) or high causal priors (>mean+SD). Part (b) shows four example concept pairs from different parts of the causal prior spectrum. The heat maps show the number of participants in our study reporting each score on the 1-5 causal scale. As these results show, causal priors can vary widely across different concept pairs. In the second study in this paper (Section 3.4), we examine the impact of these causal priors on visualization interpretation.

Abstract-"Correlation does not imply causation" is a famous mantra in statistical and visual analysis. However, consumers of visualizations often draw causal conclusions when only correlations between variables are shown. In this paper, we investigate factors that contribute to causal relationships users perceive in visualizations. We collected a corpus of concept pairs from variables in widely used datasets and created visualizations that depict varying correlative associations using three typical statistical chart types. We conducted two MTurk studies on (1) preconceived notions on causal relations without charts, and (2) perceived causal relations with charts, for each concept pair. Our results indicate that people make assumptions about causal relationships between pairs of concepts even without seeing any visualized data. Moreover, our results suggest that these assumptions constitute causal priors that, in combination with visualized association, impact how data visualizations are interpreted. The results also suggest that causal priors may lead to over-or under-estimation in perceived causal relations in different circumstances, and that those priors can also impact users' confidence in their causal assessments. In addition, our results align with prior work, indicating that chart type may also affect causal inference. Using data from the studies, we develop a model to capture the interaction between causal priors and visualized associations as they combine to impact a user's perceived causal relations. In addition to reporting the study results and analyses, we provide an open dataset of causal priors for 56 specific concept pairs that can serve as a potential benchmark for future studies. We also suggest remaining challenges and heuristic-based guidelines to help designers improve visualization design choices to better support visual causal inference.