Mitigating the Attraction Effect with Visualizations

Unmasking Dunning-Kruger Effect in Visual Reasoning & Judgment

The Dunning-Kruger Effect (DKE) is a metacognitive phenomenon where low-skilled individuals tend to overestimate their competence while high-skilled individuals tend to underestimate their competence. This effect has been observed in a number of domains including humor, grammar, and logic. In this paper, we explore if and how DKE manifests in visual reasoning and judgment tasks. Across two online user studies involving (1) a sliding puzzle game and (2) a scatterplot-based categorization task, we demonstrate that individuals are susceptible to DKE in visual reasoning and judgment tasks: those who performed best underestimated their performance, while bottom performers overestimated their performance. In addition, we contribute novel analyses that correlate susceptibility of DKE with personality traits and user interactions. Our findings pave the way for novel modes of bias detection via interaction patterns and establish promising directions towards interventions tailored to an individual's personality traits. All materials and analyses are in supplemental materials: https://github.com/CAV-Lab/DKE_supplemental.git.

Comparison of visual assessment and quantitative goodness-of-fit metrics on GUTS model fits

For the application of toxicokinetic-toxicodynamic (TKTD) models in the European environmental risk assessment (ERA) of plant protection products, it is recommended to evaluate model predictions of the calibration as well as the independent validation data set based on qualitative criteria (visual assessment) and quantitative goodness-of-fit (GoF) metrics. The aims of this study were to identify whether quantitative criteria coincide with human visual perception of model performance and which evaluator characteristics influence their perception. In an anonymous online survey, > 70 calibration and validation general unified threshold models of survival (GUTS) fits were ranked by 64 volunteers with a professional interest in ecotoxicology and TKTD modeling. Participants were asked to score model fits to the time resolved survival data from toxicity experiments and to an aggregated dose-response curve representation. Dose-response curve plots tended to be scored better than time series, although both representations were based on the same toxicity test data and model results. For the time series, quantitative indices and visual assessments generally agreed on model performance. However, rankings varied with individual perceptions of the participants. Visual assessment scores were best predicted using a combination of GoF metrics. From the survey participants' majority agreement on fit acceptance, GoF cut-off criteria could be derived that indicated sufficient fit performance. The most conservative GoF criterion well resembled current suggestions by the European Food Safety Authority. Hence, the survey results provide evidence that current quantitative GUTS assessment practice in ERA is consistent with perceptions of fit quality based on visual judgements of the dynamic model behavior by a large number of practitioners. Thus, our study fosters trust in model performance assessment.

Available and unavailable decoys in capuchin monkeys (Sapajus spp.) decision-making

Decision-making has been observed to be systematically affected by decoys, i.e., options that should be irrelevant, either because unavailable or because manifestly inferior to other alternatives, and yet shift preferences towards their target. Decoy effects have been extensively studied both in humans and in several other species; however, evidence in non-human primates remains scant and inconclusive. To address this gap, this study investigates how choices in capuchin monkeys (Sapajus spp.) are affected by different types of decoys: asymmetrically dominated decoys, i.e., available and unavailable options that are inferior to only one of the other alternatives, and phantom decoys, i.e., unavailable options that are superior to another available alternative. After controlling for the subjective strength of initial preferences and the distance of each decoy from its target in attribute space, results demonstrate a systematic shift in capuchins' preference towards the target of both asymmetrically dominated decoys (whether they are available or not) and phantom decoys, regardless of what options is being targeted by such decoys. This provides the most comprehensive evidence to date of decoy effects in non-human primates, with important theoretical and methodological implications for future comparative studies on context effects in decision-making.

From Information to Choice: A Critical Inquiry Into Visualization Tools for Decision Making

In the face of complex decisions, people often engage in a three-stage process that spans from (1) exploring and analyzing pertinent information (intelligence); (2) generating and exploring alternative options (design); and ultimately culminating in (3) selecting the optimal decision by evaluating discerning criteria (choice). We can fairly assume that all good visualizations aid in the "intelligence" stage by enabling data exploration and analysis. Yet, to what degree and how do visualization systems currently support the other decision making stages, namely "design" and "choice"? To further explore this question, we conducted a comprehensive review of decision-focused visualization tools by examining publications in major visualization journals and conferences, including VIS, EuroVis, and CHI, spanning all available years. We employed a deductive coding method and in-depth analysis to assess whether and how visualization tools support design and choice. Specifically, we examined each visualization tool by (i) its degree of visibility for displaying decision alternatives, criteria, and preferences, and (ii) its degree of flexibility for offering means to manipulate the decision alternatives, criteria, and preferences with interactions such as adding, modifying, changing mapping, and filtering. Our review highlights the opportunities and challenges that decision-focused visualization tools face in realizing their full potential to support all stages of the decision making process. It reveals a surprising scarcity of tools that support all stages, and while most tools excel in offering visibility for decision criteria and alternatives, the degree of flexibility to manipulate these elements is often limited, and the lack of tools that accommodate decision preferences and their elicitation is notable. Based on our findings, to better support the choice stage, future research could explore enhancing flexibility levels and variety, exploring novel visualization paradigms, increasing algorithmic support, and ensuring that this automation is user-controlled via the enhanced flexibility I evels. Our curated list of the 88 surveyed visualization tools is available in the OSF link (https://osf.io/nrasz/?view_only=b92a90a34ae241449b5f2cd33383bfcb).

The Influence of Visual Provenance Representations on Strategies in a Collaborative Hand-off Data Analysis Scenario

Conducting data analysis tasks rarely occur in isolation. Especially in intelligence analysis scenarios where different experts contribute knowledge to a shared understanding, members must communicate how insights develop to establish common ground among collaborators. The use of provenance to communicate analytic sensemaking carries promise by describing the interactions and summarizing the steps taken to reach insights. Yet, no universal guidelines exist for communicating provenance in different settings. Our work focuses on the presentation of provenance information and the resulting conclusions reached and strategies used by new analysts. In an open-ended, 30-minute, textual exploration scenario, we qualitatively compare how adding different types of provenance information (specifically data coverage and interaction history) affects analysts' confidence in conclusions developed, propensity to repeat work, filtering of data, identification of relevant information, and typical investigation strategies. We see that data coverage (i.e., what was interacted with) provides provenance information without limiting individual investigation freedom. On the other hand, while interaction history (i.e., when something was interacted with) does not significantly encourage more mimicry, it does take more time to comfortably understand, as represented by less confident conclusions and less relevant information-gathering behaviors. Our results contribute empirical data towards understanding how provenance summarizations can influence analysis behaviors.

VITALITY: Promoting Serendipitous Discovery of Academic Literature with Transformers & Visual Analytics

There are a few prominent practices for conducting reviews of academic literature, including searching for specific keywords on Google Scholar or checking citations from some initial seed paper(s). These approaches serve a critical purpose for academic literature reviews, yet there remain challenges in identifying relevant literature when similar work may utilize different terminology (e.g., mixed-initiative visual analytics papers may not use the same terminology as papers on model-steering, yet the two topics are relevant to one another). In this paper, we introduce a system, VITALITY, intended to complement existing practices. In particular, VITALITY promotes serendipitous discovery of relevant literature using transformer language models, allowing users to find semantically similar papers in a word embedding space given (1) a list of input paper(s) or (2) a working abstract. VITALITY visualizes this document-level embedding space in an interactive 2-D scatterplot using dimension reduction. VITALITY also summarizes meta information about the document corpus or search query, including keywords and co-authors, and allows users to save and export papers for use in a literature review. We present qualitative findings from an evaluation of VITALITY, suggesting it can be a promising complementary technique for conducting academic literature reviews. Furthermore, we contribute data from 38 popular data visualization publication venues in VITALITY, and we provide scrapers for the open-source community to continue to grow the list of supported venues.

A Critical Reflection on Visualization Research: Where Do Decision Making Tasks Hide?

It has been widely suggested that a key goal of visualization systems is to assist decision making, but is this true? We conduct a critical investigation on whether the activity of decision making is indeed central to the visualization domain. By approaching decision making as a user task, we explore the degree to which decision tasks are evident in visualization research and user studies. Our analysis suggests that decision tasks are not commonly found in current visualization task taxonomies and that the visualization field has yet to leverage guidance from decision theory domains on how to study such tasks. We further found that the majority of visualizations addressing decision making were not evaluated based on their ability to assist decision tasks. Finally, to help expand the impact of visual analytics in organizational as well as casual decision making activities, we initiate a research agenda on how decision making assistance could be elevated throughout visualization research.

Left, Right, and Gender: Exploring Interaction Traces to Mitigate Human Biases

Human biases impact the way people analyze data and make decisions. Recent work has shown that some visualization designs can better support cognitive processes and mitigate cognitive biases (i.e., errors that occur due to the use of mental "shortcuts"). In this work, we explore how visualizing a user's interaction history (i.e., which data points and attributes a user has interacted with) can be used to mitigate potential biases that drive decision making by promoting conscious reflection of one's analysis process. Given an interactive scatterplot-based visualization tool, we showed interaction history in real-time while exploring data (by coloring points in the scatterplot that the user has interacted with), and in a summative format after a decision has been made (by comparing the distribution of user interactions to the underlying distribution of the data). We conducted a series of in-lab experiments and a crowd-sourced experiment to evaluate the effectiveness of interaction history interventions toward mitigating bias. We contextualized this work in a political scenario in which participants were instructed to choose a committee of 10 fictitious politicians to review a recent bill passed in the U.S. state of Georgia banning abortion after 6 weeks, where things like gender bias or political party bias may drive one's analysis process. We demonstrate the generalizability of this approach by evaluating a second decision making scenario related to movies. Our results are inconclusive for the effectiveness of interaction history (henceforth referred to as interaction traces) toward mitigating biased decision making. However, we find some mixed support that interaction traces, particularly in a summative format, can increase awareness of potential unconscious biases.

The Weighted Average Illusion: Biases in Perceived Mean Position in Scatterplots

Scatterplots can encode a third dimension by using additional channels like size or color (e.g. bubble charts). We explore a potential misinterpretation of trivariate scatterplots, which we call the weighted average illusion, where locations of larger and darker points are given more weight toward x- and y-mean estimates. This systematic bias is sensitive to a designer's choice of size or lightness ranges mapped onto the data. In this paper, we quantify this bias against varying size/lightness ranges and data correlations. We discuss possible explanations for its cause by measuring attention given to individual data points using a vision science technique called the centroid method. Our work illustrates how ensemble processing mechanisms and mental shortcuts can significantly distort visual summaries of data, and can lead to misjudgments like the demonstrated weighted average illusion.

Impacts of Visualizations on Decoy Effects

The decoy effect is a well-known, intriguing decision-making bias that is often exploited by marketing practitioners to steer consumers towards a desired purchase outcome. It demonstrates that an inclusion of an alternative in the choice set can alter one’s preference among the other choices. Although this decoy effect has been universally observed in the real world and also studied by many economists and psychologists, little is known about how to mitigate the decoy effect and help consumers make informed decisions. In this study, we conducted two experiments: a quantitative experiment with crowdsourcing and a qualitative interview study—first, the crowdsourcing experiment to see if visual interfaces can help alleviate this cognitive bias. Four types of visualizations, one-sided bar chart, two-sided bar charts, scatterplots, and parallel-coordinate plots, were evaluated with four different types of scenarios. The results demonstrated that the two types of bar charts were effective in decreasing the decoy effect. Second, we conducted a semi-structured interview to gain a deeper understanding of the decision-making strategies while making a choice. We believe that the results have an implication on showing how visualizations can have an impact on the decision-making process in our everyday life.

The elusiveness of context effects in decision making

Contextual features influence human and non-human decision making, giving rise to preference reversals. Decades of research have documented the species and situations in which these effects are observed. More recently, however, researchers have focused on boundary conditions, that is, settings in which established effects disappear or reverse. This work is scattered across academic disciplines and some results appear to contradict each other. We synthesize recent findings and resolve apparent contradictions by considering them in terms of three core categories of decision context: spatial arrangement, attribute concreteness, and deliberation time. We suggest that these categories could be understood using theories of choice representation, which specify how context shapes the information over which deliberation processes operate.

The Curse of Knowledge in Visual Data Communication

A viewer can extract many potential patterns from any set of visualized data values. But that means that two people can see different patterns in the same visualization, potentially leading to miscommunication. Here, we show that when people are primed to see one pattern in the data as visually salient, they believe that naïve viewers will experience the same visual salience. Participants were told one of multiple backstories about political events that affected public polling data, before viewing a graph that depicted those data. One pattern in the data was particularly visually salient to them given the backstory that they heard. They then predicted what naïve viewers would most visually salient on the visualization. They were strongly influenced by their own knowledge, despite explicit instructions to ignore it, predicting that others would find the same patterns to be most visually salient. This result reflects a psychological phenomenon known as the curse of knowledge, where an expert struggles to re-create the state of mind of a novice. The present findings show that the curse of knowledge also plagues the visual perception of data, explaining why people can fail to connect with audiences when they communicate patterns in data.

Attraction comes from many sources: Attentional and comparative processes in decoy effects

The attraction effect emerges when adding a seemingly irrelevant option (decoy) to a binary choice shifts preference towards a target option. This suggests that choice behaviour is dynamic, i.e., choice values are developed during deliberation, rather than manifesting some pre-existing preference set. Whereas several models of multialternative and multiattribute decision making consider dynamic choice processes as crucial to explain the attraction effect, empirically investigating the exact nature of such processes requires complementing choice output with other data. In this study, we focused on asymmetrically dominated decoys (i.e., decoys that are clearly dominated only by the target option) to examine the attentional and comparative processes responsible for the attraction effect. Through an eye-tracker paradigm, we showed that the decoy option can affect subjects’ preferences in two different and not mutually exclusive ways: by focusing the attention on the salient option and the dominance attribute, and by increasing comparisons with the choice dominant pattern. Although conceptually and procedurally distinct, both pathways for decoy effects produce an increase in preferences for the target option, in line with attentional and dynamic models of decision making. Eye-tracking data provide further details to the verification of such models, by highlighting the context-dependent nature of attention and the development of similarity-driven competitive decisional processes.

A Task-Based Taxonomy of Cognitive Biases for Information Visualization

Information visualization designers strive to design data displays that allow for efficient exploration, analysis, and communication of patterns in data, leading to informed decisions. Unfortunately, human judgment and decision making are imperfect and often plagued by cognitive biases. There is limited empirical research documenting how these biases affect visual data analysis activities. Existing taxonomies are organized by cognitive theories that are hard to associate with visualization tasks. Based on a survey of the literature we propose a task-based taxonomy of 154 cognitive biases organized in 7 main categories. We hope the taxonomy will help visualization researchers relate their design to the corresponding possible biases, and lead to new research that detects and addresses biased judgment and decision making in data visualization.

What is Interaction for Data Visualization?

Interaction is fundamental to data visualization, but what "interaction" means in the context of visualization is ambiguous and confusing. We argue that this confusion is due to a lack of consensual definition. To tackle this problem, we start by synthesizing an inclusive view of interaction in the visualization community - including insights from information visualization, visual analytics and scientific visualization, as well as the input of both senior and junior visualization researchers. Once this view takes shape, we look at how interaction is defined in the field of human-computer interaction (HCI). By extracting commonalities and differences between the views of interaction in visualization and in HCI, we synthesize a definition of interaction for visualization. Our definition is meant to be a thinking tool and inspire novel and bolder interaction design practices. We hope that by better understanding what interaction in visualization is and what it can be, we will enrich the quality of interaction in visualization systems and empower those who use them.