Toward Objective Evaluation of Working Memory in Visualizations: A Case Study Using Pupillometry and a Dual-Task Paradigm

The Unmet Data Visualization Needs of Decision Makers Within Organizations

When an organization chooses one course of action over alternatives, this task typically falls on a decision maker with relevant knowledge, experience, and understanding of context. Decision makers rely on data analysis, which is either delegated to analysts, or done on their own. Often the decision maker combines data, likely uncertain or incomplete, with non-formalized knowledge within a multi-objective problem space, weighing the recommendations of analysts within broader contexts and goals. As most past research in visual analytics has focused on understanding the needs and challenges of data analysts, less is known about the tasks and challenges of organizational decision makers, and how visualization support tools might help. Here we characterize the decision maker as a domain expert, review relevant literature in management theories, and report the results of an empirical survey and interviews with people who make organizational decisions. We identify challenges and opportunities for novel visualization tools, including trade-off overviews, scenario-based analysis, interrogation tools, flexible data input and collaboration support. Our findings stress the need to expand visualization design beyond data analysis into tools for information management.

Examining Effort in 1D Uncertainty Communication Using Individual Differences in Working Memory and NASA-TLX

As uncertainty visualizations for general audiences become increasingly common, designers must understand the full impact of uncertainty communication techniques on viewers' decision processes. Prior work demonstrates mixed performance outcomes with respect to how individuals make decisions using various visual and textual depictions of uncertainty. Part of the inconsistency across findings may be due to an over-reliance on task accuracy, which cannot, on its own, provide a comprehensive understanding of how uncertainty visualization techniques support reasoning processes. In this work, we advance the debate surrounding the efficacy of modern 1D uncertainty visualizations by conducting converging quantitative and qualitative analyses of both the effort and strategies used by individuals when provided with quantile dotplots, density plots, interval plots, mean plots, and textual descriptions of uncertainty. We utilize two approaches for examining effort across uncertainty communication techniques: a measure of individual differences in working-memory capacity known as an operation span (OSPAN) task and self-reports of perceived workload via the NASA-TLX. The results reveal that both visualization methods and working-memory capacity impact participants' decisions. Specifically, quantile dotplots and density plots (i.e., distributional annotations) result in more accurate judgments than interval plots, textual descriptions of uncertainty, and mean plots (i.e., summary annotations). Additionally, participants' open-ended responses suggest that individuals viewing distributional annotations are more likely to employ a strategy that explicitly incorporates uncertainty into their judgments than those viewing summary annotations. When comparing quantile dotplots to density plots, this work finds that both methods are equally effective for low-working-memory individuals. However, for individuals with high-working-memory capacity, quantile dotplots evoke more accurate responses with less perceived effort. Given these results, we advocate for the inclusion of converging behavioral and subjective workload metrics in addition to accuracy performance to further disambiguate meaningful differences among visualization techniques.

Multiple Hazard Uncertainty Visualization Challenges and Paths Forward

Making decisions with uncertainty is challenging for the general public, policymakers, and even highly trained scientists. Nevertheless, when faced with the need to respond to a potential hazard, people must make high-risk decisions with uncertainty. In some cases, people have to consider multiple hazards with various types of uncertainties. Multiple hazards can be interconnected by location, time, and/or environmental systems, and the hazards may interact, producing complex relationships among their associated uncertainties. The interaction between multiple hazards and their uncertainties can have nonlinear effects, where the resultant risk and uncertainty are greater than the sum of the risk and uncertainty associated with individual hazards. Effectively communicating the uncertainties related to such complicated systems should be a high priority because the frequency and variability of multiple hazard events due to climate change continue to increase. However, the communication of multiple hazard uncertainties and their interactions remains largely unexplored. The lack of practical guidance on conveying multiple hazard uncertainties is likely due in part to the field’s vast expanse, making it challenging to identify entry points. Here, we offer a perspective on three critical challenges related to uncertainty communication across various multiple hazard contexts to galvanize the research community. We advocate for systematic considerations of multiple hazard uncertainty communication that focus on trade-offs between complexity and factors, including mental effort, trust, and usability.

Understanding the Role of Visualizations on Decision Making: A Study on Working Memory

The use of data visualization is increasing; however, there is little empirical explanation for how it supports users. Our goal in this paper is to deepen our understanding of the role of interactive visualizations in a particular context of decision making. Specifically, we attempt to understand the role of the working memory system, which is a concept to understand the mechanism of the processing and temporary storage of information in variety of cognitive tasks. We compared two interfaces, SimulSort and its non-visual counterpart Typical Sorting, with a multi-attribute decision-making problem. Because decision outcomes are known to be affected by the limitations of a person’s working memory, we conducted a crowdsourcing-based user study using SimulSort to understand how working memory, especially the phonological loop, can benefit from the using visualizations. We examined the impact on working memory with a well known dual-task methodology by designing a concurrent task to tap into the main decision-making task. The experiment was conducted with a total of 137 participants and an ordered logistic regression using a proportional odds model was applied to analyze the decision quality. The results supported the hypothesis that when using SimulSort, participants required less working memory than they required with Typical Sorting to accomplish the multi-attribute decision-making task even though SimulSort outperformed Typical Sorting in terms of decision quality. We also provide methodologies to conduct working memory studies by implementing an articulatory suppression task on crowdsourcing platforms in which experimenters have less control over the participants.

Evaluating effectiveness of information visualizations using cognitive fit theory: A neuroergonomics approach

Information visualizations may be evaluated from the perspective of how they match tasks that must be performed with them, a cognitive fit perspective. However, there is a gap between the high-level references made to cognitive fit and the low-level ability to identify and measure it during human interaction with visualizations. We bridge this gap by using an electroencephalography metric derived from frontal midline theta power and parietal alpha power, known as the task load index, to determine if cognitive effort measured at the level of cortical activity is less when cognitive fit is present compared to when cognitive fit is not. We found that when there is cognitive fit between the type of problem to be solved and the information displayed by a system, the task load index is lower compared to when cognitive fit is not present. We support this finding with subjective (NASA task load index) and performance (response time and accuracy) measures. Our approach, using electroencephalography, provides supplemental information to self-report and performance measures. Findings from this study are important because they (1) provide more validity to the cognitive fit theory using a neurophysiological measure, and (2) use the electroencephalography task load index metric as a means to assess cognitive workload and effort in general.