Bridging from Goals to Tasks with Design Study Analysis Reports

Precise Embodied Data Selection with Haptic Feedback while Retaining Room-Scale Visualisation Context

Room-scale immersive data visualisations provide viewers a wide-scale overview of a large dataset, but to interact precisely with individual data points they typically have to navigate to change their point of view. In traditional screen-based visualisations, focus-and-context techniques allow visualisation users to keep a full dataset in view while making detailed selections. Such techniques have been studied extensively on desktop to allow precise selection within large data sets, but they have not been explored in immersive 3D modalities. In this paper we develop a novel immersive focus-and-context technique based on a "magic portal" metaphor adapted specifically for data visualisation scenarios. An extendable-hand interaction technique is used to place a portal close to the region of interest. The other end of the portal then opens comfortably within the user's physical reach such that they can reach through to precisely select individual data points. Through a controlled study with 12 participants, we find strong evidence that portals reduce overshoots in selection and overall hand trajectory length, reducing arm and shoulder fatigue compared to ranged interaction without the portal. The portals also enable us to use a robot arm to provide haptic feedback for data within the limited volume of the portal region. In a second study with another 12 participants we found that haptics provided a positive experience (qualitative feedback) but did not significantly reduce fatigue. We demonstrate applications for portal-based selection through two use-case scenarios.

From Instruction to Insight: Exploring the Functional and Semantic Roles of Text in Interactive Dashboards

There is increased interest in understanding the interplay between text and visuals in the field of data visualization. However, this attention has predominantly been on the use of text in standalone visualizations (such as text annotation overlays) or augmenting text stories supported by a series of independent views. In this paper, we shift from the traditional focus on single-chart annotations to characterize the nuanced but crucial communication role of text in the complex environment of interactive dashboards. Through a survey and analysis of 190 dashboards in the wild, plus 13 expert interview sessions with experienced dashboard authors, we highlight the distinctive nature of text as an integral component of the dashboard experience, while delving into the categories, semantic levels, and functional roles of text, and exploring how these text elements are coalesced by dashboard authors to guide and inform dashboard users. Our contributions are threefold. First, we distill qualitative and quantitative findings from our studies to characterize current practices of text use in dashboards, including a categorization of text-based components and design patterns. Second, we leverage current practices and existing literature to propose, discuss, and validate recommended practices for text in dashboards, embodied as a set of 12 heuristics that underscore the semantic and functional role of text in offering navigational cues, contextualizing data insights, supporting reading order, among other concerns. Third, we reflect on our findings to identify gaps and propose opportunities for data visualization researchers to push the boundaries on text usage for dashboards, from authoring support and interactivity to text generation and content personalization. Our research underscores the significance of elevating text as a first-class citizen in data visualization, and the need to support the inclusion of textual components and their interactive affordances in dashboard design.

A Multi-Level Task Framework for Event Sequence Analysis

Despite the development of numerous visual analytics tools for event sequence data across various domains, including but not limited to healthcare, digital marketing, and user behavior analysis, comparing these domain-specific investigations and transferring the results to new datasets and problem areas remain challenging. Task abstractions can help us go beyond domain-specific details, but existing visualization task abstractions are insufficient for event sequence visual analytics because they primarily focus on multivariate datasets and often overlook automated analytical techniques. To address this gap, we propose a domain-agnostic multi-level task framework for event sequence analytics, derived from an analysis of 58 papers that present event sequence visualization systems. Our framework consists of four levels: objective, intent, strategy, and technique. Overall objectives identify the main goals of analysis. Intents comprises five high-level approaches adopted at each analysis step: augment data, simplify data, configure data, configure visualization, and manage provenance. Each intent is accomplished through a number of strategies, for instance, data simplification can be achieved through aggregation, summarization, or segmentation. Finally, each strategy can be implemented by a set of techniques depending on the input and output components. We further show that each technique can be expressed through a quartet of action-input-output-criteria. We demonstrate the framework's descriptive power through case studies and discuss its similarities and differences with previous event sequence task taxonomies.

A dynamic approach for visualizing and exploring concept hierarchies from textbooks

In this study, we propose a visualization technique to explore and visualize concept hierarchies generated from a textbook in the legal domain. Through a human-centered design process, we developed a tool that allows users to effectively navigate through and explore complex hierarchical concepts in three kinds of traversal techniques: top-down, middle-out, and bottom-up. Our concept hierarchies offer an overview over a given domain, with increasing level of detail toward the bottom of the hierarchy which is consisting of entities. In the legal use case we considered, the concepts were adapted from section headings in a legal textbook, whereas references to law or legal cases inside the textbook became entities. The design of this tool is refined following various steps such as gathering user needs, pain points of an existing visualization, prototyping, testing, and refining. The resulting interface offers users several key features such as dynamic search and filter, explorable concept nodes, and a preview of leaf nodes at every stage. A high-fidelity prototype was created to test our theory and design. To test our concept, we used the System Usability Scale as a way to measure the prototype's usability, a task-based survey to asses the tool's ability in assisting users in gathering information and interacting with the prototype, and finally mouse tracking to understand user interaction patterns. Along with this, we gathered audio and video footage of users when participating in the study. This footage also helped us in getting feedback when the survey responses required further information. The data collected provided valuable insights to set the directions for extending this study. As a result, we have accounted for varying hierarchy depths, longer text spans than only one to two words in the elements of the hierarchy, searchability, and exploration of the hierarchies. At the same time, we aimed for minimizing visual clutter and cognitive overload. We show that existing approaches are not suitable to visualize the type of data which we support with our visualization.

Exploring Interactions with Printed Data Visualizations in Augmented Reality

This paper presents a design space of interaction techniques to engage with visualizations that are printed on paper and augmented through Augmented Reality. Paper sheets are widely used to deploy visualizations and provide a rich set of tangible affordances for interactions, such as touch, folding, tilting, or stacking. At the same time, augmented reality can dynamically update visualization content to provide commands such as pan, zoom, filter, or detail on demand. This paper is the first to provide a structured approach to mapping possible actions with the paper to interaction commands. This design space and the findings of a controlled user study have implications for future designs of augmented reality systems involving paper sheets and visualizations. Through workshops ( N=20) and ideation, we identified 81 interactions that we classify in three dimensions: 1) commands that can be supported by an interaction, 2) the specific parameters provided by an (inter)action with paper, and 3) the number of paper sheets involved in an interaction. We tested user preference and viability of 11 of these interactions with a prototype implementation in a controlled study ( N=12, HoloLens 2) and found that most of the interactions are intuitive and engaging to use. We summarized interactions (e.g., tilt to pan) that have strong affordance to complement "point" for data exploration, physical limitations and properties of paper as a medium, cases requiring redundancy and shortcuts, and other implications for design.

In Defence of Visual Analytics Systems: Replies to Critics

The last decade has witnessed many visual analytics (VA) systems that make successful applications to wide-ranging domains like urban analytics and explainable AI. However, their research rigor and contributions have been extensively challenged within the visualization community. We come in defence of VA systems by contributing two interview studies for gathering critics and responses to those criticisms. First, we interview 24 researchers to collect criticisms the review comments on their VA work. Through an iterative coding and refinement process, the interview feedback is summarized into a list of 36 common criticisms. Second, we interview 17 researchers to validate our list and collect their responses, thereby discussing implications for defending and improving the scientific values and rigor of VA systems. We highlight that the presented knowledge is deep, extensive, but also imperfect, provocative, and controversial, and thus recommend reading with an inclusive and critical eye. We hope our work can provide thoughts and foundations for conducting VA research and spark discussions to promote the research field forward more rigorously and vibrantly.

SUBPLEX: A Visual Analytics Approach to Understand Local Model Explanations at the Subpopulation Level

Understanding the interpretation of machine learning (ML) models has been of paramount importance when making decisions with societal impacts, such as transport control, financial activities, and medical diagnosis. While local explanation techniques are popular methods to interpret ML models on a single instance, they do not scale to the understanding of a model's behavior on the whole dataset. In this article, we outline the challenges and needs of visually analyzing local explanations and propose SUBPLEX, a visual analytics approach to help users understand local explanations with subpopulation visual analysis. SUBPLEX provides steerable clustering and projection visualization techniques that allow users to derive interpretable subpopulations of local explanations with users' expertise. We evaluate our approach through two use cases and experts' feedback.

CcNav: Understanding Compiler Optimizations in Binary Code

Program developers spend significant time on optimizing and tuning programs. During this iterative process, they apply optimizations, analyze the resulting code, and modify the compilation until they are satisfied. Understanding what the compiler did with the code is crucial to this process but is very time-consuming and labor-intensive. Users need to navigate through thousands of lines of binary code and correlate it to source code concepts to understand the results of the compilation and to identify optimizations. We present a design study in collaboration with program developers and performance analysts. Our collaborators work with various artifacts related to the program such as binary code, source code, control flow graphs, and call graphs. Through interviews, feedback, and pair-analytics sessions, we analyzed their tasks and workflow. Based on this task analysis and through a human-centric design process, we designed a visual analytics system Compilation Navigator (CcNav) to aid exploration of the effects of compiler optimizations on the program. CcNav provides a streamlined workflow and a unified context that integrates disparate artifacts. CcNav supports consistent interactions across all the artifacts making it easy to correlate binary code with source code concepts. CcNav enables users to navigate and filter large binary code to identify and summarize optimizations such as inlining, vectorization, loop unrolling, and code hoisting. We evaluate CcNav through guided sessions and semi-structured interviews. We reflect on our design process, particularly the immersive elements, and on the transferability of design studies through our experience with a previous design study on program analysis.

Towards Modeling Visualization Processes as Dynamic Bayesian Networks

Visualization designs typically need to be evaluated with user studies, because their suitability for a particular task is hard to predict. What the field of visualization is currently lacking are theories and models that can be used to explain why certain designs work and others do not. This paper outlines a general framework for modeling visualization processes that can serve as the first step towards such a theory. It surveys related research in mathematical and computational psychology and argues for the use of dynamic Bayesian networks to describe these time-dependent, probabilistic processes. It is discussed how these models could be used to aid in design evaluation. The development of concrete models will be a long process. Thus, the paper outlines a research program sketching how to develop prototypes and their extensions from existing models, controlled experiments, and observational studies.

A Meta-Model Integration for Supporting Knowledge Discovery in Specific Domains: A Case Study in Healthcare

Knowledge management is one of the key priorities of many organizations. They face different challenges in the implementation of knowledge management processes, including the transformation of tacit knowledge-experience, skills, insights, intuition, judgment and know-how-into explicit knowledge. Furthermore, the increasing number of information sources and services in some domains, such as healthcare, increase the amount of information available. Therefore, there is a need to transform that information in knowledge. In this context, learning ecosystems emerge as solutions to support knowledge management in a different context. On the other hand, the dashboards enable the generation of knowledge through the exploitation of the data provided from different sources. The model-driven development of these solutions is possible through two meta-models developed in previous works. Even though those meta-models solve several problems, the learning ecosystem meta-model has a lack of decision-making support. In this context, this work provides two main contributions to face this issue. First, the definition of a holistic meta-model to support decision-making processes in ecosystems focused on knowledge management, also called learning ecosystems. The second contribution of this work is an instantiation of the presented holistic meta-model in the healthcare domain.

Representing Data Visualization Goals and Tasks through Meta-Modeling to Tailor Information Dashboards

Information dashboards are everywhere. They support knowledge discovery in a huge variety of contexts and domains. Although powerful, these tools can be complex, not only for the end-users but also for developers and designers. Information dashboards encode complex datasets into different visual marks to ease knowledge discovery. Choosing a wrong design could compromise the entire dashboard’s effectiveness, selecting the appropriate encoding or configuration for each potential context, user, or data domain is a crucial task. For these reasons, there is a necessity to automatize the recommendation of visualizations and dashboard configurations to deliver tools adapted to their context. Recommendations can be based on different aspects, such as user characteristics, the data domain, or the goals and tasks that will be achieved or carried out through the visualizations. This work presents a dashboard meta-model that abstracts all these factors and the integration of a visualization task taxonomy to account for the different actions that can be performed with information dashboards. This meta-model has been used to design a domain specific language to specify dashboards requirements in a structured way. The ultimate goal is to obtain a dashboard generation pipeline to deliver dashboards adapted to any context, such as the educational context, in which a lot of data are generated, and there are several actors involved (students, teachers, managers, etc.) that would want to reach different insights regarding their learning performance or learning methodologies.

Visualizing a Moving Target: A Design Study on Task Parallel Programs in the Presence of Evolving Data and Concerns

Common pitfalls in visualization projects include lack of data availability and the domain users' needs and focus changing too rapidly for the design process to complete. While it is often prudent to avoid such projects, we argue it can be beneficial to engage them in some cases as the visualization process can help refine data collection, solving a "chicken and egg" problem of having the data and tools to analyze it. We found this to be the case in the domain of task parallel computing where such data and tooling is an open area of research. Despite these hurdles, we conducted a design study. Through a tightly-coupled iterative design process, we built Atria, a multi-view execution graph visualization to support performance analysis. Atria simplifies the initial representation of the execution graph by aggregating nodes as related to their line of code. We deployed Atria on multiple platforms, some requiring design alteration. We describe how we adapted the design study methodology to the "moving target" of both the data and the domain experts' concerns and how this movement kept both the visualization and programming project healthy. We reflect on our process and discuss what factors allow the project to be successful in the presence of changing data and user needs.

A Provenance Task Abstraction Framework

Visual analytics tools integrate provenance recording to externalize analytic processes or user insights. Provenance can be captured on varying levels of detail, and in turn activities can be characterized from different granularities. However, current approaches do not support inferring activities that can only be characterized across multiple levels of provenance. We propose a task abstraction framework that consists of a three stage approach, composed of 1) initializing a provenance task hierarchy, 2) parsing the provenance hierarchy by using an abstraction mapping mechanism, and 3) leveraging the task hierarchy in an analytical tool. Furthermore, we identify implications to accommodate iterative refinement, context, variability, and uncertainty during all stages of the framework. We describe a use case which exemplifies our abstraction framework, demonstrating how context can influence the provenance hierarchy to support analysis. The article concludes with an agenda, raising and discussing challenges that need to be considered for successfully implementing such a framework.

Motion Browser: Visualizing and Understanding Complex Upper Limb Movement Under Obstetrical Brachial Plexus Injuries

The brachial plexus is a complex network of peripheral nerves that enables sensing from and control of the movements of the arms and hand. Nowadays, the coordination between the muscles to generate simple movements is still not well understood, hindering the knowledge of how to best treat patients with this type of peripheral nerve injury. To acquire enough information for medical data analysis, physicians conduct motion analysis assessments with patients to produce a rich dataset of electromyographic signals from multiple muscles recorded with joint movements during real-world tasks. However, tools for the analysis and visualization of the data in a succinct and interpretable manner are currently not available. Without the ability to integrate, compare, and compute multiple data sources in one platform, physicians can only compute simple statistical values to describe patient's behavior vaguely, which limits the possibility to answer clinical questions and generate hypotheses for research. To address this challenge, we have developed MOTION BROWSER, an interactive visual analytics system which provides an efficient framework to extract and compare muscle activity patterns from the patient's limbs and coordinated views to help users analyze muscle signals, motion data, and video information to address different tasks. The system was developed as a result of a collaborative endeavor between computer scientists and orthopedic surgery and rehabilitation physicians. We present case studies showing physicians can utilize the information displayed to understand how individuals coordinate their muscles to initiate appropriate treatment and generate new hypotheses for future research.

IDMVis: Temporal Event Sequence Visualization for Type 1 Diabetes Treatment Decision Support

Type 1 diabetes is a chronic, incurable autoimmune disease affecting millions of Americans in which the body stops producing insulin and blood glucose levels rise. The goal of intensive diabetes management is to lower average blood glucose through frequent adjustments to insulin protocol, diet, and behavior. Manual logs and medical device data are collected by patients, but these multiple sources are presented in disparate visualization designs to the clinician-making temporal inference difficult. We conducted a design study over 18 months with clinicians performing intensive diabetes management. We present a data abstraction and novel hierarchical task abstraction for this domain. We also contribute IDMVis: a visualization tool for temporal event sequences with multidimensional, interrelated data. IDMVis includes a novel technique for folding and aligning records by dual sentinel events and scaling the intermediate timeline. We validate our design decisions based on our domain abstractions, best practices, and through a qualitative evaluation with six clinicians. The results of this study indicate that IDMVis accurately reflects the workflow of clinicians. Using IDMVis, clinicians are able to identify issues of data quality such as missing or conflicting data, reconstruct patient records when data is missing, differentiate between days with different patterns, and promote educational interventions after identifying discrepancies.

Design Exposition with Literate Visualization

We propose a new approach to the visualization design and communication process, literate visualization, based upon and extending, Donald Knuth's idea of literate programming. It integrates the process of writing data visualization code with description of the design choices that led to the implementation (design exposition). We develop a model of design exposition characterised by four visualization designer architypes: the evaluator, the autonomist, the didacticist and the rationalist. The model is used to justify the key characteristics of literate visualization: 'notebook' documents that integrate live coding input, rendered output and textual narrative; low cost of authoring textual narrative; guidelines to encourage structured visualization design and its documentation. We propose narrative schemas for structuring and validating a wide range of visualization design approaches and models, and branching narratives for capturing alternative designs and design views. We describe a new open source literate visualization environment, litvis, based on a declarative interface to Vega and Vega-Lite through the functional programming language Elm combined with markdown for formatted narrative. We informally assess the approach, its implementation and potential by considering three examples spanning a range of design abstractions: new visualization idioms; validation though visualization algebra; and feminist data visualization. We argue that the rich documentation of the design process provided by literate visualization offers the potential to improve the validity of visualization design and so benefit both academic visualization and visualization practice.