What's the Situation with Situated Visualization? A Survey and Perspectives on Situatedness

Formula-S: Situated Visualization for Traditional Chinese Medicine Formula Learning

OBJECTIVES

The study of medicine formulas is a core component of traditional Chinese medicine (TCM), yet traditional learning methods often lack interactivity and contextual understanding, making it challenging for beginners to grasp the intricate composition rules of formulas. To address this gap, we introduce Formula-S, a situated visualization method for TCM formula learning in augmented reality (AR) and evaluate its performance. This study aims to evaluate the effectiveness of Formula-S in enhancing TCM formula learning for beginners by comparing it with traditional text-based formula learning and web-based visualization.

METHODS

Formula-S is an interactive AR tool designed for TCM formula learning, featuring three modes (3D, Web, and Table). The dataset included TCM formulas and herb properties extracted from authoritative references, including textbook and the SymMap database. In Formula-S, the hierarchical visualization of the formulas as herbal medicine compositions, is linked to the multidimensional herb attribute visualization and embedded in the real world, where real herb samples are presented. To evaluate its effectiveness, a controlled study (n=30) was conducted.Participants who had no formal TCM knowledge were tasked with herbal medicine identification, formula composition, and recognition. In the study, participants interacted with the AR tool through HoloLens 2. Data were collected on both task performance (accuracy and response time) and user experience, with a focus on task efficiency, accuracy, and user preference across the different learning modes. Results The situated visualization method of Formula-S had comparable accuracy to other methods but shorter response time for herbal formula learning tasks. Regarding user experience, our new approach demonstrated the highest system usability and lowest task load, effectively reducing cognitive load and allowing users to complete tasks with greater ease and efficiency. Participants reported that Formula-S enhanced their learning experience through its intuitive interface and immersive AR environment, suggesting this approach offers usability advantages for TCM education.

CONCLUSIONS

The situated visualization method in Formula-S offers more efficient and accurate searching capabilities compared to traditional and web-based methods. Additionally, it provides superior contextual understanding of TCM formulas, making it a promising new solution for TCM learning.

User Experience of Visualizations in Motion: A Case Study and Design Considerations

We present a systematic review, an empirical study, and a first set of considerations for designing visualizations in motion, derived from a concrete scenario in which these visualizations were used to support a primary task. In practice, when viewers are confronted with embedded visualizations, they often have to focus on a primary task and can only quickly glance at a visualization showing rich, often dynamically updated, information. As such, the visualizations must be designed so as not to distract from the primary task, while at the same time being readable and useful for aiding the primary task. For example, in games, players who are engaged in a battle have to look at their enemies but also read the remaining health of their own game character from the health bar over their character's head. Many trade-ofts are possible in the design of embedded visualizations in such dynamic scenarios, which we explore in-depth in this paper with a focus on user experience. We use video games as an example of an application context with a rich existing set of visualizations in motion. We begin our work with a systematic review of in-game visualizations in motion. Next, we conduct an empirical user study to investigate how different embedded visualizations in motion designs impact user experience. We conclude with a set of considerations and trade-offs for designing visualizations in motion more broadly as derived from what we learned about video games. All supplemental materials of this paper are available at osf.io/3v8wm/.

"I Came Across a Junk": Understanding Design Flaws of Data Visualization from the Public's Perspective

The visualization community has a rich history of reflecting upon visualization design flaws. Although research in this area has remained lively, we believe it is essential to continuously revisit this classic and critical topic in visualization research by incorporating more empirical evidence from diverse sources, characterizing new design flaws, building more systematic theoretical frameworks, and understanding the underlying reasons for these flaws. To address the above gaps, this work investigated visualization design flaws through the lens of the public, constructed a framework to summarize and categorize the identified flaws, and explored why these flaws occur. Specifically, we analyzed 2227 flawed data visualizations collected from an online gallery and derived a design task-associated taxonomy containing 76 specific design flaws. These flaws were further classified into three high-level categories (i.e., misinformation, uninformativeness, unsociability) and ten subcategories (e.g., inaccuracy, unfairness, ambiguity). Next, we organized five focus groups to explore why these design flaws occur and identified seven causes of the flaws. Finally, we proposed a research agenda for combating visualization design flaws and summarize nine research opportunities.

Situated Visualization in Motion

We define visualization in motion and make several contributions to how to visualize and design situated visualizations in motion. In situated data visualization, the data are directly visualized near their data referent (i.e., the physical space, object, or person it refers to) (Bressa et al., 2022). Situated visualizations are often useful in contexts where the data referent or the viewer does not remain stationary but is in relative motion. For example, a runner looks at visualizations from their fitness band while running. Reading visualizations in such scenarios might be impacted by motion factors. As such, understanding how to best design visualizations with motion factors is important. We define visualizations in motion as visual data representations used in contexts that exhibit relative motion between a viewer and an entire visualization. We propose a research agenda to understand what research opportunities and challenges are under visualization in motion (Yao et al., 2022). Next, we investigate (a) how motion factors can affect the reading accuracy of visualizations (Yao et al., 2022), (b) how to design and embed visualizations in motion in a real application scenario (Yao et al., 2024), and (c) the user experience and design tradeoffs of visualization in motion through a case study (Yao et al., 2024).

Design Patterns for Situated Visualization in Augmented Reality

Situated visualization has become an increasingly popular research area in the visualization community, fueled by advancements in augmented reality (AR) technology and immersive analytics. Visualizing data in spatial proximity to their physical referents affords new design opportunities and considerations not present in traditional visualization, which researchers are now beginning to explore. However, the AR research community has an extensive history of designing graphics that are displayed in highly physical contexts. In this work, we leverage the richness of AR research and apply it to situated visualization. We derive design patterns which summarize common approaches of visualizing data in situ. The design patterns are based on a survey of 293 papers published in the AR and visualization communities, as well as our own expertise. We discuss design dimensions that help to describe both our patterns and previous work in the literature. This discussion is accompanied by several guidelines which explain how to apply the patterns given the constraints imposed by the real world. We conclude by discussing future research directions that will help establish a complete understanding of the design of situated visualization, including the role of interactivity, tasks, and workflows.

Wearable Augmented Reality: Research Trends and Future Directions from Three Major Venues

Wearable Augmented Reality (AR) has attracted considerable attention in recent years, as evidenced by the growing number of research publications and industry investments. With swift advancements and a multitude of interdisciplinary research areas within wearable AR, a comprehensive review is crucial for integrating the current state of the field. In this paper, we present a review of 389 research papers on wearable AR, published between 2018 and 2022 in three major venues: ISMAR, TVCG, and CHI. Drawing inspiration from previous works by Zhou et al. and Kim et al., which summarized AR research at ISMAR over the past two decades (1998-2017), we categorize the papers into different topics and identify prevailing trends. One notable finding is that wearable AR research is increasingly geared towards enabling broader consumer adoption. From our analysis, we highlight key observations related to potential future research areas essential for capitalizing on this trend and achieving widespread adoption. These include addressing challenges in Display, Tracking, Interaction, and Applications, and exploring emerging frontiers in Ethics, Accessibility, Avatar and Embodiment, and Intelligent Virtual Agents.

Sporthesia: Augmenting Sports Videos Using Natural Language

Augmented sports videos, which combine visualizations and video effects to present data in actual scenes, can communicate insights engagingly and thus have been increasingly popular for sports enthusiasts around the world. Yet, creating augmented sports videos remains a challenging task, requiring considerable time and video editing skills. On the other hand, sports insights are often communicated using natural language, such as in commentaries, oral presentations, and articles, but usually lack visual cues. Thus, this work aims to facilitate the creation of augmented sports videos by enabling analysts to directly create visualizations embedded in videos using insights expressed in natural language. To achieve this goal, we propose a three-step approach - 1) detecting visualizable entities in the text, 2) mapping these entities into visualizations, and 3) scheduling these visualizations to play with the video - and analyzed 155 sports video clips and the accompanying commentaries for accomplishing these steps. Informed by our analysis, we have designed and implemented Sporthesia, a proof-of-concept system that takes racket-based sports videos and textual commentaries as the input and outputs augmented videos. We demonstrate Sporthesia's applicability in two exemplar scenarios, i.e., authoring augmented sports videos using text and augmenting historical sports videos based on auditory comments. A technical evaluation shows that Sporthesia achieves high accuracy (F1-score of 0.9) in detecting visualizable entities in the text. An expert evaluation with eight sports analysts suggests high utility, effectiveness, and satisfaction with our language-driven authoring method and provides insights for future improvement and opportunities.

Effects of View Layout on Situated Analytics for Multiple-View Representations in Immersive Visualization

Multiple-view (MV) representations enabling multi-perspective exploration of large and complex data are often employed on 2D displays. The technique also shows great potential in addressing complex analytic tasks in immersive visualization. However, although useful, the design space of MV representations in immersive visualization lacks in deep exploration. In this paper, we propose a new perspective to this line of research, by examining the effects of view layout for MV representations on situated analytics. Specifically, we disentangle situated analytics in perspectives of situatedness regarding spatial relationship between visual representations and physical referents, and analytics regarding cross-view data analysis including filtering, refocusing, and connecting tasks. Through an in-depth analysis of existing layout paradigms, we summarize design trade-offs for achieving high situatedness and effective analytics simultaneously. We then distill a list of design requirements for a desired layout that balances situatedness and analytics, and develop a prototype system with an automatic layout adaptation method to fulfill the requirements. The method mainly includes a cylindrical paradigm for egocentric reference frame, and a force-directed method for proper view-view, view-user, and view-referent proximities and high view visibility. We conducted a formal user study that compares layouts by our method with linked and embedded layouts. Quantitative results show that participants finished filtering- and connecting-centered tasks significantly faster with our layouts, and user feedback confirms high usability of the prototype system.

The Influence of Public Mental Health Based on Artificial Intelligence Technology on the Teaching Effect of Business Administration Major

As an emerging technology, public mental health based on artificial intelligence engineering has broad application and development prospects in improving teaching effects, realizing high efficiency and intelligence, and improving refined education. The application of public mental health based on artificial intelligence engineering in the practice of the teaching effect of business administration has a very positive significance for accurately discovering the mental health and teaching effect of business administration students. For the sake of improving the teaching effect of business administration major, we come up with a method for the influence of public mental health on the teaching effect of business administration profession based on artificial intelligence engineering. Firstly, we collect the video and audio of the teaching of business administration, conducts data screening and cleaning, and sorts out the video and audio synchronization clips as training data. Secondly, we propose a multimodal feature extraction network and a multimodal fusion network for the extraction and fusion of video and audio clips, respectively. Then, the fully connected network structure is used to evaluate and classify the effect of business administration professional teaching, and use mental health factors for evaluation to avoid human intervention and improve the prediction effect. Finally, through intensive experimental results, we prove that the method raised by us can use artificial intelligence engineering to evaluate the teaching effect of business administration majors from the perspective of public mental health and achieve good experimental results.

Visual Art Design of Digital Works Guided by Big Data

With the rapid development of digital technology, the development speed of digital media is also relatively fast. Digital media technology has a great impact on people’s lifestyles and aesthetic concepts, and it also has a greater impact on visual art, creative thinking communication methods, and expression methods. In this study, the quality enhancement of digital images has been intensively studied based on the guidance of big data of eye-movement gaze points. A large amount of visual data are collected from public social resources, and the optimization research of image sensory quality is carried out in-depth using the acquired big data. Next, the region of interest (ROI) is obtained by combining the data with a two-dimensional Gaussian distribution model-fitting method, and the obtained data clustered and improved based on the K-means clustering algorithm to obtain ROI fixation points. Finally, discontinuities in the choice of sharpness in graphics and video playback are pointed out, and the final fixation data analysis is utilized. Results show that targeted optimization is very effective in improving the quality of digital images and saving space, enabling users to enjoy higher-quality visual digital images. The proposed method can be used to improve the dynamic resolution of images and videos.

The Hitchhiker’s Guide to Fused Twins: A Review of Access to Digital Twins In Situ in Smart Cities

Smart Cities already surround us, and yet they are still incomprehensibly far from directly impacting everyday life. While current Smart Cities are often inaccessible, the experience of everyday citizens may be enhanced with a combination of the emerging technologies Digital Twins (DTs) and Situated Analytics. DTs represent their Physical Twin (PT) in the real world via models, simulations, (remotely) sensed data, context awareness, and interactions. However, interaction requires appropriate interfaces to address the complexity of the city. Ultimately, leveraging the potential of Smart Cities requires going beyond assembling the DT to be comprehensive and accessible. Situated Analytics allows for the anchoring of city information in its spatial context. We advance the concept of embedding the DT into the PT through Situated Analytics to form Fused Twins (FTs). This fusion allows access to data in the location that it is generated in in an embodied context that can make the data more understandable. Prototypes of FTs are rapidly emerging from different domains, but Smart Cities represent the context with the most potential for FTs in the future. This paper reviews DTs, Situated Analytics, and Smart Cities as the foundations of FTs. Regarding DTs, we define five components (physical, data, analytical, virtual, and Connection Environments) that we relate to several cognates (i.e., similar but different terms) from existing literature. Regarding Situated Analytics, we review the effects of user embodiment on cognition and cognitive load. Finally, we classify existing partial examples of FTs from the literature and address their construction from Augmented Reality, Geographic Information Systems, Building/City Information Models, and DTs and provide an overview of future directions.

Situated Visual Analysis and Live Monitoring for Manufacturing

Modern machines continuously log status reports over long periods of time, which are valuable data to optimize working routines. Data visualization is a commonly used tool to gain insights into these data, mostly in retrospective (e.g., to determine causal dependencies between the faults of different machines). We present an approach to bring such visual analyses to the shop floor to support reacting to faults in real time. This approach combines spatio-temporal analyses of time series using a handheld touch device with augmented reality for live monitoring. Important information augments machines directly in their real-world context, and detailed logs of current and historical events are displayed on the handheld device. In collaboration with an industry partner, we designed and tested our approach on a live production line to obtain feedback from operators. We compare our approach for monitoring and analysis with existing solutions that are currently deployed.

One View Is Not Enough: Review of and Encouragement for Multiple and Alternative Representations in 3D and Immersive Visualisation

The opportunities for 3D visualisations are huge. People can be immersed inside their data, interface with it in natural ways, and see it in ways that are not possible on a traditional desktop screen. Indeed, 3D visualisations, especially those that are immersed inside head-mounted displays are becoming popular. Much of this growth is driven by the availability, popularity and falling cost of head-mounted displays and other immersive technologies. However, there are also challenges. For example, data visualisation objects can be obscured, important facets missed (perhaps behind the viewer), and the interfaces may be unfamiliar. Some of these challenges are not unique to 3D immersive technologies. Indeed, developers of traditional 2D exploratory visualisation tools would use alternative views, across a multiple coordinated view (MCV) system. Coordinated view interfaces help users explore the richness of the data. For instance, an alphabetical list of people in one view shows everyone in the database, while a map view depicts where they live. Each view provides a different task or purpose. While it is possible to translate some desktop interface techniques into the 3D immersive world, it is not always clear what equivalences would be. In this paper, using several case studies, we discuss the challenges and opportunities for using multiple views in immersive visualisation. Our aim is to provide a set of concepts that will enable developers to perform critical thinking, creative thinking and push the boundaries of what is possible with 3D and immersive visualisation. In summary developers should consider how to integrate many views, techniques and presentation styles, and one view is not enough when using 3D and immersive visualisations.