Designing for Mobile and Immersive Visual Analytics in the Field

Effects of View Layout on Situated Analytics for Multiple-View Representations in Immersive Visualization

Multiple-view (MV) representations enabling multi-perspective exploration of large and complex data are often employed on 2D displays. The technique also shows great potential in addressing complex analytic tasks in immersive visualization. However, although useful, the design space of MV representations in immersive visualization lacks in deep exploration. In this paper, we propose a new perspective to this line of research, by examining the effects of view layout for MV representations on situated analytics. Specifically, we disentangle situated analytics in perspectives of situatedness regarding spatial relationship between visual representations and physical referents, and analytics regarding cross-view data analysis including filtering, refocusing, and connecting tasks. Through an in-depth analysis of existing layout paradigms, we summarize design trade-offs for achieving high situatedness and effective analytics simultaneously. We then distill a list of design requirements for a desired layout that balances situatedness and analytics, and develop a prototype system with an automatic layout adaptation method to fulfill the requirements. The method mainly includes a cylindrical paradigm for egocentric reference frame, and a force-directed method for proper view-view, view-user, and view-referent proximities and high view visibility. We conducted a formal user study that compares layouts by our method with linked and embedded layouts. Quantitative results show that participants finished filtering- and connecting-centered tasks significantly faster with our layouts, and user feedback confirms high usability of the prototype system.

Exploring the SenseMaking Process through Interactions and fNIRS in Immersive Visualization

Theories of cognition inform our decisions when designing human-computer interfaces, and immersive systems enable us to examine these theories. This work explores the sensemaking process in an immersive environment through studying both internal and external user behaviors with a classical visualization problem: a visual comparison and clustering task. We developed an immersive system to perform a user study, collecting user behavior data from different channels: AR HMD for capturing external user interactions, functional near-infrared spectroscopy (fNIRS) for capturing internal neural sequences, and video for references. To examine sensemaking, we assessed how the layout of the interface (planar 2D vs. cylindrical 3D layout) and the challenge level of the task (low vs. high cognitive load) influenced the users' interactions, how these interactions changed over time, and how they influenced task performance. We also developed a visualization system to explore joint patterns among all the data channels. We found that increased interactions and cerebral hemodynamic responses were associated with more accurate performance, especially on cognitively demanding trials. The layout types did not reliably influence interactions or task performance. We discuss how these findings inform the design and evaluation of immersive systems, predict user performance and interaction, and offer theoretical insights about sensemaking from the perspective of embodied and distributed cognition.

Developing the Next Generation of Augmented Reality Games for Pediatric Healthcare: An Open-Source Collaborative Framework Based on ARCore for Implementing Teaching, Training and Monitoring Applications

Augmented Reality (AR) provides an alternative to the traditional forms of interaction between humans and machines, and facilitates the access to certain technologies to groups of people with special needs like children. For instance, in pediatric healthcare, it is important to help children to feel comfortable during medical procedures and tests that may be performed on them. To tackle such an issue with the help of AR-based solutions, this article presents the design, implementation and evaluation of a novel open-source collaborative framework that enables to develop teaching, training, and monitoring pediatric healthcare applications. Specifically, such a framework allows for building collaborative applications and shared experiences for AR devices, providing functionalities for connecting with other AR devices and enabling real-time visualization and simultaneous interaction with virtual objects. Since all the communications involved in AR interactions are handled by AR devices, the proposed collaborative framework is able to operate autonomously through a Local Area Network (LAN), thus requiring no cloud or external servers. In order to demonstrate the potential of the proposed framework, a practical use case application is presented. Such an application has been designed to motivate pediatric patients and to encourage them to increase their physical activity through AR games. The presented games do not require any previous configuration, as they use ARCore automatic surface detection technology. Moreover, the AR mobile gaming framework allows multiple players to engage in the same AR experience, so children can interact and collaborate among them sharing the same AR content. In addition, the proposed AR system provides a remote web application that is able to collect and to visualize data on patient use, aiming to provide healthcare professionals with qualified data about the mobility and mood of their patients through an intuitive and user-friendly web tool. Finally, to determine the performance of the proposed AR system, this article presents its evaluation in terms of latency and processing time. The results show that both times are low enough to provide a good user experience.

Vilar-Montesinos M, Fernández-Caramés TM. Creating the Internet of Augmented Things: An Open-Source Framework to Make IoT Devices and Augmented and Mixed Reality Systems Talk to Each Other

Augmented Reality (AR) and Mixed Reality (MR) devices have evolved significantly in the last years, providing immersive AR/MR experiences that allow users to interact with virtual elements placed on the real-world. However, to make AR/MR devices reach their full potential, it is necessary to go further and let them collaborate with the physical elements around them, including the objects that belong to the Internet of Things (IoT). Unfortunately, AR/MR and IoT devices usually make use of heterogeneous technologies that complicate their intercommunication. Moreover, the implementation of the intercommunication mechanisms requires involving specialized developers with have experience on the necessary technologies. To tackle such problems, this article proposes the use of a framework that makes it easy to integrate AR/MR and IoT devices, allowing them to communicate dynamically and in real time. The presented AR/MR-IoT framework makes use of standard and open-source protocols and tools like MQTT, HTTPS or Node-RED. After detailing the inner workings of the framework, it is illustrated its potential through a practical use case: a smart power socket that can be monitored and controlled through Microsoft HoloLens AR/MR glasses. The performance of such a practical use case is evaluated and it is demonstrated that the proposed framework, under normal operation conditions, enables to respond in less than 100 ms to interaction and data update requests.