Spotlights and Soundscapes

Exploring Audio Interfaces for Vertical Guidance in Augmented Reality via Hand-Based Feedback

This research proposes an evaluation of pitch-based sonification methods via user experiments in real-life scenarios, specifically vertical guidance, with the aim of standardizing the use of audio interfaces in AR in guidance tasks. Using literature on assistive technology for people who are blind or visually impaired, we aim to generalize their applicability to a broader population and for different use cases. We propose and test sonification methods for vertical guidance in a series of hand-navigation assessments with users without visual feedback. Including feedback from a visually impaired expert in digital accessibility, results (N=19) outlined that methods that do not rely on memorizing pitch had the most promising accuracy and self-reported workload performances. Ultimately, we argue for audio AR's ability to enhance user performance in different scenarios, from video games to finding objects in a pantry.

Incorporating Situation Awareness Cues in Virtual Reality for Users in Dynamic in-Vehicle Environments

The increasing ubiquity and mobility of virtual reality (VR) devices has introduced novel use cases, one of which is using VR in vehicles, both human-driven and fully automated. However, the effects of the adoption of VR-in-the-car on user task performance, safety, trust, and perceived risk are still largely unknown or not fully understood. Blocking out the physical world and substituting it with a virtual environment has many potential benefits including fewer distractions and greater productivity. However, one shortcoming of this seclusion is losing situation awareness which becomes critical in dynamic, in-vehicle environments, even when the user is not in the driver's seat. Hence, this study aims to understand the effects of providing VR users with situation awareness cues about the real world, when riding in a human-driven or a fully automated car. The results of this driving simulator experiment provide valuable insights into passengers' experience and their information needs while immersed in VR environments. Identifying passengers' unique challenges and needs, as well as developing solutions for them, is expected to improve users' travel experience towards a wider adoption of VR devices.

Beyond the Cane: Describing Urban Scenes to Blind People for Mobility Tasks

Blind people face difficulties with independent mobility, impacting employment prospects, social inclusion, and quality of life. Given the advancements in computer vision, with more efficient and effective automated information extraction from visual scenes, it is important to determine what information is worth conveying to blind travelers, especially since people have a limited capacity to receive and process sensory information. We aimed to investigate which objects in a street scene are useful to describe and how those objects should be described. Thirteen cane-using participants, five of whom were early blind, took part in two urban walking experiments. In the first experiment, participants were asked to voice their information needs in the form of questions to the experimenter. In the second experiment, participants were asked to score scene descriptions and navigation instructions, provided by the experimenter, in terms of their usefulness. The descriptions included a variety of objects with various annotations per object. Additionally, we asked participants to rank order the objects and the different descriptions per object in terms of priority and explain why the provided information is or is not useful to them. The results reveal differences between early and late blind participants. Late blind participants requested information more frequently and prioritized information about objects’ locations. Our results illustrate how different factors, such as the level of detail, relative position, and what type of information is provided when describing an object, affected the usefulness of scene descriptions. Participants explained how they (indirectly) used information, but they were frequently unable to explain their ratings. The results distinguish between various types of travel information, underscore the importance of featuring these types at multiple levels of abstraction, and highlight gaps in current understanding of travel information needs. Elucidating the information needs of blind travelers is critical for the development of more useful assistive technologies.

A Participatory Design Approach to Creating Echolocation-Enabled Virtual Environments

As virtual environments—in the form of videogames and augmented and virtual reality experiences—become more popular, it is important to ensure that they are accessible to all. Previous research has identified echolocation as a useful interaction approach to enable people with visual impairment to access virtual environments. In this paper, we further investigate the usefulness of echolocation to explore virtual environments. We follow a participatory design approach that comprised a focus group session coupled with two fast prototyping and evaluation iterations. During the focus group session, expert echolocators produced a series of seven design recommendations, of which we implemented and trialed four. Our trials revealed that the use of ambient sounds, the ability to place landmarks, directional control, and the ability to use pre-recorded mouth-clicks produced by expert echolocators improved the overall experience of our participants by facilitating the detection of openings and obstacles. The recommendations presented and evaluated in this paper may help to develop virtual environments that support a broader range of users while recognising the value of the lived experience of people with disability as a source of knowledge.

Towards Understanding Sensory Substitution for Accessible Visualization: An Interview Study

For all its potential in supporting data analysis, particularly in exploratory situations, visualization also creates barriers: accessibility for blind and visually impaired individuals. Regardless of how effective a visualization is, providing equal access for blind users requires a paradigm shift for the visualization research community. To enact such a shift, it is not sufficient to treat visualization accessibility as merely another technical problem to overcome. Instead, supporting the millions of blind and visually impaired users around the world who have equally valid needs for data analysis as sighted individuals requires a respectful, equitable, and holistic approach that includes all users from the onset. In this paper, we draw on accessibility research methodologies to make inroads towards such an approach. We first identify the people who have specific insight into how blind people perceive the world: orientation and mobility (O&M) experts, who are instructors that teach blind individuals how to navigate the physical world using non-visual senses. We interview 10 O&M experts-all of them blind-to understand how best to use sensory substitution other than the visual sense for conveying spatial layouts. Finally, we investigate our qualitative findings using thematic analysis. While blind people in general tend to use both sound and touch to understand their surroundings, we focused on auditory affordances and how they can be used to make data visualizations accessible-using sonification and auralization. However, our experts recommended supporting a combination of senses-sound and touch-to make charts accessible as blind individuals may be more familiar with exploring tactile charts. We report results on both sound and touch affordances, and conclude by discussing implications for accessible visualization for blind individuals.