Mapping the Sensory-Perceptual Space of Vibration for User-Centered Intuitive Tactile Design

A Simulation and Training Platform for Remote-Sighted Assistance

Remote-sighted assistance (RSA) is a technology designed to provide assistance for visually impaired people (VIPs). In this scene, a remote-sighted agent communicates and sends commands to navigate and assist VIPs via real-time video sent back. However, the latency in real-time video and the deviation in the execution of instructions by VIPs are two important factors that affect the performance of agents to guide them. Therefore, how to enable agents to better guide VIPs under conditions of video transmission latency and deviation in instruction execution is an important issue. In this paper, we utilize Unreal Engine to create a virtual training platform for RSA, which simulates VIPs executing instructions in the real world and resembles the environment in RSA systems. We aim to help remote-sighted agents quickly master the set of vibration commands formed after encoding tactile vibrations and enable them to guide VIPs more effectively. Our experiment results show that, compared with untrained novices, when guiding people through the same path, agents trained on this platform reduce their average time by 32.09% and their average number of contacts with the environment by 57.57%. Our work provides agents with a simple and convenient simulation and training platform designed to enhance their performance by guiding VIPs with less travel time and fewer environmental contacts. Through this platform, agents can more effectively assist the visually impaired.

Cortical processes of multisensory plausibility modulation of vibrotactile perception in virtual environments in middled-aged and older adults

Digital technologies, such as virtual or augmented reality, can potentially support neurocognitive functions of the aging populations worldwide and complement existing intervention methods. However, aging-related declines in the frontal-parietal network and dopaminergic modulation which progress gradually across the later periods of the adult lifespan may affect the processing of multisensory congruence and expectancy based contextual plausibility. We assessed hemodynamic brain responses while middle-aged and old adults experienced car-riding virtual-reality scenarios where the plausibility of vibrotactile stimulations was manipulated by delivering stimulus intensities that were either congruent or incongruent with the digitalized audio-visual contexts of the respective scenarios. Relative to previous findings observed in young adults, although highly plausible vibrotactile stimulations confirming with contextual expectations also elicited higher brain hemodynamic responses in middle-aged and old adults, this effect was limited to virtual scenarios with extreme expectancy violations. Moreover, individual differences in plausibility-related frontal activity did not correlate with plausibility violation costs in the sensorimotor cortex, indicating less systematic frontal context-based sensory filtering in older ages. These findings have practical implications for advancing digital technologies to support aging societies.

Phantom Illusion Based Vibrotactile Rendering of Affective Touch Patterns

Physically accurate (authentic) reproduction of affective touch patterns on the forearm is limited by actuator technology. However, in most VR applications a direct comparison with actual touch is not possible. Here, the plausibility is only compared to the user's expectation. Focusing on the approach of plausible instead of authentic touch reproduction enables new rendering techniques, like the utilization of the phantom illusion to create the sensation of moving vibrations. Following this idea, a haptic armband array (4x2 vibrational actuators) was built to investigate the possibilities of recreating plausible affective touch patterns with vibration. The novel aspect of this work is the approach of touch reproduction with a parameterized rendering strategy, enabling the integration in VR. A first user study evaluates suitable parameter ranges for vibrational touch rendering. Duration of vibration and signal shape influence plausibility the most. A second user study found high plausibility ratings in a multimodal scenario and confirmed the expressiveness of the system. Rendering device and strategy are suitable for a various stroking patterns and applicable for emerging research on social affective touch reproduction.

A Perceptual Model-Based Approach to Plausible Authoring of Vibration for the Haptic Metaverse

Haptic virtual reality is often misunderstood as being solely a physically identical copy of real environments. Thus, a perfect recording and reproduction of vibration that is indistinguishable in an A:B comparison is often the aim. However, in most virtual reality applications the real environment is not available for direct comparison. Instead, when judging the plausibility of a presented scene, the user compares the vibration to his expectations shaped by the audiovisual context. Therefore, it should be sufficient to find any vibration that the user expects to potentially occur in the given context. Such a vibration needs to elicit a perceptual profile with a minimal distance to an expected profile in the sensory tactile perceptual space. Building onto this formalization, this article demonstrates a novel generative model-based approach to authoring vibrations. First, users quantify expectations as tactile profiles consisting of ratings of six sensory tactile attributes without the presence of vibrations. Subsequently, the model predicts vibration parameters from such profiles. This ensures the fulfillment of user expectations and thus high plausibility. Furthermore, it eliminates the necessity of recordings, infeasible for scenes with no real counterpart and opens the door to crowdsourcing the authoring process with laypersons for the haptic metaverse.

Congruence-based contextual plausibility modulates cortical activity during vibrotactile perception in virtual multisensory environments

How congruence cues and congruence-based expectations may together shape perception in virtual reality (VR) still need to be unravelled. We linked the concept of plausibility used in VR research with congruence-based modulation by assessing brain responses while participants experienced vehicle riding experiences in VR scenarios. Perceptual plausibility was manipulated by sensory congruence, with multisensory stimulations confirming with common expectations of road scenes being plausible. We hypothesized that plausible scenarios would elicit greater cortical responses. The results showed that: (i) vibrotactile stimulations at expected intensities, given embedded audio-visual information, engaged greater cortical activities in frontal and sensorimotor regions; (ii) weaker plausible stimulations resulted in greater responses in the sensorimotor cortex than stronger but implausible stimulations; (iii) frontal activities under plausible scenarios negatively correlated with plausibility violation costs in the sensorimotor cortex. These results potentially indicate frontal regulation of sensory processing and extend previous evidence of contextual modulation to the tactile sense.

Perceptual Space of Algorithms for Three-to-One Dimensional Reduction of Realistic Vibrations

Haptics researchers often endeavor to deliver realistic vibrotactile feedback through broad-bandwidth actuators; however, these actuators typically generate only single-axis vibrations, not 3D vibrations like those that occur in natural tool-mediated interactions. Several three-to-one (321) dimensional reduction algorithms have thus been developed to combine 3D vibrations into 1D vibrations. Surprisingly, the perceptual quality of 321-converted vibrations has never been comprehensively compared to rendering of the original 3D signals. In this study, we develop a multi-dimensional vibration rendering system using a magnetic levitation haptic interface. We verify the system's ability to generate realistic 3D vibrations recorded in both tapping and dragging interactions with four surfaces. We then conduct a study with 15 participants to measure the perceived dissimilarities between five 321 algorithms (SAZ, SUM, VM, DFT, PCA) and the original recordings. The resulting perceptual space is investigated with multiple regression and Procrustes analysis to unveil the relationship between the physical and perceptual properties of 321-converted vibrations. Surprisingly, we found that participants perceptually discriminated the original 3D vibrations from all tested 1D versions. Overall, our results indicate that spectral, temporal, and directional attributes may all contribute to the perceived similarities of vibration signals.