Audio-Tactile Rendering: A Review on Technology and Methods to Convey Musical Information through the Sense of Touch

Reducing preferred listening levels in headphones through coherent audiotactile stimulation

Using headphones may expose the listener to potentially harmful levels of sound. This study examines whether introducing tactile vibrations to the listening experience would encourage them to reduce their headphone volume. Fifteen participants adjusted their preferred listening levels for four diverse music tracks under audio-only and audiotactile conditions. Results indicated a significant decrease in preferred audio levels with added tactile stimulation. This effect was particularly significant in songs featuring a strong beat. In contrast, only a minimal effect was observed for genres such as classical music, which typically lack a pronounced beat, at higher vibration intensities. These findings suggest that integrating tactile feedback could be a viable strategy for lowering sound exposure risk.

Recent Developments in Haptic Devices Designed for Hearing-Impaired People: A Literature Review

Haptic devices transmit information to the user, using tactile stimuli to augment or replace sensory input. People with limited sensory abilities, such as vision or hearing can receive supplementary information by relying on them. This review analyses recent developments in haptic devices for deaf and hard-of-hearing individuals by extracting the most relevant information from each of the selected papers. The process of finding relevant literature is detailed using the PRISMA guidelines for literature reviews. In this review, the devices are categorized to better understand the review topic. The categorization results have highlighted several areas of future research into haptic devices for hearing-impaired users. We believe this review may be useful to researchers interested in haptic devices, assistive technologies, and human-computer interaction.

Effect of audio-tactile congruence on vibrotactile music enhancement

Music listening experiences can be enhanced with tactile vibrations. However, it is not known which parameters of the tactile vibration must be congruent with the music to enhance it. Devices that aim to enhance music with tactile vibrations often require coding an acoustic signal into a congruent vibrotactile signal. Therefore, understanding which of these audio-tactile congruences are important is crucial. Participants were presented with a simple sine wave melody through supra-aural headphones and a haptic actuator held between the thumb and forefinger. Incongruent versions of the stimuli were made by randomizing physical parameters of the tactile stimulus independently of the auditory stimulus. Participants were instructed to rate the stimuli against the incongruent stimuli based on preference. It was found making the intensity of the tactile stimulus incongruent with the intensity of the auditory stimulus, as well as misaligning the two modalities in time, had the biggest negative effect on ratings for the melody used. Future vibrotactile music enhancement devices can use time alignment and intensity congruence as a baseline coding strategy, which improved strategies can be tested against.

Customizing and Evaluating Accessible Multisensory Music Experiences with Pre-Verbal Children—A Case Study on the Perception of Musical Haptics Using Participatory Design with Proxies

Research on Accessible Digital Musical Instruments (ADMIs) has highlighted the need for participatory design methods, i.e., to actively include users as co-designers and informants in the design process. However, very little work has explored how pre-verbal children with Profound and Multiple Disabilities (PMLD) can be involved in such processes. In this paper, we apply in-depth qualitative and mixed methodologies in a case study with four students with PMLD. Using Participatory Design with Proxies (PDwP), we assess how these students can be involved in the customization and evaluation of the design of a multisensory music experience intended for a large-scale ADMI. Results from an experiment focused on communication of musical haptics highlighted the diversity in employed interaction strategies used by the children, accessibility limitations of the current multisensory experience design, and the importance of using a multifaceted variety of qualitative and quantitative methods to arrive at more informed conclusions when applying a design with proxies methodology.

Gesture, Music and Computer: The Centro di Sonologia Computazionale at Padova University, a 50-Year History

With the advent of digital technologies, the computer has become a generalized tool for music production. Music can be seen as a creative form of human-human communication via a computer, and therefore, research on human-computer and computer-human interfaces is very important. This paper, for the Sensors Special Issue on 800 Years of Research at Padova University, presents a review of the research in the field of music technologies at Padova University by the Centro di Sonologia Computazionale (CSC), focusing on scientific, technological and musical aspects of interaction between musician and computer and between computer and audience. We discuss input devices for detecting information from gestures or audio signals and rendering systems for audience and user engagement. Moreover, we discuss a multilevel conceptual framework, which allows multimodal expressive content processing and coordination, which is important in art and music. Several paradigmatic musical works that stated new lines of both musical and scientific research are then presented in detail. The preservation of this heritage presents problems very different from those posed by traditional artworks. CSC is actively engaged in proposing new paradigms for the preservation of digital art.

Effects of vibrotactile-enhanced music-based intervention on sensorimotor control capacity in the hand of an aging brain: a pilot feasibility randomized crossover trial

BACKGROUND

Music-based interventions (MBI), using music as a therapeutic medium, has been utilized as a promising strategy for motor relearning and shaping. However, currently, MBI with active performance training is restricted to being extensively applied for patients with various levels of defects in fine motor skills and cognitive functions. Therefore, the integration of vibrotactile stimulation with MBI has been adopted as a motor training strategy intended to enhance motor learning through use of vibration stimuli. The current study was designed to investigate differences in the sensorimotor performance of older adults' hands under baseline, a single session of active MBI, and vibrotactile-enriched MBI conditions.

METHODS

Thirty healthy older adults were recruited and randomized to receive either the single session of 30-min of vibrotactile-enriched MBI or 30-min of active MBI at the beginning of the experiment. After a one-week washout period, they switched their treatment programs and then were assessed to study the training effects of both approaches through measuring precision pinch performance, hand function, and sensory status.

RESULTS

The results of the Pinch-Holding-Up Activity test revealed a statistically significant difference in the FRpeak parameter (F = 14.37, p < 0.001, η2p = 0.507) under the vibrotactile-enriched MBI condition compared to the baseline and active MBI conditions. In addition, significant beneficial effects were found on the results of the barognosis (F = 19.126, p < 0.001, η2p = 0. 577) and roughness differentiation subtests (F = 15.036, p < 0.001, η2p = 0.518) in the Manual Tactile Test for the participants in the vibrotactile-enriched MBI group. In addition, the participants under both the active MBI and vibrotactile-enriched MBI conditions exhibited better performance in the three subtests of the Purdue Pegboard Test as compared to under the baseline condition (p < 0.016).

CONCLUSIONS

The findings indicated that vibrotactile-enriched MBI potentially improves the precision pinch performance of hands in healthy older adults. In addition, the add-on effect of vibrotactile stimulation to the MBI condition provides beneficial effects on the sensory functions of the upper extremities.

TRIAL REGISTRATION

NCT04802564 . Date of registration: 15/03/2021. The first posted date: 17/03/2021.