The percept of reverberation is not affected by visual room impression in virtual environments

Investigating the Impact of Visual Input on Voice Production in Virtual Reality

OBJECTIVE

This study explored the independent effects of visual input on voice production using virtual reality. Specifically, its effects on acoustic voice parameters and vocal status ratings, with the hypothesis that larger (a virtual lecture hall and theater compared to an office) and more full virtual rooms (75% of the capacity compared to 45%) would result in changes to the voice parameters and vocal status ratings.

METHODS

Voice production from 30 vocally healthy participants was recorded in six virtual reality conditions. After each condition, the participants provided vocal status ratings. The voice recordings were processed to calculate mean and standard deviation of sound pressure level and fundamental frequency, mean pitch strength, time dose, and cepstral peak prominence smoothed. The effects of the virtual reality conditions on these voice acoustic parameters and the vocal status ratings were analyzed.

RESULTS

The full virtual reality rooms resulted in significantly higher vocal fatigue and vocal discomfort ratings. The larger virtual reality rooms were significantly related to increases in mean and the standard deviation of sound pressure level, mean pitch strength, and cepstral peak prominence smoothed.

CONCLUSIONS

This study demonstrated that visual size and visual fullness input during speech tasks have distinct effects on voice production and self-reported vocal status. Visual size is related to voice acoustic outcomes, while visual fullness is related to self-reported outcomes.

Auditory orientation and distance estimation of sighted humans using virtual echolocation with artificial and self-generated sounds

Active echolocation of sighted humans using predefined synthetic and self-emitted sounds, as habitually used by blind individuals, was investigated. Using virtual acoustics, distance estimation and directional localization of a wall in different rooms were assessed. A virtual source was attached to either the head or hand with realistic or increased source directivity. A control condition was tested with a virtual sound source located at the wall. Untrained echolocation performance comparable to performance in the control condition was achieved on an individual level. On average, the echolocation performance was considerably lower than in the control condition, however, it benefitted from increased directivity.

Separate effects of auditory and visual room size on auditorium seat preference: a virtual reality study

Audiovisual integrations and interactions happen everywhere, including in music concerts, where combined visual and auditory perception contributes to overall enjoyment. Thirty-three participants evaluated their overall subjective preference at various seats in four virtual auditoria, which comprised congruent and incongruent auditory and visual renders of two auditoria that differ only in size. Results show no significant difference between participants who completed the experiment in a fully calibrated and standardized laboratory environment and participants who completed remotely using various VR equipment in various environments. Both visual and auditory auditorium size have significant main effects, but no interaction. The larger hall is preferred for both conditions. Audiovisual congruency does not significantly affect preference.

Auditory spatial analysis in reverberant multi-talker environments with congruent and incongruent audio-visual room information

In a multi-talker situation, listeners have the challenge of identifying a target speech source out of a mixture of interfering background noises. In the current study, it was investigated how listeners analyze audio-visual scenes with varying complexity in terms of number of talkers and reverberation. The visual information of the room was either congruent with the acoustic room or incongruent. The listeners' task was to locate an ongoing speech source in a mixture of other speech sources. The three-dimensional audio-visual scenarios were presented using a loudspeaker array and virtual reality glasses. It was shown that room reverberation, as well as the number of talkers in a scene, influence the ability to analyze an auditory scene in terms of accuracy and response time. Incongruent visual information of the room did not affect this ability. When few talkers were presented simultaneously, listeners were able to detect a target talker quickly and accurately even in adverse room acoustical conditions. Reverberation started to affect the response time when four or more talkers were presented. The number of talkers became a significant factor for five or more simultaneous talkers.

Best Distance Perception in Virtual Audiovisual Environment

Auditory distance perception is important to virtual sound production and is affected by many factors. In order to investigate the effect of the visual cue on auditory distance perception and the best auditory distance perception in the existence of virtual sound source, experiments on the effect of visual cue on distance perception and the best auditory distance perception were conducted, respectively. The results of the first experiment showed that there was no obvious difference between the auditory distance perception in the existence and absence of virtual sound source, but visual cue can decrease the fluctuation of the perception. From the best auditory distance perception experiment, the attenuating SPL of the initial sound signal that made the subjects perceive the best auditory distance in the existence of virtual sound source at the distance of 4m, 6m, 8m, 10m, and 12m was measured, respectively, and compared the difference of attenuating SPL between experiment measurement and theoretical calculation.

Perceptual Matching of Room Acoustics for Auditory Augmented Reality in Small Rooms - Literature Review and Theoretical Framework

For the realization of auditory augmented reality (AAR), it is important that the room acoustical properties of the virtual elements are perceived in agreement with the acoustics of the actual environment. This perceptual matching of room acoustics is the subject reviewed in this paper. Realizations of AAR that fulfill the listeners’ expectations were achieved based on pre-characterization of the room acoustics, for example, by measuring acoustic impulse responses or creating detailed room models for acoustic simulations. For future applications, the goal is to realize an online adaptation in (close to) real-time. Perfect physical matching is hard to achieve with these practical constraints. For this reason, an understanding of the essential psychoacoustic cues is of interest and will help to explore options for simplifications. This paper reviews a broad selection of previous studies and derives a theoretical framework to examine possibilities for psychoacoustical optimization of room acoustical matching.

Spatial Resolution of Late Reverberation in Virtual Acoustic Environments

Late reverberation involves the superposition of many sound reflections, approaching the properties of a diffuse sound field. Since the spatially resolved perception of individual late reflections is impossible, simplifications can potentially be made for modelling late reverberation in room acoustics simulations with reduced spatial resolution. Such simplifications are desired for interactive, real-time virtual acoustic environments with applications in hearing research and for the evaluation of hearing supportive devices. In this context, the number and spatial arrangement of loudspeakers used for playback additionally affect spatial resolution. The current study assessed the minimum number of spatially evenly distributed virtual late reverberation sources required to perceptually approximate spatially highly resolved isotropic and anisotropic late reverberation and to technically approximate a spherically isotropic sound field. The spatial resolution of the rendering was systematically reduced by using subsets of the loudspeakers of an 86-channel spherical loudspeaker array in an anechoic chamber, onto which virtual reverberation sources were mapped using vector base amplitude panning. It was tested whether listeners can distinguish lower spatial resolutions of reproduction of late reverberation from the highest achievable spatial resolution in different simulated rooms. The rendering of early reflections remained unchanged. The coherence of the sound field across a pair of microphones at ear and behind-the-ear hearing device distance was assessed to separate the effects of number of virtual sources and loudspeaker array geometry. Results show that between 12 and 24 reverberation sources are required for the rendering of late reverberation in virtual acoustic environments.