Fidelity of three-dimensional-sound reproduction using a virtual auditory display

Effects of Maturation and Chronological Aging on Auditory Spatial Processing: A Cross-Sectional Study Across Life Span

OBJECTIVE

The primary aim of the research was to document spatial acuity changes across the life span using a battery of psychoacoustical and perceptual tests. The secondary aim was to identify the optimal metric for measuring spatial processing changes across the life span (ages 10-70 years).

DESIGN AND STUDY SAMPLE

A cross-sectional study comprising 115 participants with clinically normal hearing was conducted. Purposive sampling was adopted to recruit participants in the study, who were divided into six groups based on their chronological age.

METHOD

Temporal, intensity, spectral, and composite correlates of spatial acuity were assessed using psychoacoustic measures and perceptual questionnaires. The temporal (interaural time difference [ITD]) and intensity correlates (interaural level difference [ILD]) of spatial perception were obtained using a MATLAB (v 2020a), whereas the composite correlate (virtual auditory space identification scores [VASIs]) and perceptual ratings of spatial processing were measured using Paradigm software and speech spatial and qualities in Kannada (SSQ-K).

RESULTS

Results across all tests (multivariate analyses variance: 6 age groups × 4 tests, followed by post hoc tests) consistently demonstrate poor ITD and ILD thresholds and overall lower spatial accuracy (VASI, SSQ-K) with increasing age. Discriminant function analyses (DFAs) revealed that VASI had a higher predictive power in capturing age-related changes in auditory spatial processing. The group segregation on spatial performance in DFA became evident after 50 years.

CONCLUSION

This study provides evidence of gradual change in all three correlates of spatial processing, with statistically demonstrable deficits appearing from fourth decade of life on VASI and fifth decade of life on binaural processing.

Head-Related Transfer Functions for Dynamic Listeners in Virtual Reality

In dynamic virtual reality, visual cues and motor actions aid auditory perception. With multimodal integration and auditory adaptation effects, generic head-related transfer functions (HRTFs) may yield no significant disadvantage to individual HRTFs regarding accurate auditory perception. This study compares two individual HRTF sets against a generic HRTF set by way of objective analysis and two subjective experiments. First, auditory-model-based predictions examine the objective deviations in localization cues between the sets. Next, the HRTFs are compared in a static subjective (N=8) localization experiment. Finally, the localization accuracy, timbre, and overall quality of the HRTF sets are evaluated subjectively (N=12) in a six-degrees-of-freedom audio-visual virtual environment. The results show statistically significant objective deviations between the sets, but no perceived localization or overall quality differences in the dynamic virtual reality.

Sound Externalization: A Review of Recent Research

Sound externalization, or the perception that a sound source is outside of the head, is an intriguing phenomenon that has long interested psychoacousticians. While previous reviews are available, the past few decades have produced a substantial amount of new data.In this review, we aim to synthesize those data and to summarize advances in our understanding of the phenomenon. We also discuss issues related to the definition and measurement of sound externalization and describe quantitative approaches that have been taken to predict the outcomes of externalization experiments. Last, sound externalization is of practical importance for many kinds of hearing technologies. Here, we touch on two examples, discussing the role of sound externalization in augmented/virtual reality systems and bringing attention to the somewhat overlooked issue of sound externalization in wearers of hearing aids.

Pinna-related transfer functions and lossless wave equation using finite-difference methods: Validation with measurements

Nowadays, wave-based simulations of head-related transfer functions (HRTFs) lack strong justifications to replace HRTF measurements. The main cause is the complex interactions between uncertainties and biases in both simulated and measured HRTFs. This paper deals with the validation of pinna-related high-frequency information in the ipsilateral directions-of-arrival, computed by lossless wave-based simulations with finite-difference models. A simpler yet related problem is given by the pinna-related transfer function (PRTF), which encodes the acoustical effects of only the external ear. Results stress that PRTF measurements are generally highly repeatable but not necessarily easily reproducible, leading to critical issues in terms of reliability for any ground truth condition. On the other hand, PRTF simulations exhibit an increasing uncertainty with frequency and grid-dependent frequency changes, which are here quantified analyzing the benefits in the use of a unique asymptotic solution. In this validation study, the employed finite-difference model accurately and reliably predict the PRTF magnitude mostly within ±1 dB up to ≈8 kHz and a space- and frequency-averaged spectral distortion within about 2 dB up to ≈ 18 kHz.

Pinna-related transfer functions and lossless wave equation using finite-difference methods: Verification and asymptotic solution

A common approach when employing discrete mathematical models is to assess the reliability and credibility of the computation of interest through a process known as solution verification. Present-day computed head-related transfer functions (HRTFs) seem to lack robust and reliable assessments of the numerical errors embedded in the results which makes validation of wave-based models difficult. This process requires a good understanding of the involved sources of error which are systematically reviewed here. The current work aims to quantify the pinna-related high-frequency computational errors in the context of HRTFs and wave-based simulations with finite-difference models. As a prerequisite for solution verification, code verification assesses the reliability of the proposed implementation. In this paper, known and manufactured formal solutions are used and tailored for the wave equation and frequency-independent boundary conditions inside a rectangular room of uniform acoustic wall-impedance. Asymptotic estimates for pinna acoustics are predicted in the frequency domain based on regression models and a convergence study on sub-millimeter grids. Results show an increasing uncertainty with frequency and a significant frequency-dependent change among computations on different grids.

On the authenticity of individual dynamic binaural synthesis

A simulation that is perceptually indistinguishable from the corresponding real sound field could be termed authentic. Using binaural technology, such a simulation would theoretically be achieved by reconstructing the sound pressure at a listener's ears. However, inevitable errors in the measurement, rendering, and reproduction introduce audible degradations, as it has been demonstrated in previous studies for anechoic environments and static binaural simulations (fixed head orientation). The current study investigated the authenticity of individual dynamic binaural simulations for three different acoustic environments (anechoic, dry, wet) using a highly sensitive listening test design. The results show that about half of the participants failed to reliably detect any differences for a speech stimulus, whereas all participants were able to do so for pulsed pink noise. Higher detection rates were observed in the anechoic condition, compared to the reverberant spaces, while the source position had no significant effect. It is concluded that the authenticity mainly depends on how comprehensive the spectral cues are provided by the audio content, and the amount of reverberation, whereas the source position plays a minor role. This is confirmed by a broad qualitative evaluation, suggesting that remaining differences mainly affect the tone color rather than the spatial, temporal or dynamical qualities.

The localization of non-individualized virtual sounds by hearing impaired listeners

Although many studies have evaluated the performance of virtual audio displays with normal hearing listeners, very little information is available on the effect that hearing loss has on the localization of virtual sounds. In this study, normal hearing (NH) and hearing impaired (HI) listeners were asked to localize noise stimuli with short (250 ms), medium (1000 ms), and long (4000 ms) durations both in the free field and with a non-individualized head-tracked virtual audio display. The results show that the HI listeners localized sounds less accurately than the NH listeners, and that both groups consistently localized virtual sounds less accurately than free-field sounds. These results indicate that HI listeners are sensitive to individual differences in head related transfer functions (HRTFs), which means that they might have difficulty using auditory display systems that rely on generic HRTFs to control the apparent locations of virtual sounds. However, the results also reveal a high correlation between free-field and virtual localization performance in the HI listeners. This suggests that it may be feasible to use non-individualized virtual audio display systems to predict the auditory localization performance of HI listeners in clinical environments where free-field speaker arrays are not available.

Efficient HRTF-based Spatial Audio for Area and Volumetric Sources

We present a novel spatial audio rendering technique to handle sound sources that can be represented by either an area or a volume in VR environments. As opposed to point-sampled sound sources, our approach projects the area-volumetric source to the spherical domain centered at the listener and represents this projection area compactly using the spherical harmonic (SH) basis functions. By representing the head-related transfer function (HRTF) in the same basis, we demonstrate that spatial audio which corresponds to an area-volumetric source can be efficiently computed as a dot product of the SH coefficients of the projection area and the HRTF. This results in an efficient technique whose computational complexity and memory requirements are independent of the complexity of the sound source. Our approach can support dynamic area-volumetric sound sources at interactive rates. We evaluate the performance of our technique in large complex VR environments and demonstrate significant improvement over the naive point-sampling technique. We also present results of a user evaluation, conducted to quantify the subjective preference of the user for our approach over the point-sampling approach in VR environments.

Sound localization with bilateral cochlear implants in noise: how much do head movements contribute to localization?

Bilateral cochlear implant (CI) users encounter difficulties in localizing sound sources in everyday environments, especially in the presence of background noise and reverberation. They tend to show large directional errors and front-back confusions compared to normal hearing (NH) subjects in the same conditions. In this study, the ability of bilateral CI users to use head movements to improve sound source localization was evaluated. Speech sentences of 0.5, 2, and 4.5 seconds were presented in noise to the listeners in conditions with and without head movements. The results show that for middle and long signal durations, the CI users could significantly reduce the number of front-back confusions. The angular accuracy, however, did not improve. Analysis of head trajectories showed that the CI users had great difficulties in moving their head towards the position of the source, whereas the NH listeners targeted the source loudspeaker correctly.

Parametric head-related transfer function modeling and interpolation for cost-efficient binaural sound applications

Parametric methods for modeling the perceptually relevant features of head-related transfer functions (HRTFs) are very important for the development of low-cost immersive sound applications. This letter describes an efficient method based on a low-order infinite impulse response filter implemented by a chain of second order sections of conventional shelving and peak audio filters. The parameters (central frequency, gain, and quality factor) are numerically adjusted by iteratively fitting the frequency response of the filter to the desired HRTF. Besides allowing for low-order binaural models, the proposed approach provides an efficient way to synthesize HRTFs for non-measured angles by applying a simple interpolation between the parameters from neighboring responses. Additionally, the HRTF database size is significantly reduced.

Visual Biasing of Auditory Localization in Azimuth and Depth

Correctly integrating sensory information across different modalities is a vital task, yet there are illusions which cause the incorrect localization of multisensory stimuli. A common example of these phenomena is the “ventriloquism effect.” In this illusion, the localization of auditory signals is biased by the presence of visual stimuli. For instance, when a light and sound are simultaneously presented, observers may erroneously locate the sound closer to the light than its actual position. While this phenomenon has been studied extensively in azimuth at a single depth, little is known about the interactions of stimuli at different depth planes. In the current experiment, virtual acoustics and stereo-image displays were used to test the integration of visual and auditory signals across azimuth and depth. The results suggest that greater variability in the localization of sounds in depth may lead to a greater bias from visual stimuli in depth than in azimuth. These results offer interesting implications for understanding multisensory integration.

Attentional gain control of ongoing cortical speech representations in a "cocktail party"

Normal listeners possess the remarkable perceptual ability to select a single speech stream among many competing talkers. However, few studies of selective attention have addressed the unique nature of speech as a temporally extended and complex auditory object. We hypothesized that sustained selective attention to speech in a multitalker environment would act as gain control on the early auditory cortical representations of speech. Using high-density electroencephalography and a template-matching analysis method, we found selective gain to the continuous speech content of an attended talker, greatest at a frequency of 4-8 Hz, in auditory cortex. In addition, the difference in alpha power (8-12 Hz) at parietal sites across hemispheres indicated the direction of auditory attention to speech, as has been previously found in visual tasks. The strength of this hemispheric alpha lateralization, in turn, predicted an individual's attentional gain of the cortical speech signal. These results support a model of spatial speech stream segregation, mediated by a supramodal attention mechanism, enabling selection of the attended representation in auditory cortex.

Perceptually relevant parameters for virtual listening simulation of small room acoustics

Various physical aspects of room-acoustic simulation techniques have been extensively studied and refined, yet the perceptual attributes of the simulations have received relatively little attention. Here a method of evaluating the perceptual similarity between rooms is described and tested using 15 small-room simulations based on binaural room impulse responses (BRIRs) either measured from a real room or estimated using simple geometrical acoustic modeling techniques. Room size and surface absorption properties were varied, along with aspects of the virtual simulation including the use of individualized head-related transfer function (HRTF) measurements for spatial rendering. Although differences between BRIRs were evident in a variety of physical parameters, a multidimensional scaling analysis revealed that when at-the-ear signal levels were held constant, the rooms differed along just two perceptual dimensions: one related to reverberation time (T(60)) and one related to interaural coherence (IACC). Modeled rooms were found to differ from measured rooms in this perceptual space, but the differences were relatively small and should be easily correctable through adjustment of T(60) and IACC in the model outputs. Results further suggest that spatial rendering using individualized HRTFs offers little benefit over nonindividualized HRTF rendering for room simulation applications where source direction is fixed.

Auditory attentional control and selection during cocktail party listening

In realistic auditory environments, people rely on both attentional control and attentional selection to extract intelligible signals from a cluttered background. We used functional magnetic resonance imaging to examine auditory attention to natural speech under such high processing-load conditions. Participants attended to a single talker in a group of 3, identified by the target talker's pitch or spatial location. A catch-trial design allowed us to distinguish activity due to top-down control of attention versus attentional selection of bottom-up information in both the spatial and spectral (pitch) feature domains. For attentional control, we found a left-dominant fronto-parietal network with a bias toward spatial processing in dorsal precentral sulcus and superior parietal lobule, and a bias toward pitch in inferior frontal gyrus. During selection of the talker, attention modulated activity in left intraparietal sulcus when using talker location and in bilateral but right-dominant superior temporal sulcus when using talker pitch. We argue that these networks represent the sources and targets of selective attention in rich auditory environments.

The binaural performance of a cross-talk cancellation system with matched or mismatched setup and playback acoustics

Cross-talk cancellation is a method for synthesizing virtual auditory space using loudspeakers. One implementation is the "Optimal Source Distribution" technique [T. Takeuchi and P. Nelson, J. Acoust. Soc. Am. 112, 2786-2797 (2002)], in which the audio bandwidth is split across three pairs of loudspeakers, placed at azimuths of +/-90 degrees, +/-15 degrees, and +/-3 degrees, conveying low, mid, and high frequencies, respectively. A computational simulation of this system was developed and verified against measurements made on an acoustic system using a manikin. Both the acoustic system and the simulation gave a wideband average cancellation of almost 25 dB. The simulation showed that when there was a mismatch between the head-related transfer functions used to set up the system and those of the final listener, the cancellation was reduced to an average of 13 dB. Moreover, in this case the binaural interaural time differences and interaural level differences delivered by the simulation of the optimal source distribution (OSD) system often differed from the target values. It is concluded that only when the OSD system is set up with "matched" head-related transfer functions can it deliver accurate binaural cues.

Fast head-related transfer function measurement via reciprocity

An efficient method for head-related transfer function (HRTF) measurement is presented. By applying the acoustical principle of reciprocity, one can swap the speaker and the microphone positions in the traditional (direct) HRTF measurement setup, that is, insert a microspeaker into the subject's ear and position several microphones around the subject, enabling simultaneous HRTF acquisition at all microphone positions. The setup used for reciprocal HRTF measurement is described, and the obtained HRTFs are compared with the analytical solution for a sound-hard sphere and with KEMAR manikin HRTF obtained by the direct method. The reciprocally measured sphere HRTF agrees well with the analytical solution. The reciprocally measured and the directly measured KEMAR HRTFs are not exactly identical but agree well in spectrum shape and feature positions. To evaluate if the observed differences are significant, an auditory localization model based on work by J. C. Middlebrooks [J. Acoust. Soc. Am. 92, 2607-2624 (1992)] was used to predict where a virtual sound source synthesized with the reciprocally measured HRTF would be localized if the directly measured HRTF were used for the localization. It was found that the predicted localization direction generally lies close to the measurement direction, indicating that the HRTFs obtained via the two methods are in good agreement.

Modeling individual differences in ferret external ear transfer functions

Individual variations in head and outer ear size, as well as growth of these structures during development, can markedly alter the values of the binaural and monaural cues which form the basis for auditory localization. This study investigated individual differences in the directional component of the head-related transfer function of both adult and juvenile ferrets. In line with previous studies in humans and cats, intersubject spectral differences were found to be reduced by scaling one of the directional transfer functions on a log-frequency axis. The optimal scale factor correlated most highly with pinna cavity height. Optimal frequency scaling reduced interear spectral difference equally well for adult-juvenile comparisons as for comparisons between pairs of adult ears. This illustrates that the developmental changes in localization cue values should be at least partly predictable on the basis of the expected growth rate of the outer ear structures. Predictions of interaural time differences (ITDs) were also derived from the physical dimensions of the head. ITDs were found to be poorly fitted by the spherical head model, while much better predictions could be derived from a model based on von Mises spherical basis functions. Together, these findings show how more accurate estimates of spatial cue values can be made from knowledge of the dimensions of the head and outer ears, and may facilitate the generation of virtual acoustic space stimuli in the absence of acoustical measurements from individual subjects.

Contribution of spectral cues to human sound localization

The contribution of spectral cues to human sound localization was investigated by removing cues in 1/2-, 1- or 2-octave bands in the frequency range above 4 kHz. Localization responses were given by placing an acoustic pointer at the same apparent position as a virtual target. The pointer was generated by filtering a 100-ms harmonic complex with equalized head-related transfer functions (HRTFs). Listeners controlled the pointer via a hand-held stick that rotated about a fixed point. In the baseline condition, the target, a 200-ms noise burst, was filtered with the same HRTFs as the pointer. In other conditions, the spectral information within a certain frequency band was removed by replacing the directional transfer function within this band with the average transfer of this band. Analysis of the data showed that removing cues in 1/2-octave bands did not affect localization, whereas for the 2-octave band correct localization was virtually impossible. The results obtained for the 1-octave bands indicate that up-down cues are located mainly in the 6-12-kHz band, and front-back cues in the 8-16-kHz band. The interindividual spread in response patterns suggests that different listeners use different localization cues. The response patterns in the median plane can be predicted using a model based on spectral comparison of directional transfer functions for target and response directions.

"What" and "where" in the human auditory system

The extent to which sound identification and sound localization depend on specialized auditory pathways was examined by using functional magnetic resonance imaging and event-related brain potentials. Participants performed an S1-S2 match-to-sample task in which S1 differed from S2 in its pitch and/or location. In the pitch task, participants indicated whether S2 was lower, identical, or higher in pitch than S1. In the location task, participants were asked to localize S2 relative to S1 (i.e., leftward, same, or rightward). Relative to location, pitch processing generated greater activation in auditory cortex and the inferior frontal gyrus. Conversely, identifying the location of S2 relative to S1 generated greater activation in posterior temporal cortex, parietal cortex, and the superior frontal sulcus. Differential task-related effects on event-related brain potentials (ERPs) were seen in anterior and posterior brain regions beginning at 300 ms poststimulus and lasting for several hundred milliseconds. The converging evidence from two independent measurements of dissociable brain activity during identification and localization of identical stimuli provides strong support for specialized auditory streams in the human brain. These findings are analogous to the "what" and "where" segregation of visual information processing, and suggest that a similar functional organization exists for processing information from the auditory modality.