Temporal weighting functions for interaural time and level differences. IV. Effects of carrier frequency

Rate dependent neural responses of interaural-time-difference cues in fine-structure and envelope

Advancements in cochlear implants (CIs) have led to a significant increase in bilateral CI users, especially among children. Yet, most bilateral CI users do not fully achieve the intended binaural benefit due to potential limitations in signal processing and/or surgical implant positioning. One crucial auditory cue that normal hearing (NH) listeners can benefit from is the interaural time difference (ITD), i.e., the time difference between the arrival of a sound at two ears. The ITD sensitivity is thought to be heavily relying on the effective utilization of temporal fine structure (very rapid oscillations in sound). Unfortunately, most current CIs do not transmit such true fine structure. Nevertheless, bilateral CI users have demonstrated sensitivity to ITD cues delivered through envelope or interaural pulse time differences, i.e., the time gap between the pulses delivered to the two implants. However, their ITD sensitivity is significantly poorer compared to NH individuals, and it further degrades at higher CI stimulation rates, especially when the rate exceeds 300 pulse per second. The overall purpose of this research thread is to improve spatial hearing abilities in bilateral CI users. This study aims to develop electroencephalography (EEG) paradigms that can be used with clinical settings to assess and optimize the delivery of ITD cues, which are crucial for spatial hearing in everyday life. The research objective of this article was to determine the effect of CI stimulation pulse rate on the ITD sensitivity, and to characterize the rate-dependent degradation in ITD perception using EEG measures. To develop protocols for bilateral CI studies, EEG responses were obtained from NH listeners using sinusoidal-amplitude-modulated (SAM) tones and filtered clicks with changes in either fine structure ITD (ITDFS) or envelope ITD (ITDENV). Multiple EEG responses were analyzed, which included the subcortical auditory steady-state responses (ASSRs) and cortical auditory evoked potentials (CAEPs) elicited by stimuli onset, offset, and changes. Results indicated that acoustic change complex (ACC) responses elicited by ITDENV changes were significantly smaller or absent compared to those elicited by ITDFS changes. The ACC morphologies evoked by ITDFS changes were similar to onset and offset CAEPs, although the peak latencies were longest for ACC responses and shortest for offset CAEPs. The high-frequency stimuli clearly elicited subcortical ASSRs, but smaller than those evoked by lower carrier frequency SAM tones. The 40-Hz ASSRs decreased with increasing carrier frequencies. Filtered clicks elicited larger ASSRs compared to high-frequency SAM tones, with the order being 40 > 160 > 80> 320 Hz ASSR for both stimulus types. Wavelet analysis revealed a clear interaction between detectable transient CAEPs and 40-Hz ASSRs in the time-frequency domain for SAM tones with a low carrier frequency.

Fast binaural processing but sluggish masker representation reconfiguration

Perceptual organization of complex acoustic scenes requires fast binaural processing for accurate localization or lateralization based on short single-source-dominated glimpses. This sensitivity also manifests in the ability to detect rapid oscillating interaural time and phase differences as well as interaural correlation. However, binaural processing has also been termed "sluggish" based on experiments that require binaural detection in a masker with an additional binaural cue change in temporal proximity. The present study shows that the temporal integration windows obtained from data on binaural sluggishness cannot account for the detection of rapid binaural oscillations. A model with fast IPD encoding but a slower process of updating the internal representation of the masker IPD statistics accounted for both experiments of the "fast" and the "sluggish" categories.

Effect of Realistic Test Conditions on Perception of Speech, Music, and Binaural Cues in Normal-Hearing Listeners

PURPOSE

The purpose of this study was to determine the feasibility of online testing in a quiet room for three auditory perception experiments in normal-hearing listeners: speech, music, and binaural cue.

METHOD

Under Experiment 1, sentence perception was measured using fixed signal-to-noise ratios (SNRs: +10 dB, 0 dB, and -10 dB) and using adaptive speech reception threshold (SRT) procedures. The correct scores were compared between quiet room and soundproof booth listening environments. Experiment 2 was designed to compare melodic contour identification between the two listening environments. Melodic contour identification was assessed with 1, 2, and 4 semitone spacings. Under Experiment 3, interaural level difference (ILD) and interaural time differences (ITD) were measured as a function of carrier frequency. For both measures, two modulated tones (400-ms duration and 100-Hz modulation rate) were sequentially presented through headphones to both ears, and subjects were asked to indicate whether the sound moved to the left or right ear. The measured ITD and ILD were then compared between the two listening environments.

RESULTS

There were no significant differences in any outcome measures (SNR- and SRT-based speech perception, melodic contour identification, and ITD/ILD) between the two listening environments.

CONCLUSIONS

These results suggest that normal-hearing listeners may not require a controlled listening environment in any of the three auditory assessments. As comparable data can be obtained via the online testing tool, using the online auditory experiments is recommended.

The precedence effect in spatial hearing manifests in cortical neural population responses

BACKGROUND

To localize sound sources accurately in a reverberant environment, human binaural hearing strongly favors analyzing the initial wave front of sounds. Behavioral studies of this "precedence effect" have so far largely been confined to human subjects, limiting the scope of complementary physiological approaches. Similarly, physiological studies have mostly looked at neural responses in the inferior colliculus, the main relay point between the inner ear and the auditory cortex, or used modeling of cochlear auditory transduction in an attempt to identify likely underlying mechanisms. Studies capable of providing a direct comparison of neural coding and behavioral measures of sound localization under the precedence effect are lacking.

RESULTS

We adapted a "temporal weighting function" paradigm previously developed to quantify the precedence effect in human for use in laboratory rats. The animals learned to lateralize click trains in which each click in the train had a different interaural time difference. Computing the "perceptual weight" of each click in the train revealed a strong onset bias, very similar to that reported for humans. Follow-on electrocorticographic recording experiments revealed that onset weighting of interaural time differences is a robust feature of the cortical population response, but interestingly, it often fails to manifest at individual cortical recording sites.

CONCLUSION

While previous studies suggested that the precedence effect may be caused by early processing mechanisms in the cochlea or inhibitory circuitry in the brainstem and midbrain, our results indicate that the precedence effect is not fully developed at the level of individual recording sites in the auditory cortex, but robust and consistent precedence effects are observable only in the auditory cortex at the level of cortical population responses. This indicates that the precedence effect emerges at later cortical processing stages and is a significantly "higher order" feature than has hitherto been assumed.

Probing temporal modulation detection in white noise using intrinsic envelope fluctuations: A reverse-correlation study

Part of the detrimental effect caused by a stationary noise on sound perception results from the masking of relevant amplitude modulations (AM) in the signal by random intrinsic envelope fluctuations arising from the filtering of noise by cochlear channels. This study capitalizes on this phenomenon to probe AM detection strategies for human listeners using a reverse correlation analysis. Eight normal-hearing listeners were asked to detect the presence of a 4-Hz sinusoidal AM target applied to a 1-kHz tone carrier using a yes-no task with 3000 trials/participant. All stimuli were embedded in a white-noise masker. A reverse-correlation analysis was then carried on the data to compute "psychophysical kernels" showing which aspects of the stimulus' temporal envelope influenced the listener's responses. These results were compared to data simulated with different implementations of a modulation-filterbank model. Psychophysical kernels revealed that human listeners were able to track the position of AM peaks in the target, similar to the models. However, they also showed a marked temporal decay and a consistent phase shift compared to the ideal template. In light of the simulated data, this was interpreted as an evidence for the presence of phase uncertainty in the processing of intrinsic envelope fluctuations.

High-resolution temporal weighting of interaural time differences in speech

Previous studies have shown that for high-rate click trains and low-frequency pure tones, interaural time differences (ITDs) at the onset of stimulus contribute most strongly to the overall lateralization percept (receive the largest perceptual weight). Previous studies have also shown that when these stimuli are modulated, ITDs during the rising portion of the modulation cycle receive increased perceptual weight. Baltzell, Cho, Swaminathan, and Best [(2020). J. Acoust. Soc. Am. 147, 3883-3894] measured perceptual weights for a pair of spoken words ("two" and "eight"), and found that word-initial phonemes receive larger weight than word-final phonemes, suggesting a "word-onset dominance" for speech. Generalizability of this conclusion was limited by a coarse temporal resolution and limited stimulus set. In the present study, temporal weighting functions (TWFs) were measured for four spoken words ("two," "eight," "six," and "nine"). Stimuli were partitioned into 30-ms bins, ITDs were applied independently to each bin, and lateralization judgements were obtained. TWFs were derived using a hierarchical regression model. Results suggest that "word-initial" onset dominance does not generalize across words and that TWFs depend in part on acoustic changes throughout the stimulus. Two model-based predictions were generated to account for observed TWFs, but neither could fully account for the perceptual data.

The influence of envelope shape on the lateralization of amplitude-modulated, low-frequency sound

For abruptly gated sound, interaural time difference (ITD) cues at onset carry greater perceptual weight than those following. This research explored how envelope shape influences such carrier ITD weighting. Experiment 1 assessed the perceived lateralization of a tonal binaural beat that transitioned through ITD (diotic envelope, mean carrier frequency of 500 Hz). Listeners' left/right lateralization judgments were compared to those for static-ITD tones. For an 8 Hz sinusoidally amplitude-modulated envelope, ITD cues 24 ms after onset well-predicted reported sidedness. For an equivalent-duration "abrupt" envelope, which was unmodulated besides 20-ms onset/offset ramps, reported sidedness corresponded to ITDs near onset (e.g., 6 ms). However, unlike for sinusoidal amplitude modulation, ITDs toward offset seemingly also influenced perceived sidedness. Experiment 2 adjusted the duration of the offset ramp (25-75 ms) and found evidence for such offset weighting only for the most abrupt ramp tested. In experiment 3, an ITD was imposed on a brief segment of otherwise diotic filtered noise. Listeners discriminated right- from left-leading ITDs. In sinusoidal amplitude modulation, thresholds were lowest when the ITD segment occurred during rising amplitude. For the abrupt envelope, the lowest thresholds were observed when the segment occurred at either onset or offset. These experiments demonstrate the influence of envelope profile on carrier ITD sensitivity.

Sub-optimal construction of an auditory profile from temporally distributed spectral information

When spectral components of a complex sound are presented not simultaneously but distributed over time, human listeners can still, to a degree, perceptually recover the spectral profile of the sound. This capability of integrating spectral information over time was investigated using a cued informational masking paradigm. Listeners detected a 1-kHz pure tone in a simultaneous masker composed of six random-frequency tones drawn on every trial. The spectral profile of the masker was cued using a precursor sound that consisted of a sequence of 50-ms bursts, separated by inter-burst intervals of 100 ms. Each burst in the precursor consisted of pure tones at the masker frequencies with tones appearing at each of the masker frequencies at different presentation probabilities. As the presentation probability increased in different conditions, the detectability of the target improved, indicating reliable precursor cuing regarding the spectral content of the masker. For many listeners, performance did not significantly improve as the number of precursor bursts increased from 2 to 16, indicating inefficient integration of information beyond 2 bursts. Additional analyses suggest that when intensity of the bursts is relatively constant, the contribution of the precursor is dominated by information in the initial burst.

Perceptual Weighting of Binaural Lateralization Cues across Frequency Bands

The auditory system uses interaural time and level differences (ITD and ILD) as cues to localize and lateralize sounds. The availability of ITDs and ILDs in the auditory system is limited by neural phase-locking and by the head size, respectively. Although the frequency-specific limitations are well known, the relative contribution of ITDs and ILDs in individual frequency bands in broadband stimuli is unknown. To determine these relative contributions, or spectral weights, listeners were asked to lateralize stimuli consisting of eleven simultaneously presented 1-ERB-wide noise bands centered between 442 and 5544 Hz and separated by 1-ERB-wide gaps. Either ITDs or ILDs were varied independently across each noise band, while fixing the other interaural disparity to either 0 dB or 0 μs. The weights were obtained using a multiple linear regression analysis. In a second experiment, the effect of auditory enhancement on the spectral weights was investigated. The enhancement of single noise bands was realized by presenting ten of the noise bands as preceding and following sounds (pre- and post-cursors, respectively). Listeners were asked to lateralize the stimuli as in the first experiment. Results show that in the absence of pre- and post-cursors, only the lowest or highest frequency band received highest weight for ITD and ILD, respectively. Auditory enhancement led to significantly enhanced weights given to the band without the pre- and post-cursor. The weight enhancement could only be observed at low frequencies, when determined with ITD cues and for low and high frequencies for ILDs. Hence, the auditory system seems to be able to change the spectral weighting of binaural information depending on the information content.

The development of perceptual averaging: Efficiency metrics in children and adults using a multiple-observation sound-localization task

This study examined the ability of older children to integrate spatial information across sequential observations of bandpass noise. In experiment I, twelve adults and twelve 8-14 yr olds localized 1-5 sounds, all presented at the same location along a 34° speaker array. Rate of gain in response precision (as a function of N observations) was used to measure integration efficiency. Children were no worse at localizing a single sound than adults, and-unexpectedly-were no less efficient at integrating information across observations. Experiment II repeated the task using a Reverse Correlation paradigm. The number of observations was fixed (N = 5), and the location of each sound was independently randomly jittered. Relative weights were computed for each observation interval. Distance from the ideal weight-vector was used to index integration efficiency. The data showed that children were significantly less efficient integrators than adults: only reaching adult-like performance by around 11 yrs. The developmental effect was small, however, relative to the amount of individual variability, with some younger children exhibiting greater efficiency than some adults. This work indicates that sensory integration continues to mature into late childhood, but that this development is relatively gradual.

Reverberation enhances onset dominance in sound localization

Temporal variation in sensitivity to sound-localization cues was measured in anechoic conditions and in simulated reverberation using the temporal weighting function (TWF) paradigm [Stecker and Hafter (2002). J. Acoust. Soc. Am. 112, 1046-1057]. Listeners judged the locations of Gabor click trains (4 kHz center frequency, 5-ms interclick interval) presented from an array of loudspeakers spanning 360° azimuth. Targets ranged ±56.25° across trials. Individual clicks within each train varied by an additional ±11.25° to allow TWF calculation by multiple regression. In separate conditions, sounds were presented directly or in the presence of simulated reverberation: 13 orders of lateral reflection were computed for a 10 m × 10 m room ( RT60≊300 ms) and mapped to the appropriate locations in the loudspeaker array. Results reveal a marked increase in perceptual weight applied to the initial click in reverberation, along with a reduction in the impact of late-arriving sound. In a second experiment, target stimuli were preceded by trains of "conditioner" sounds with or without reverberation. Effects were modest and limited to the first few clicks in a train, suggesting that impacts of reverberant pre-exposure on localization may be limited to the processing of information from early reflections.

Temporal weighting functions for interaural time and level differences. V. Modulated noise carriers

Sound onsets dominate spatial judgments of many types of periodic sound. Conversely, ongoing cues often dominate in spatial judgments of aperiodic noise. This study quantified onset dominance as a function of both the bandwidth and the temporal regularity of stimuli by measuring temporal weighting functions (TWF) from Stecker, Ostreicher, and Brown [(2013) J. Acoust. Soc. Am. 134, 1242-1252] for lateralization of periodic and aperiodic noise-burst trains. Stimuli consisted of 16 noise bursts (1 ms each) repeating at an interval of 2 or 5 ms. TWFs were calculated by multiple regression of lateralization judgments onto interaural time and level differences, which varied independently ( ±100 μs, ±2 dB) across bursts. Noise tokens were either refreshed on each burst (aperiodic) or repeated across sets of 2, 4, 8, or 16 bursts. TWFs revealed strong onset dominance for periodic noise-burst trains (16 repeats per token), which was markedly reduced in aperiodic trains. A second experiment measured TWFs for periodic but sinusoidally amplitude-modulated noise burst trains, revealing greater weight on the earliest and least intense bursts of the rising envelope slope. The results support the view that envelope fluctuations drive access to binaural information in both periodic and aperiodic sounds.

Lateralization of simulated sources and echoes differing in frequency based on interaural temporal differences

This study examined listeners' ability to process interaural temporal differences (ITDs) in one of two sequential sounds when the two differed in spectral content. A correlational analysis assessed weights given to ITDs of simulated source and echo pulses for echo delays of 8-128 ms for conditions in which responses were based on the source or echo, a 3000-Hz Gaussian (target) pulse. The other (distractor) pulse was spectrally centered at 1500, 2000, 3000, 4000, or 5000 Hz. Also measured were proportion correct and proportion of responses predicted from the weights. Regardless of whether the echo or source pulse served as the target, target weight, and proportion correct increased with increasing distractor frequency, consistent with low-frequency dominance [Divenyi, J. Acoust. Soc. Am. 91, 1078-1084 (1992)]. Effects of distractor frequency were observed at echo delays out to 128 ms when the source served as the target, but only out to 64 ms when the echo served as the target. At echo delays beyond 8 ms, recency effects were exhibited with higher proportions correct obtained for judgments based on the echo pulse than the source pulse.

Spectrotemporal weighting of binaural cues: Effects of a diotic interferer on discrimination of dynamic interaural differences

Spatial judgments are often dominated by low-frequency binaural cues and onset cues when binaural cues vary across the spectrum and duration, respectively, of a brief sound. This study combined these dimensions to assess the spectrotemporal weighting of binaural information. Listeners discriminated target interaural time difference (ITD) and interaural level difference (ILD) carried by the onset, offset, or full duration of a 4-kHz Gabor click train with a 2-ms period in the presence or absence of a diotic 500-Hz interferer tone. ITD and ILD thresholds were significantly elevated by the interferer in all conditions and by a similar amount to previous reports for static cues. Binaural interference was dramatically greater for ITD targets lacking onset cues compared to onset and full-duration conditions. Binaural interference for ILD targets was similar across dynamic-cue conditions. These effects mirror the baseline discriminability of dynamic ITD and ILD cues [Stecker and Brown. (2010). J. Acoust. Soc. Am. 127, 3092-3103], consistent with stronger interference for less-robust/higher-variance cues. The results support the view that binaural cue integration occurs simultaneously across multiple variance-weighted dimensions, including time and frequency.

Sound source localization identification accuracy: Level and duration dependencies

Sound source localization accuracy for noises was measured for sources in the front azimuthal open field mainly as a function of overall noise level and duration. An identification procedure was used in which listeners identify which loudspeakers presented a sound. Noises were filtered and differed in bandwidth and center frequency. Sound source localization accuracy depended on the bandwidth of the stimuli, and for the narrow bandwidths, accuracy depended on the filter's center frequency. Sound source localization accuracy did not depend on overall level or duration.

Sensitivity to Envelope Interaural Time Differences at High Modulation Rates

Sensitivity to interaural time differences (ITDs) conveyed in the temporal fine structure of low-frequency tones and the modulated envelopes of high-frequency sounds are considered comparable, particularly for envelopes shaped to transmit similar fidelity of temporal information normally present for low-frequency sounds. Nevertheless, discrimination performance for envelope modulation rates above a few hundred Hertz is reported to be poor-to the point of discrimination thresholds being unattainable-compared with the much higher (>1,000 Hz) limit for low-frequency ITD sensitivity, suggesting the presence of a low-pass filter in the envelope domain. Further, performance for identical modulation rates appears to decline with increasing carrier frequency, supporting the view that the low-pass characteristics observed for envelope ITD processing is carrier-frequency dependent. Here, we assessed listeners' sensitivity to ITDs conveyed in pure tones and in the modulated envelopes of high-frequency tones. ITD discrimination for the modulated high-frequency tones was measured as a function of both modulation rate and carrier frequency. Some well-trained listeners appear able to discriminate ITDs extremely well, even at modulation rates well beyond 500 Hz, for 4-kHz carriers. For one listener, thresholds were even obtained for a modulation rate of 800 Hz. The highest modulation rate for which thresholds could be obtained declined with increasing carrier frequency for all listeners. At 10 kHz, the highest modulation rate at which thresholds could be obtained was 600 Hz. The upper limit of sensitivity to ITDs conveyed in the envelope of high-frequency modulated sounds appears to be higher than previously considered.

Temporal weighting of binaural information at low frequencies: Discrimination of dynamic interaural time and level differences

The importance of sound onsets in binaural hearing has been addressed in many studies, particularly at high frequencies, where the onset of the envelope may carry much of the useful binaural information. Some studies suggest that sound onsets might play a similar role in the processing of binaural cues [e.g., fine-structure interaural time differences (ITD)] at low frequencies. This study measured listeners' sensitivity to ITD and interaural level differences (ILD) present in early (i.e., onset) and late parts of 80-ms pure tones of 250-, 500-, and 1000-Hz frequency. Following previous studies, tones carried static interaural cues or dynamic cues that peaked at sound onset and diminished to zero at sound offset or vice versa. Although better thresholds were observed in static than dynamic conditions overall, ITD discrimination was especially impaired, regardless of frequency, when cues were not available at sound onset. Results for ILD followed a similar pattern at 1000 Hz; at lower frequencies, ILD thresholds did not differ significantly between dynamic-cue conditions. The results support the "onset" hypothesis of Houtgast and Plomp [(1968). J. Acoust. Soc. Am. 44, 807-812] for ITD discrimination, but not necessarily ILD discrimination, in low-frequency pure tones.