Interaural correlation fails to account for detection in a classic binaural task: dynamic ITDs dominate N0Spi detection

Neural Fluctuation Contrast as a Code for Complex Sounds: The Role and Control of Peripheral Nonlinearities

The nonlinearities of the inner ear are often considered to be obstacles that the central nervous system has to overcome to decode neural responses to sounds. This review describes how peripheral nonlinearities, such as saturation of the inner-hair-cell response and of the IHC-auditory-nerve synapse, are instead beneficial to the neural encoding of complex sounds such as speech. These nonlinearities set up contrast in the depth of neural-fluctuations in auditory-nerve responses along the tonotopic axis, referred to here as neural fluctuation contrast (NFC). Physiological support for the NFC coding hypothesis is reviewed, and predictions of several psychophysical phenomena, including masked detection and speech intelligibility, are presented. Lastly, a framework based on the NFC code for understanding how the medial olivocochlear (MOC) efferent system contributes to the coding of complex sounds is presented. By modulating cochlear gain control in response to both sound energy and fluctuations in neural responses, the MOC system is hypothesized to function not as a simple feedback gain-control device, but rather as a mechanism for enhancing NFC along the tonotopic axis, enabling robust encoding of complex sounds across a wide range of sound levels and in the presence of background noise. Effects of sensorineural hearing loss on the NFC code and on the MOC feedback system are presented and discussed.

Dynamic spectral cues do not affect human sound localization during small head movements

Natural listening involves a constant deployment of small head movement. Spatial listening is facilitated by head movements, especially when resolving front-back confusions, an otherwise common issue during sound localization under head-still conditions. The present study investigated which acoustic cues are utilized by human listeners to localize sounds using small head movements (below ±10° around the center). Seven normal-hearing subjects participated in a sound localization experiment in a virtual reality environment. Four acoustic cue stimulus conditions were presented (full spectrum, flattened spectrum, frozen spectrum, free-field) under three movement conditions (no movement, head rotations over the yaw axis and over the pitch axis). Localization performance was assessed using three metrics: lateral and polar precision error and front-back confusion rate. Analysis through mixed-effects models showed that even small yaw rotations provide a remarkable decrease in front-back confusion rate, whereas pitch rotations did not show much of an effect. Furthermore, MSS cues improved localization performance even in the presence of dITD cues. However, performance was similar between stimuli with and without dMSS cues. This indicates that human listeners utilize the MSS cues before the head moves, but do not rely on dMSS cues to localize sounds when utilizing small head movements.

Responses to dichotic tone-in-noise stimuli in the inferior colliculus

Human listeners are more sensitive to tones embedded in diotic noise when the tones are out-of-phase at the two ears (N0Sπ) than when they are in-phase (N0S0). The difference between the tone-detection thresholds for these two conditions is referred to as the binaural masking level difference (BMLD) and reflects a benefit of binaural processing. Detection in the N0Sπ condition has been explained in modeling studies by changes in interaural correlation (IAC), but this model has only been directly tested physiologically for low frequencies. Here, the IAC-based hypothesis for binaural detection was examined across a wide range of frequencies and masker levels using recordings in the awake rabbit inferior colliculus (IC). IAC-based cues were strongly correlated with neural responses to N0Sπ stimuli. Additionally, average rate-based thresholds were calculated for both N0S0 and N0Sπ conditions. The rate-based neural BMLD at 500 Hz matched rabbit behavioral data, but the trend of neural BMLDs across frequency differed from that of humans.

Sensitivity to a Break in Interaural Correlation in Frequency-Gliding Noises

This study was to investigate whether human listeners are able to detect a binaurally uncorrelated arbitrary-noise fragment embedded in binaurally identical arbitrary-noise markers [a break in correlation, break in interaural correlation (BIAC)] in either frequency-constant (frequency-steady) or frequency-varied (unidirectionally frequency gliding) noise. Ten participants with normal hearing were tested in Experiment 1 for up-gliding, down-gliding, and frequency-steady noises. Twenty-one participants with normal hearing were tested in Experiment 2a for both up-gliding and frequency-steady noises. Another nineteen participants with normal hearing were tested in Experiment 2b for both down-gliding and frequency-steady noises. Listeners were able to detect a BIAC in the frequency-steady noise (center frequency = 400 Hz) and two types of frequency-gliding noises (center frequency: between 100 and 1,600 Hz). The duration threshold for detecting the BIAC in frequency-gliding noises was significantly longer than that in the frequency-steady noise (Experiment 1), and the longest interaural delay at which a duration-fixed BIAC (200 ms) in frequency-gliding noises could be detected was significantly shorter than that in the frequency-steady noise (Experiment 2). Although human listeners can detect a BIAC in frequency-gliding noises, their sensitivity to a BIAC in frequency-gliding noises is much lower than that in frequency-steady noise.

Neural coding and perception of auditory motion direction based on interaural time differences

While motion is important for parsing a complex auditory scene into perceptual objects, how it is encoded in the auditory system is unclear. Perceptual studies suggest that the ability to identify the direction of motion is limited by the duration of the moving sound, yet we can detect changes in interaural differences at even shorter durations. To understand the source of these distinct temporal limits, we recorded from single units in the inferior colliculus (IC) of unanesthetized rabbits in response to noise stimuli containing a brief segment with linearly time-varying interaural time difference ("ITD sweep") temporally embedded in interaurally uncorrelated noise. We also tested the ability of human listeners to either detect the ITD sweeps or identify the motion direction. Using a point-process model to separate the contributions of stimulus dependence and spiking history to single-neuron responses, we found that the neurons respond primarily by following the instantaneous ITD rather than exhibiting true direction selectivity. Furthermore, using an optimal classifier to decode the single-neuron responses, we found that neural threshold durations of ITD sweeps for both direction identification and detection overlapped with human threshold durations even though the average response of the neurons could track the instantaneous ITD beyond psychophysical limits. Our results suggest that the IC does not explicitly encode motion direction, but internal neural noise may limit the speed at which we can identify the direction of motion.NEW & NOTEWORTHY Recognizing motion and identifying an object's trajectory are important for parsing a complex auditory scene, but how we do so is unclear. We show that neurons in the auditory midbrain do not exhibit direction selectivity as found in the visual system but instead follow the trajectory of the motion in their temporal firing patterns. Our results suggest that the inherent variability in neural firings may limit our ability to identify motion direction at short durations.

Early Binaural Hearing: The Comparison of Temporal Differences at the Two Ears

Many mammals, including humans, are exquisitely sensitive to tiny time differences between sounds at the two ears. These interaural time differences are an important source of information for sound detection, for sound localization in space, and for environmental awareness. Two brainstem circuits are involved in the initial temporal comparisons between the ears, centered on the medial and lateral superior olive. Cells in these nuclei, as well as their afferents, display a large number of striking physiological and anatomical specializations to enable submillisecond sensitivity. As such, they provide an important model system to study temporal processing in the central nervous system. We review the progress that has been made in characterizing these primary binaural circuits as well as the variety of mechanisms that have been proposed to underlie their function.

Binaural unmasking with temporal envelope and fine structure in listeners with cochlear implants

For normal-hearing (NH) listeners, interaural information in both temporal envelope and temporal fine structure contribute to binaural unmasking of target signals in background noise; however, in many conditions low-frequency interaural information in temporal fine structure produces greater binaural unmasking. For bilateral cochlear-implant (CI) listeners, interaural information in temporal envelope contributes to binaural unmasking; however, the effect of encoding temporal fine structure information in electrical pulse timing (PT) is not fully understood. In this study, diotic and dichotic signal detection thresholds were measured in CI listeners using bilaterally synchronized single-electrode stimulation for conditions in which the temporal envelope was presented without temporal fine structure encoded (constant-rate pulses) or with temporal fine structure encoded (pulses timed to peaks of the temporal fine structure). CI listeners showed greater binaural unmasking at 125 pps with temporal fine structure encoded than without. There was no significant effect of encoding temporal fine structure at 250 pps. A similar pattern of performance was shown by NH listeners presented with acoustic pulse trains designed to simulate CI stimulation. The results suggest a trade-off across low rates between interaural information obtained from temporal envelope and that obtained from temporal fine structure encoded in PT.

No more than "slight" hearing loss and degradations in binaural processing

Listeners having, at most, "slight" hearing loss may exhibit substantial deficits in binaural detection [Bernstein and Trahiotis. (2016). J. Acoust. Soc. Am. 140, 3540-3548; (2018). J. Acoust. Soc. Am. 144, 292-307]. This study assessed whether such listeners also exhibit deficits discriminating interaural temporal disparities (ITDs) or interaural intensitive disparities (IIDs) and whether any deficits observed in those discrimination tasks would be accounted for by the interaural cross-correlation based model that successfully accounts for binaural detection. Thresholds were measured for detection of tones masked by noise in the NoSπ configuration and discrimination of ITD or IID. Gaussian noises (100 Hz-wide), served as maskers in the detection task and as reference and target stimuli in the discrimination tasks. Stimuli were centered at 500 Hz or 4 kHz. The latter were transpositions of stimuli centered at 125 Hz. Results demonstrate that listeners having, at most, slight hearing loss and who exhibit deficits in binaural detection, also exhibit deficits in ITD- and IID-discrimination. Coupled with appropriate decision variables, the cross-correlation-based model that accounts for elevated binaural detection thresholds among such listeners also accounted for their elevated ITD- and IID-thresholds. The deficits in all three tasks appear to stem from increased levels of stimulus-dependent, additive internal noise.

Untrained listeners experience difficulty detecting interaural correlation changes in narrowband noises

Interaural correlation change detection was measured in untrained normal-hearing listeners. Narrowband (10-Hz) noises were varied by center frequency (CF; 500 or 4000 Hz) and diotic level roving (absent or present). For the 500-Hz CF, 96% of listeners could achieve threshold (79.4% correct at the easiest testing level) if roving was absent, but only 36% of listeners could if level roving was present. No one could achieve threshold at the 4000-Hz CF, unlike trained listeners in the literature. The results raise questions about how individual differences affect learning and generalization of monaural and binaural cues related to interaural correlation detection.

Interaural envelope correlation change discrimination in bilateral cochlear implantees: effects of mismatch, centering, and onset of deafness

Bilateral cochlear implant (CI) listeners can perform binaural tasks, but they are typically worse than normal-hearing (NH) listeners. To understand why this difference occurs and the mechanisms involved in processing dynamic binaural differences, interaural envelope correlation change discrimination sensitivity was measured in real and simulated CI users. In experiment 1, 11 CI (eight late deafened, three early deafened) and eight NH listeners were tested in an envelope correlation change discrimination task. Just noticeable differences (JNDs) were best for a matched place-of-stimulation and increased for an increasing mismatch. In experiment 2, attempts at intracranially centering stimuli did not produce lower JNDs. In experiment 3, the percentage of correct identifications of antiphasic carrier pulse trains modulated by correlated envelopes was measured as a function of mismatch and pulse rate. Sensitivity decreased for increasing mismatch and increasing pulse rate. The experiments led to two conclusions. First, envelope correlation change discrimination necessitates place-of-stimulation matched inputs. However, it is unclear if previous experience with acoustic hearing is necessary for envelope correlation change discrimination. Second, NH listeners presented with CI simulations demonstrated better performance than real CI listeners. If the simulations are realistic representations of electrical stimuli, real CI listeners appear to have difficulty processing interaural information in modulated signals.

The effect of interaural fluctuation rate on correlation change discrimination

While bilateral cochlear implants (CIs) provide some binaural benefits, these benefits are limited compared to those observed in normal-hearing (NH) listeners. The large frequency-to-electrode allocation bandwidths (BWs) in CIs compared to auditory filter BWs in NH listeners increases the interaural fluctuation rate available for binaural unmasking, which may limit binaural benefits. The purpose of this work was to investigate the effect of interaural fluctuation rate on correlation change discrimination and binaural masking-level differences in NH listeners presented a CI simulation using a pulsed-sine vocoder. In experiment 1, correlation-change just-noticeable differences (JNDs) and tone-in-noise thresholds were measured for narrowband noises with different BWs and center frequencies (CFs). The results suggest that the BW, CF, and/or interaural fluctuation rate are important factors for correlation change discrimination. In experiment 2, the interaural fluctuation rate was systematically varied and dissociated from changes in BW and CF by using a pulsed-sine vocoder. Results indicated that the interaural fluctuation rate did not affect correlation change JNDs for correlated reference noises; however, slow interaural fluctuations increased correlation change JNDs for uncorrelated reference noises. In experiment 3, the BW, CF, and vocoder pulse rate were varied while interaural fluctuation rate was held constant. JNDs increased for increasing BW and decreased for increasing CF. In summary, relatively fast interaural fluctuation rates are not detrimental for detecting changes in interaural correlation. Thus, limiting factors to binaural benefits in CI listeners could be a result of other temporal and/or spectral deficiencies from electrical stimulation.

Binaural detection with narrowband and wideband reproducible noise maskers. IV. Models using interaural time, level, and envelope differences

The addition of out-of-phase tones to in-phase noises results in dynamic interaural level difference (ILD) and interaural time difference (ITD) cues for the dichotic tone-in-noise detection task. Several models have been used to predict listeners' detection performance based on ILD, ITD, or different combinations of the two cues. The models can be tested using detection performance from an ensemble of reproducible-noise maskers. Previous models cannot predict listeners' detection performance for reproducible-noise maskers without fitting the data. Here, two models were tested for narrowband and wideband reproducible-noise experiments. One model was a linear combination of ILD and ITD that included the generally ignored correlation between the two cues. The other model was based on a newly proposed cue, the slope of the interaural envelope difference (SIED). Predictions from both models explained a significant portion of listeners' performance for detection of a 500-Hz tone in wideband noise. Predictions based on the SIED approached the predictable variance in the wideband condition. The SIED represented a nonlinear combination of ILD and ITD, with the latter cue dominating. Listeners did not use a common strategy (cue) to detect tones in the narrowband condition and may use different single frequencies or different combinations of frequency channels.

Predicting spike timing in highly synchronous auditory neurons at different sound levels

A challenge for sensory systems is to encode natural signals that vary in amplitude by orders of magnitude. The spike trains of neurons in the auditory system must represent the fine temporal structure of sounds despite a tremendous variation in sound level in natural environments. It has been shown in vitro that the transformation from dynamic signals into precise spike trains can be accurately captured by simple integrate-and-fire models. In this work, we show that the in vivo responses of cochlear nucleus bushy cells to sounds across a wide range of levels can be precisely predicted by deterministic integrate-and-fire models with adaptive spike threshold. Our model can predict both the spike timings and the firing rate in response to novel sounds, across a large input level range. A noisy version of the model accounts for the statistical structure of spike trains, including the reliability and temporal precision of responses. Spike threshold adaptation was critical to ensure that predictions remain accurate at different levels. These results confirm that simple integrate-and-fire models provide an accurate phenomenological account of spike train statistics and emphasize the functional relevance of spike threshold adaptation.

The role of envelope statistics in detecting changes in interaural correlation

The role of envelope statistics in binaural masking-level differences (BMLDs) and correlation change detection was investigated in normal-hearing listeners. Thresholds and just-noticeable differences (JNDs) were measured for different bandwidths and center frequencies (500, 2000, 4000, and 8000 Hz) using Gaussian noises (GNs) and low-fluctuation noises (LFNs). At a 500-Hz center frequency, GN NoSo thresholds were higher than, NoSπ thresholds were lower than, and correlation change detection JNDs were the same as LFN thresholds and JNDs. At higher center frequencies, GN NoSπ thresholds were the same or higher than LFN thresholds and GN correlation change detection JNDs were much smaller than LFN JNDs. Using a pulsed sine vocoder, a second experiment was performed to investigate if binaural adaptation might contribute to the difference in GN and LFN detection. There was no effect of pulse rate, thus providing no clear evidence that binaural adaptation plays a role in these tasks. Both a cross-correlation model and a model that utilized the fluctuations in the interaural differences could explain a majority of the variance in the LFN correlation change JNDs.

The effect of different cochlear implant microphones on acoustic hearing individuals' binaural benefits for speech perception in noise

OBJECTIVES

Cochlear implant microphones differ in placement, frequency response, and other characteristics such as whether they are directional. Although normal-hearing (NH) individuals are often used as controls in studies examining cochlear implant users' binaural benefits, the considerable differences across cochlear implant microphones make such comparisons potentially misleading. The goal of this study was to examine binaural benefits for speech perception in noise for NH individuals using stimuli processed by head-related transfer functions (HRTFs) based on the different cochlear implant microphones.

DESIGN

HRTFs were created for different cochlear implant microphones and used to test participants on the Hearing in Noise Test. Experiment 1 tested cochlear implant users and NH individuals with HRTF-processed stimuli and with sound field (SF) testing to determine whether the HRTFs adequately simulated SF testing. Experiment 2 determined the measurement error and performance-intensity function for the Hearing in Noise Test with NH individuals listening to stimuli processed with the various HRTFs. Experiment 3 compared NH listeners' performance across HRTFs to determine how the HRTFs affected performance. Experiment 4 evaluated binaural benefits for NH listeners using the various HRTFs, including ones that were modified to investigate the contributions of interaural time and level cues.

RESULTS

The results indicated that the HRTFs adequately simulated SF testing for the Hearing in Noise Test. They also demonstrated that the test-retest reliability and performance-intensity function were consistent across HRTFs, and that the measurement error for the test was 1.3 dB, with a change in signal-to-noise ratio of 1 dB reflecting a 10% change in intelligibility. There were significant differences in performance when using the various HRTFs, with particularly good thresholds for the HRTF based on the directional microphone when the speech and masker were spatially separated, emphasizing the importance of measuring binaural benefits separately for each HRTF. Evaluation of binaural benefits indicated that binaural squelch and spatial release from masking were found for all HRTFs, and binaural summation was found for all but one HRTF, although binaural summation was less robust than the other types of binaural benefits. In addition, the results indicated that neither interaural time nor level cues dominated binaural benefits for the NH participants.

CONCLUSIONS

This study provides a means to measure the degree to which cochlear implant microphones affect acoustic hearing with respect to speech perception in noise. It also provides measures that can be used to evaluate the independent contributions of interaural time and level cues. These measures provide tools that can aid researchers in understanding and improving binaural benefits in acoustic hearing individuals listening via cochlear implant microphones.

Subcomponent cues in binaural unmasking

The addition of a signal in the N0Sπ binaural configuration gives rise to fluctuations in interaural phase and amplitude. Sensitivity to these individual cues was measured by applying sinusoidal amplitude modulation (AM) or quasi-frequency modulation (QFM) to a band of noise. Discrimination between interaurally in-phase and out-of-phase modulation was measured using an adaptive task for narrow bands of noise at center frequencies from 250 to 1500 Hz, for modulation rates of 2-40 Hz, and with or without flanking bands of diotic noise. Discrimination thresholds increased steeply for QFM with increasing center frequency, but increased only modestly for AM, and mainly for modulation rates below 10 Hz. Flanking bands of noise increased thresholds for AM, but had no consistent effect for QFM. The results suggest that two underlying mechanisms may support binaural unmasking: one most sensitive to interaural amplitude modulations that is susceptible to across-frequency interference, and a second, most sensitive to interaural phase modulations that is immune to such effects.

Interaural fluctuations and the detection of interaural incoherence. IV. The effect of compression on stimulus statistics

The purpose of this experiment was to determine whether the normalized interaural cross-correlation (CC) model or a model based on interaural phase and level differences can better describe incoherence detection data. The ability to detect interaural incoherence in three sets of reproducible dichotic noises was tested in six listeners. The first set contained noises with a constrained value of the CC and the CC including signal compression. The second set contained noises with a constrained value of the CC including signal compression. The third set contained noises with constrained values in the fluctuations in the interaural differences. Modeling showed that neither the CC model nor the model using the interaural differences could account for the data in any set. Examination of the statistical properties of the stimuli showed that including compression before the calculation of the interaural CC causes a substantial correlation of this metric to the fluctuations in the interaural phase difference. This finding implies that it may be more difficult to discriminate between the common types of binaural models than previously thought.

Estimated Cochlear Delays in Low Best-Frequency Neurons in the Barn Owl Cannot Explain Coding of Interaural Time Difference

The functional role of the low-frequency range (<3 kHz) in barn owl hearing is not well understood. Here, it was tested whether cochlear delays could explain the representation of interaural time difference (ITD) in this frequency range. Recordings were obtained from neurons in the core of the central nucleus of the inferior colliculus. The response of these neurons varied with the ITD of the stimulus. The response peak shared by all neurons in a dorsoventral penetration was called the array-specific ITD and served as criterion for the representation of a given ITD in a neuron. Array-specific ITDs were widely distributed. Isolevel frequency response functions obtained with binaural, contralateral, and ispilateral stimulation exhibited a clear response peak and the accompanying frequency was called the best frequency. The data were tested with respect to predictions of a model, the stereausis model, assuming cochlear delays as source for the best ITD of a neuron. According to this model, different cochlear delays determined by mismatches between the ipsilateral and contralateral best frequencies are the source for the ITD in a binaural neuron. The mismatch should depend on the best frequency and the best ITD. The predictions of the stereausis model were not fulfilled in the low best-frequency neurons analyzed here. It is concluded that cochlear delays are not responsible for the representation of best ITD in the barn owl.

Neural correlates of binaural masking level difference in the inferior colliculus of the barn owl (Tyto alba)

Humans and animals are able to detect signals in noisy environments. Detection improves when the noise and the signal have different interaural phase relationships. The resulting improvement in detection threshold is called the binaural masking level difference. We investigated neural mechanisms underlying the release from masking in the inferior colliculus of barn owls in low-frequency and high-frequency neurons. A tone (signal) was presented either with the same interaural time difference as the noise (masker) or at a 180 degrees phase shift as compared with the interaural time difference of the noise. The changes in firing rates induced by the addition of a signal of increasing level while masker level was kept constant was well predicted by the relative responses to the masker and signal alone. In many cases, the response at the highest signal levels was dominated by the response to the signal alone, in spite of a significant response to the masker at low signal levels, suggesting the presence of occlusion. Detection thresholds and binaural masking level differences were widely distributed. The amount of release from masking increased with increasing masker level. Narrowly tuned neurons in the central nucleus of the inferior colliculus had detection thresholds that were lower than or similar to those of broadly tuned neurons in the external nucleus of the inferior colliculus. Broadly tuned neurons exhibited higher masking level differences than narrowband neurons. These data suggest that detection has different spectral requirements from localization.