Spectral and temporal modulation tradeoff in the inferior colliculus

Parallel Encoding of Speech in Human Frontal and Temporal Lobes

Models of speech perception are centered around a hierarchy in which auditory representations in the thalamus propagate to primary auditory cortex, then to the lateral temporal cortex, and finally through dorsal and ventral pathways to sites in the frontal lobe. However, evidence for short latency speech responses and low-level spectrotemporal representations in frontal cortex raises the question of whether speech-evoked activity in frontal cortex strictly reflects downstream processing from lateral temporal cortex or whether there are direct parallel pathways from the thalamus or primary auditory cortex to the frontal lobe that supplement the traditional hierarchical architecture. Here, we used high-density direct cortical recordings, high-resolution diffusion tractography, and hemodynamic functional connectivity to evaluate for evidence of direct parallel inputs to frontal cortex from low-level areas. We found that neural populations in the frontal lobe show speech-evoked responses that are synchronous or occur earlier than responses in the lateral temporal cortex. These short latency frontal lobe neural populations encode spectrotemporal speech content indistinguishable from spectrotemporal encoding patterns observed in the lateral temporal lobe, suggesting parallel auditory speech representations reaching temporal and frontal cortex simultaneously. This is further supported by white matter tractography and functional connectivity patterns that connect the auditory nucleus of the thalamus (medial geniculate body) and the primary auditory cortex to the frontal lobe. Together, these results support the existence of a robust pathway of parallel inputs from low-level auditory areas to frontal lobe targets and illustrate long-range parallel architecture that works alongside the classical hierarchical speech network model.

Adaptation to Noise in Spectrotemporal Modulation Detection and Word Recognition

Noise adaptation is the improvement in auditory function as the signal of interest is delayed in the noise. Here, we investigated if noise adaptation occurs in spectral, temporal, and spectrotemporal modulation detection as well as in speech recognition. Eighteen normal-hearing adults participated in the experiments. In the modulation detection tasks, the signal was a 200ms spectrally and/or temporally modulated ripple noise. The spectral modulation rate was two cycles per octave, the temporal modulation rate was 10 Hz, and the spectrotemporal modulations combined these two modulations, which resulted in a downward-moving ripple. A control experiment was performed to determine if the results generalized to upward-moving ripples. In the speech recognition task, the signal consisted of disyllabic words unprocessed or vocoded to maintain only envelope cues. Modulation detection thresholds at 0 dB signal-to-noise ratio and speech reception thresholds were measured in quiet and in white noise (at 60 dB SPL) for noise-signal onset delays of 50 ms (early condition) and 800 ms (late condition). Adaptation was calculated as the threshold difference between the early and late conditions. Adaptation in word recognition was statistically significant for vocoded words (2.1 dB) but not for natural words (0.6 dB). Adaptation was found to be statistically significant in spectral (2.1 dB) and temporal (2.2 dB) modulation detection but not in spectrotemporal modulation detection (downward ripple: 0.0 dB, upward ripple: -0.4 dB). Findings suggest that noise adaptation in speech recognition is unrelated to improvements in the encoding of spectrotemporal modulation cues.

Neuronal responses in mouse inferior colliculus correlate with behavioral detection of amplitude-modulated sound

Amplitude modulation (AM) is a common feature of natural sounds, including speech and animal vocalizations. Here, we used operant conditioning and in vivo electrophysiology to determine the AM detection threshold of mice as well as its underlying neuronal encoding. Mice were trained in a Go-NoGo task to detect the transition to AM within a noise stimulus designed to prevent the use of spectral side-bands or a change in intensity as alternative cues. Our results indicate that mice, compared with other species, detect high modulation frequencies up to 512 Hz well, but show much poorer performance at low frequencies. Our in vivo multielectrode recordings in the inferior colliculus (IC) of both anesthetized and awake mice revealed a few single units with remarkable phase-locking ability to 512 Hz modulation, but not sufficient to explain the good behavioral detection at that frequency. Using a model of the population response that combined dimensionality reduction with threshold detection, we reproduced the general band-pass characteristics of behavioral detection based on a subset of neurons showing the largest firing rate change (both increase and decrease) in response to AM, suggesting that these neurons are instrumental in the behavioral detection of AM stimuli by the mice.NEW & NOTEWORTHY The amplitude of natural sounds, including speech and animal vocalizations, often shows characteristic modulations. We examined the relationship between neuronal responses in the mouse inferior colliculus and the behavioral detection of amplitude modulation (AM) in sound and modeled how the former can give rise to the latter. Our model suggests that behavioral detection can be well explained by the activity of a subset of neurons showing the largest firing rate changes in response to AM.

Two stages of bandwidth scaling drives efficient neural coding of natural sounds

Theories of efficient coding propose that the auditory system is optimized for the statistical structure of natural sounds, yet the transformations underlying optimal acoustic representations are not well understood. Using a database of natural sounds including human speech and a physiologically-inspired auditory model, we explore the consequences of peripheral (cochlear) and mid-level (auditory midbrain) filter tuning transformations on the representation of natural sound spectra and modulation statistics. Whereas Fourier-based sound decompositions have constant time-frequency resolution at all frequencies, cochlear and auditory midbrain filters bandwidths increase proportional to the filter center frequency. This form of bandwidth scaling produces a systematic decrease in spectral resolution and increase in temporal resolution with increasing frequency. Here we demonstrate that cochlear bandwidth scaling produces a frequency-dependent gain that counteracts the tendency of natural sound power to decrease with frequency, resulting in a whitened output representation. Similarly, bandwidth scaling in mid-level auditory filters further enhances the representation of natural sounds by producing a whitened modulation power spectrum (MPS) with higher modulation entropy than both the cochlear outputs and the conventional Fourier MPS. These findings suggest that the tuning characteristics of the peripheral and mid-level auditory system together produce a whitened output representation in three dimensions (frequency, temporal and spectral modulation) that reduces redundancies and allows for a more efficient use of neural resources. This hierarchical multi-stage tuning strategy is thus likely optimized to extract available information and may underlies perceptual sensitivity to natural sounds.

Recognition of brief sounds in rapid serial auditory presentation

Two experiments were conducted to test the role of participant factors (i.e., musical sophistication, working memory capacity) and stimulus factors (i.e., sound duration, timbre) on auditory recognition using a rapid serial auditory presentation paradigm. Participants listened to a rapid stream of very brief sounds ranging from 30 to 150 milliseconds and were tested on their ability to distinguish the presence from the absence of a target sound selected from various sound sources placed amongst the distracters. Experiment 1a established that brief exposure to stimuli (60 to 150 milliseconds) does not necessarily correspond to impaired recognition. In Experiment 1b we found evidence that 30 milliseconds of exposure to the stimuli significantly impairs recognition of single auditory targets, but the recognition for voice and sine tone targets impaired the least, suggesting that the lower limit required for successful recognition could be lower than 30 milliseconds for voice and sine tone targets. Critically, the effect of sound duration on recognition completely disappeared when differences in musical sophistication were controlled for. Participants' working memory capacities did not seem to predict their recognition performances. Our behavioral results extend the studies oriented to understand the processing of brief timbres under temporal constraint by suggesting that the musical sophistication may play a larger role than previously thought. These results can also provide a working hypothesis for future research, namely, that underlying neural mechanisms for the processing of various sound sources may have different temporal constraints.

5-HT1A Receptors Alter Temporal Responses to Broadband Vocalizations in the Mouse Inferior Colliculus Through Response Suppression

Neuromodulatory systems may provide information on social context to auditory brain regions, but relatively few studies have assessed the effects of neuromodulation on auditory responses to acoustic social signals. To address this issue, we measured the influence of the serotonergic system on the responses of neurons in a mouse auditory midbrain nucleus, the inferior colliculus (IC), to vocal signals. Broadband vocalizations (BBVs) are human-audible signals produced by mice in distress as well as by female mice in opposite-sex interactions. The production of BBVs is context-dependent in that they are produced both at early stages of interactions as females physically reject males and at later stages as males mount females. Serotonin in the IC of males corresponds to these events, and is elevated more in males that experience less female rejection. We measured the responses of single IC neurons to five recorded examples of BBVs in anesthetized mice. We then locally activated the 5-HT1A receptor through iontophoretic application of 8-OH-DPAT. IC neurons showed little selectivity for different BBVs, but spike trains were characterized by local regions of high spike probability, which we called "response features." Response features varied across neurons and also across calls for individual neurons, ranging from 1 to 7 response features for responses of single neurons to single calls. 8-OH-DPAT suppressed spikes and also reduced the numbers of response features. The weakest response features were the most likely to disappear, suggestive of an "iceberg"-like effect in which activation of the 5-HT1A receptor suppressed weakly suprathreshold response features below the spiking threshold. Because serotonin in the IC is more likely to be elevated for mounting-associated BBVs than for rejection-associated BBVs, these effects of the 5-HT1A receptor could contribute to the differential auditory processing of BBVs in different behavioral subcontexts.

Music-selective neural populations arise without musical training

Recent work has shown that human auditory cortex contains neural populations anterior and posterior to primary auditory cortex that respond selectively to music. However, it is unknown how this selectivity for music arises. To test whether musical training is necessary, we measured fMRI responses to 192 natural sounds in 10 people with almost no musical training. When voxel responses were decomposed into underlying components, this group exhibited a music-selective component that was very similar in response profile and anatomical distribution to that previously seen in individuals with moderate musical training. We also found that musical genres that were less familiar to our participants (e.g., Balinese gamelan) produced strong responses within the music component, as did drum clips with rhythm but little melody, suggesting that these neural populations are broadly responsive to music as a whole. Our findings demonstrate that the signature properties of neural music selectivity do not require musical training to develop, showing that the music-selective neural populations are a fundamental and widespread property of the human brain.NEW & NOTEWORTHY We show that music-selective neural populations are clearly present in people without musical training, demonstrating that they are a fundamental and widespread property of the human brain. Additionally, we show music-selective neural populations respond strongly to music from unfamiliar genres as well as music with rhythm but little pitch information, suggesting that they are broadly responsive to music as a whole.

"Mickey Mousing" in the Brain: Motion-Sound Synesthesia and the Subcortical Substrate of Audio-Visual Integration

Motion-sound synesthesia is characterized by illusory auditory sensations linked to the pattern and rhythms of motion (dubbed "Mickey Mousing" as in cinema) of visually experienced but soundless object, like an optical flow array, a ball bouncing or a horse galloping. In an MRI study with a group of three synesthetes and a group of eighteen control participants, we found structural changes in the brains of synesthetes in the subcortical multisensory areas of the superior and inferior colliculi. In addition, functional magnetic resonance imaging data showed activity in motion-sensitive regions, as well as temporal and occipital areas, and the cerebellum. However, the synesthetes had a higher activation within the left and right cuneus, with stronger activations when viewing optical flow stimuli. There was also a general difference in connectivity of the colliculi with the above mentioned regions between the two groups. These findings implicate low-level mechanisms within the human neuroaxis as a substrate for local connectivity and cross activity between perceptual processes that are "distant" in terms of cortical topography. The present findings underline the importance of considering the role of subcortical systems and their connectivity to multimodal regions of the cortex and they strengthen a parsimonious account of synesthesia, at the least of the visual-auditory type.

Binaural detection as a joint function of masker bandwidth, masker interaural correlation, and interaural time delay: Empirical data and modeling

Empirical data are reported demonstrating how binaural detection is affected by joint variation of masker bandwidth, masker interaural correlation, and interaural time delay (ITD) of both masker and tonal signal. Most of the data were obtained with stimuli centered at 500 Hz; supplemental measures were obtained with stimuli centered at 4 kHz. The results indicate that as the interaural correlation of the masker (ρ) is decreased there is (1) an overall increase in threshold signal-to-noise ratio (S/N) and (2) a progressively smaller effect on threshold S/N as ITD is increased. All of the data were accounted for quite accurately using the same quantitative, interaural cross-correlation-based model that was recently shown to account for binaural detection and discrimination data obtained in previous experiments. Importantly, the new data were predicted and explained using values of model parameters that were identical or very close to those found to predict accurately the earlier data. The success of the enterprise attests to the robustness of the approach and the generality of the model's ability to make accurate predictions of binaural performance over a wide range of historically important stimulus conditions.

Frequency response areas of neurons in the mouse inferior colliculus. III. Time-domain responses: Constancy, dynamics, and precision in relation to spectral resolution, and perception in the time domain

The auditory midbrain (central nucleus of inferior colliculus, ICC) receives multiple brainstem projections and recodes auditory information for perception in higher centers. Many neural response characteristics are represented in gradients (maps) in the three-dimensional ICC space. Map overlap suggests that neurons, depending on their ICC location, encode information in several domains simultaneously by different aspects of their responses. Thus, interdependence of coding, e.g. in spectral and temporal domains, seems to be a general ICC principle. Studies on covariation of response properties and possible impact on sound perception are, however, rare. Here, we evaluated tone-evoked single neuron activity from the mouse ICC and compared shapes of excitatory frequency-response areas (including strength and shape of inhibition within and around the excitatory area; classes I, II, III) with types of temporal response patterns and first-spike response latencies. Analyses showed covariation of sharpness of frequency tuning with constancy and precision of responding to tone onsets. Highest precision (first-spike latency jitter < 1 ms) and stable phasic responses throughout frequency-response areas were the quality mainly of class III neurons with broad frequency tuning, least influenced by inhibition. Class II neurons with narrow frequency tuning and dominating inhibitory influence were unsuitable for time domain coding with high precision. The ICC center seems specialized rather for high spectral resolution (class II presence), lateral parts for constantly precise responding to sound onsets (class III presence). Further, the variation of tone-response latencies in the frequency-response areas of individual neurons with phasic, tonic, phasic-tonic, or pauser responses gave rise to the definition of a core area, which represented a time window of about 20 ms from tone onset for tone-onset responding of the whole ICC. This time window corresponds to the roughly 20 ms shortest time interval that was found critical in several auditory perceptual tasks in humans and mice.

Auditory processing in the zebra finch midbrain: single unit responses and effect of rearing experience

In birds the auditory system plays a key role in providing the sensory input used to discriminate between conspecific and heterospecific vocal signals. In those species that are known to learn their vocalizations, for example, songbirds, it is generally considered that this ability arises and is manifest in the forebrain, although there is no a priori reason why brainstem components of the auditory system could not also play an important part. To test this assumption, we used groups of normal reared and cross-fostered zebra finches that had previously been shown in behavioural experiments to reduce their preference for conspecific songs subsequent to cross fostering experience with Bengalese finches, a related species with a distinctly different song. The question we asked, therefore, is whether this experiential change also changes the bias in favour of conspecific song displayed by auditory midbrain units of normally raised zebra finches. By recording the responses of single units in MLd to a variety of zebra finch and Bengalese finch songs in both normally reared and cross-fostered zebra finches, we provide a positive answer to this question. That is, the difference in response to conspecific and heterospecific songs seen in normal reared zebra finches is reduced following cross-fostering. In birds the virtual absence of mammalian-like cortical projections upon auditory brainstem nuclei argues against the interpretation that MLd units change, as observed in the present experiments, as a result of top-down influences on sensory processing. Instead, it appears that MLd units can be influenced significantly by sensory inputs arising directly from a change in auditory experience during development.

Spiking network optimized for word recognition in noise predicts auditory system hierarchy

The auditory neural code is resilient to acoustic variability and capable of recognizing sounds amongst competing sound sources, yet, the transformations enabling noise robust abilities are largely unknown. We report that a hierarchical spiking neural network (HSNN) optimized to maximize word recognition accuracy in noise and multiple talkers predicts organizational hierarchy of the ascending auditory pathway. Comparisons with data from auditory nerve, midbrain, thalamus and cortex reveals that the optimal HSNN predicts several transformations of the ascending auditory pathway including a sequential loss of temporal resolution and synchronization ability, increasing sparseness, and selectivity. The optimal organizational scheme enhances performance by selectively filtering out noise and fast temporal cues such as voicing periodicity, that are not directly relevant to the word recognition task. An identical network arranged to enable high information transfer fails to predict auditory pathway organization and has substantially poorer performance. Furthermore, conventional single-layer linear and nonlinear receptive field networks that capture the overall feature extraction of the HSNN fail to achieve similar performance. The findings suggest that the auditory pathway hierarchy and its sequential nonlinear feature extraction computations enhance relevant cues while removing non-informative sources of noise, thus enhancing the representation of sounds in noise impoverished conditions.

The brain’s ability to recognize sounds in the presence of competing sounds or background noise is essential for everyday hearing tasks. How the brain accomplishes noise resiliency, however, is poorly understood. Using neural recordings from the ascending auditory pathway and an auditory spiking network model trained for sound recognition in noise we explore the computational strategies that enable noise robustness. Our results suggest that the hierarchical feature organization of the ascending auditory pathway and the resulting computations are critical for sound recognition in the presence of noise.

Ensemble modeling of auditory streaming reveals potential sources of bistability across the perceptual hierarchy

Perceptual bistability-the spontaneous, irregular fluctuation of perception between two interpretations of a stimulus-occurs when observing a large variety of ambiguous stimulus configurations. This phenomenon has the potential to serve as a tool for, among other things, understanding how function varies across individuals due to the large individual differences that manifest during perceptual bistability. Yet it remains difficult to interpret the functional processes at work, without knowing where bistability arises during perception. In this study we explore the hypothesis that bistability originates from multiple sources distributed across the perceptual hierarchy. We develop a hierarchical model of auditory processing comprised of three distinct levels: a Peripheral, tonotopic analysis, a Central analysis computing features found more centrally in the auditory system, and an Object analysis, where sounds are segmented into different streams. We model bistable perception within this system by applying adaptation, inhibition and noise into one or all of the three levels of the hierarchy. We evaluate a large ensemble of variations of this hierarchical model, where each model has a different configuration of adaptation, inhibition and noise. This approach avoids the assumption that a single configuration must be invoked to explain the data. Each model is evaluated based on its ability to replicate two hallmarks of bistability during auditory streaming: the selectivity of bistability to specific stimulus configurations, and the characteristic log-normal pattern of perceptual switches. Consistent with a distributed origin, a broad range of model parameters across this hierarchy lead to a plausible form of perceptual bistability.

Distinct sensitivity to spectrotemporal modulation supports brain asymmetry for speech and melody

Does brain asymmetry for speech and music emerge from acoustical cues or from domain-specific neural networks? We selectively filtered temporal or spectral modulations in sung speech stimuli for which verbal and melodic content was crossed and balanced. Perception of speech decreased only with degradation of temporal information, whereas perception of melodies decreased only with spectral degradation. Functional magnetic resonance imaging data showed that the neural decoding of speech and melodies depends on activity patterns in left and right auditory regions, respectively. This asymmetry is supported by specific sensitivity to spectrotemporal modulation rates within each region. Finally, the effects of degradation on perception were paralleled by their effects on neural classification. Our results suggest a match between acoustical properties of communicative signals and neural specializations adapted to that purpose.

GABAA receptors contribute more to rate than temporal coding in the IC of awake mice

Speech is our most important form of communication, yet we have a poor understanding of how communication sounds are processed by the brain. Mice make great model organisms to study neural processing of communication sounds because of their rich repertoire of social vocalizations and because they have brain structures analogous to humans, such as the auditory midbrain nucleus inferior colliculus (IC). Although the combined roles of GABAergic and glycinergic inhibition on vocalization selectivity in the IC have been studied to a limited degree, the discrete contributions of GABAergic inhibition have only rarely been examined. In this study, we examined how GABAergic inhibition contributes to shaping responses to pure tones as well as selectivity to complex sounds in the IC of awake mice. In our set of long-latency neurons, we found that GABAergic inhibition extends the evoked firing rate range of IC neurons by lowering the baseline firing rate but maintaining the highest probability of firing rate. GABAergic inhibition also prevented IC neurons from bursting in a spontaneous state. Finally, we found that although GABAergic inhibition shaped the spectrotemporal response to vocalizations in a nonlinear fashion, it did not affect the neural code needed to discriminate vocalizations, based either on spiking patterns or on firing rate. Overall, our results emphasize that even if GABAergic inhibition generally decreases the firing rate, it does so while maintaining or extending the abilities of neurons in the IC to code the wide variety of sounds that mammals are exposed to in their daily lives.NEW & NOTEWORTHY GABAergic inhibition adds nonlinearity to neuronal response curves. This increases the neuronal range of evoked firing rate by reducing baseline firing. GABAergic inhibition prevents bursting responses from neurons in a spontaneous state, reducing noise in the temporal coding of the neuron. This could result in improved signal transmission to the cortex.

Pitch of harmonic complex tones: rate and temporal coding of envelope repetition rate in inferior colliculus of unanesthetized rabbits

Harmonic complex tones (HCTs) found in speech, music, and animal vocalizations evoke strong pitch percepts at their fundamental frequencies. The strongest pitches are produced by HCTs that contain harmonics resolved by cochlear frequency analysis, but HCTs containing solely unresolved harmonics also evoke a weaker pitch at their envelope repetition rate (ERR). In the auditory periphery, neurons phase lock to the stimulus envelope, but this temporal representation of ERR degrades and gives way to rate codes along the ascending auditory pathway. To assess the role of the inferior colliculus (IC) in such transformations, we recorded IC neuron responses to HCT and sinusoidally modulated broadband noise (SAMN) with varying ERR from unanesthetized rabbits. Different interharmonic phase relationships of HCT were used to manipulate the temporal envelope without changing the power spectrum. Many IC neurons demonstrated band-pass rate tuning to ERR between 60 and 1,600 Hz for HCT and between 40 and 500 Hz for SAMN. The tuning was not related to the pure-tone best frequency of neurons but was dependent on the shape of the stimulus envelope, indicating a temporal rather than spectral origin. A phenomenological model suggests that the tuning may arise from peripheral temporal response patterns via synaptic inhibition. We also characterized temporal coding to ERR. Some IC neurons could phase lock to the stimulus envelope up to 900 Hz for either HCT or SAMN, but phase locking was weaker with SAMN. Together, the rate code and the temporal code represent a wide range of ERR, providing strong cues for the pitch of unresolved harmonics.NEW & NOTEWORTHY Envelope repetition rate (ERR) provides crucial cues for pitch perception of frequency components that are not individually resolved by the cochlea, but the neural representation of ERR for stimuli containing many harmonics is poorly characterized. Here we show that the pitch of stimuli with unresolved harmonics is represented by both a rate code and a temporal code for ERR in auditory midbrain neurons and propose possible underlying neural mechanisms with a computational model.

A neural ensemble correlation code for sound category identification

Humans and other animals effortlessly identify natural sounds and categorize them into behaviorally relevant categories. Yet, the acoustic features and neural transformations that enable sound recognition and the formation of perceptual categories are largely unknown. Here, using multichannel neural recordings in the auditory midbrain of unanesthetized female rabbits, we first demonstrate that neural ensemble activity in the auditory midbrain displays highly structured correlations that vary with distinct natural sound stimuli. These stimulus-driven correlations can be used to accurately identify individual sounds using single-response trials, even when the sounds do not differ in their spectral content. Combining neural recordings and an auditory model, we then show how correlations between frequency-organized auditory channels can contribute to discrimination of not just individual sounds but sound categories. For both the model and neural data, spectral and temporal correlations achieved similar categorization performance and appear to contribute equally. Moreover, both the neural and model classifiers achieve their best task performance when they accumulate evidence over a time frame of approximately 1-2 seconds, mirroring human perceptual trends. These results together suggest that time-frequency correlations in sounds may be reflected in the correlations between auditory midbrain ensembles and that these correlations may play an important role in the identification and categorization of natural sounds.

A temporal integration mechanism enhances frequency selectivity of broadband inputs to inferior colliculus

Accurately resolving frequency components in sounds is essential for sound recognition, yet there is little direct evidence for how frequency selectivity is preserved or newly created across auditory structures. We demonstrate that prepotentials (PPs) with physiological properties resembling presynaptic potentials from broadly tuned brainstem inputs can be recorded concurrently with postsynaptic action potentials in inferior colliculus (IC). These putative brainstem inputs (PBIs) are broadly tuned and exhibit delayed and spectrally interleaved excitation and inhibition not present in the simultaneously recorded IC neurons (ICNs). A sharpening of tuning is accomplished locally at the expense of spike-timing precision through nonlinear temporal integration of broadband inputs. A neuron model replicates the finding and demonstrates that temporal integration alone can degrade timing precision while enhancing frequency tuning through interference of spectrally in- and out-of-phase inputs. These findings suggest that, in contrast to current models that require local inhibition, frequency selectivity can be sharpened through temporal integration, thus supporting an alternative computational strategy to quickly refine frequency selectivity.

A novel class of inferior colliculus principal neurons labeled in vasoactive intestinal peptide-Cre mice

Located in the midbrain, the inferior colliculus (IC) is the hub of the central auditory system. Although the IC plays important roles in speech processing, sound localization, and other auditory computations, the organization of the IC microcircuitry remains largely unknown. Using a multifaceted approach in mice, we have identified vasoactive intestinal peptide (VIP) neurons as a novel class of IC principal neurons. VIP neurons are glutamatergic stellate cells with sustained firing patterns. Their extensive axons project to long-range targets including the auditory thalamus, auditory brainstem, superior colliculus, and periaqueductal gray. Using optogenetic circuit mapping, we found that VIP neurons integrate input from the contralateral IC and the dorsal cochlear nucleus. The dorsal cochlear nucleus also drove feedforward inhibition to VIP neurons, indicating that inhibitory circuits within the IC shape the temporal integration of ascending inputs. Thus, VIP neurons are well-positioned to influence auditory computations in a number of brain regions.

No more than "slight" hearing loss and degradations in binaural processing

Listeners having, at most, "slight" hearing loss may exhibit substantial deficits in binaural detection [Bernstein and Trahiotis. (2016). J. Acoust. Soc. Am. 140, 3540-3548; (2018). J. Acoust. Soc. Am. 144, 292-307]. This study assessed whether such listeners also exhibit deficits discriminating interaural temporal disparities (ITDs) or interaural intensitive disparities (IIDs) and whether any deficits observed in those discrimination tasks would be accounted for by the interaural cross-correlation based model that successfully accounts for binaural detection. Thresholds were measured for detection of tones masked by noise in the NoSπ configuration and discrimination of ITD or IID. Gaussian noises (100 Hz-wide), served as maskers in the detection task and as reference and target stimuli in the discrimination tasks. Stimuli were centered at 500 Hz or 4 kHz. The latter were transpositions of stimuli centered at 125 Hz. Results demonstrate that listeners having, at most, slight hearing loss and who exhibit deficits in binaural detection, also exhibit deficits in ITD- and IID-discrimination. Coupled with appropriate decision variables, the cross-correlation-based model that accounts for elevated binaural detection thresholds among such listeners also accounted for their elevated ITD- and IID-thresholds. The deficits in all three tasks appear to stem from increased levels of stimulus-dependent, additive internal noise.

A hierarchical sparse coding model predicts acoustic feature encoding in both auditory midbrain and cortex

The auditory pathway consists of multiple stages, from the cochlear nucleus to the auditory cortex. Neurons acting at different stages have different functions and exhibit different response properties. It is unclear whether these stages share a common encoding mechanism. We trained an unsupervised deep learning model consisting of alternating sparse coding and max pooling layers on cochleogram-filtered human speech. Evaluation of the response properties revealed that computing units in lower layers exhibited spectro-temporal receptive fields (STRFs) similar to those of inferior colliculus neurons measured in physiological experiments, including properties such as sound onset and termination, checkerboard pattern, and spectral motion. Units in upper layers tended to be tuned to phonetic features such as plosivity and nasality, resembling the results of field recording in human auditory cortex. Variation of the sparseness level of the units in each higher layer revealed a positive correlation between the sparseness level and the strength of phonetic feature encoding. The activities of the units in the top layer, but not other layers, correlated with the dynamics of the first two formants (F1, F2) of all phonemes, indicating the encoding of phoneme dynamics in these units. These results suggest that the principles of sparse coding and max pooling may be universal in the human auditory pathway.

When speech enters the ear, it is subjected to a series of processing stages prior to arriving at the auditory cortex. Neurons acting at different processing stages have different response properties. For example, at the auditory midbrain, a neuron may specifically detect the onsets of a frequency component in the speech, whereas in the auditory cortex, a neuron may specifically detect phonetic features. The encoding mechanisms underlying these neuronal functions remain unclear. To address this issue, we designed a hierarchical sparse coding model, inspired by the sparse activity of neurons in the sensory system, to learn features in speech signals. We found that the computing units in different layers exhibited hierarchical extraction of speech sound features, similar to those of neurons in the auditory midbrain and auditory cortex, although the computational principles in these layers were the same. The results suggest that sparse coding and max pooling represent universal computational principles throughout the auditory pathway.

Evaluating the Columnar Stability of Acoustic Processing in the Human Auditory Cortex

Using ultra-high field fMRI, we explored the cortical depth-dependent stability of acoustic feature preference in human auditory cortex. We collected responses from human auditory cortex (subjects from either sex) to a large number of natural sounds at submillimeter spatial resolution, and observed that these responses were well explained by a model that assumes neuronal population tuning to frequency-specific spectrotemporal modulations. We observed a relatively stable (columnar) tuning to frequency and temporal modulations. However, spectral modulation tuning was variable throughout the cortical depth. This difference in columnar stability between feature maps could not be explained by a difference in map smoothness, as the preference along the cortical sheet varied in a similar manner for the different feature maps. Furthermore, tuning to all three features was more columnar in primary than nonprimary auditory cortex. The observed overall lack of overlapping columnar regions across acoustic feature maps suggests, especially for primary auditory cortex, a coding strategy in which across cortical depths tuning to some features is kept stable, whereas tuning to other features systematically varies.SIGNIFICANCE STATEMENT In the human auditory cortex, sound aspects are processed in large-scale maps. Invasive animal studies show that an additional processing organization may be implemented orthogonal to the cortical sheet (i.e., in the columnar direction), but it is unknown whether observed organizational principles apply to the human auditory cortex. Combining ultra-high field fMRI with natural sounds, we explore the columnar organization of various sound aspects. Our results suggest that the human auditory cortex contains a modular coding strategy, where, for each module, several sound aspects act as an anchor along which computations are performed while the processing of another sound aspect undergoes a transformation. This strategy may serve to optimally represent the content of our complex acoustic natural environment.

Origins of scale invariance in vocalization sequences and speech

To communicate effectively animals need to detect temporal vocalization cues that vary over several orders of magnitude in their amplitude and frequency content. This large range of temporal cues is evident in the power-law scale-invariant relationship between the power of temporal fluctuations in sounds and the sound modulation frequency (f). Though various forms of scale invariance have been described for natural sounds, the origins and implications of scale invariant phenomenon remain unknown. Using animal vocalization sequences, including continuous human speech, and a stochastic model of temporal amplitude fluctuations we demonstrate that temporal acoustic edges are the primary acoustic cue accounting for the scale invariant phenomenon. The modulation spectrum of vocalization sequences and the model both exhibit a dual regime lowpass structure with a flat region at low modulation frequencies and scale invariant 1/f² trend for high modulation frequencies. Moreover, we find a time-frequency tradeoff between the average vocalization duration of each vocalization sequence and the cutoff frequency beyond which scale invariant behavior is observed. These results indicate that temporal edges are universal features responsible for scale invariance in vocalized sounds. This is significant since temporal acoustic edges are salient perceptually and the auditory system could exploit such statistical regularities to minimize redundancies and generate compact neural representations of vocalized sounds.

The efficient coding hypothesis posits that the brain encodes sensory signals efficiently in order to reduce metabolic cost and preserve behaviorally relevant environment information. In audition, recognition and coding depends on the brain’s ability to accurately and efficiently encode statistical regularities that are prevalent in natural sounds. Similarly, efficient audio coding and compression schemes attempt to preserve salient sound qualities while minimizing data bandwidth. A widely observed statistical regularity in nearly all natural sounds is the presence of scale invariance where the power of amplitude fluctuations is inversely related to the sound amplitude modulation frequency. In this study, we explore the physical sound cues responsible for the scale invariant phenomenon previously observed. We demonstrate that for animal vocalizations, including human speech, the scale invariant behavior is fully accounted by the presence of temporal acoustic edges that are largely created by opening and closing of the oral cavity and which mark the beginning and end of isolated vocalizations. The findings thus identify a single physical cue responsible for the universal scale invariant phenomenon that the brain can exploit to optimize coding and perception of vocalized sounds.

Responses of neurons in the feline inferior colliculus to modulated electrical stimuli applied on and within the ventral cochlear nucleus; Implications for an advanced auditory brainstem implant

Auditory brainstem implants (ABIs) can restore useful hearing to persons with deafness who cannot benefit from cochlear implants. However, the quality of hearing restored by ABIs rarely is comparable to that provided by cochlear implants in persons for whom those are appropriate. In an animal model, we evaluated elements of a prototype of an ABI in which the functions of macroelectrodes on the surface of the dorsal cochlear nucleus would be integrated with the function of multiple penetrating microelectrodes implanted into the ventral cochlear nucleus. The surface electrodes would convey most of the range of loudness percepts while the intranuclear microelectrodes would sharpen and focus pitch percepts. In the present study, stimulating electrodes were implanted chronically on the surface of the animal's dorsal cochlear nucleus (DCN) and also within their ventral cochlear nucleus (VCN). Recording microelectrodes were implanted into the central nucleus of the inferior colliculus (ICC). The electrical stimuli were sinusoidally modulated stimulus pulse trains applied on the DCN and within the VCN. Temporal encoding of neuronal responses was quantified as vector strength (VS) and as full-cycle rate of neuronal activity in the ICC. VS and full-cycle AP rate were measured for 4 stimulation modes; continuous and transient amplitude modulation of the stimulus pulse trains, each delivered via the macroelectrode on the surface of the DCN and then by the intranuclear penetrating microelectrodes. In the proposed clinical device the functions of the surface and intranuclear microelectrodes could best be integrated if there is minimal variation in the neuronal responses across the range of modulation depth, modulation frequencies, and across the four stimulation modes. In this study VS did vary as much as 34% across modulation frequency and modulation depth within a stimulation mode, and up to 40% between modulation modes. However, these intra- and inter-mode variances differed for different stimulation rates, and at 500 Hz the inter-mode differences in VS and across the range of modulation frequencies and modulation depths was<Roman> = </Roman>24% and the intra-modal differences were<Roman> = </Roman>15%. The findings were generally similar for rate encoding of modulation depth, although the depth of transient amplitude modulation delivered by the surface electrode was weakly encoded as full-cycle rate. Overall, our findings support the concept of a clinical ABI that employs surface stimulation and intranuclear microstimulation in an integrated manner.

Cluster-based analysis improves predictive validity of spike-triggered receptive field estimates

Spectrotemporal receptive field (STRF) characterization is a central goal of auditory physiology. STRFs are often approximated by the spike-triggered average (STA), which reflects the average stimulus preceding a spike. In many cases, the raw STA is subjected to a threshold defined by gain values expected by chance. However, such correction methods have not been universally adopted, and the consequences of specific gain-thresholding approaches have not been investigated systematically. Here, we evaluate two classes of statistical correction techniques, using the resulting STRF estimates to predict responses to a novel validation stimulus. The first, more traditional technique eliminated STRF pixels (time-frequency bins) with gain values expected by chance. This correction method yielded significant increases in prediction accuracy, including when the threshold setting was optimized for each unit. The second technique was a two-step thresholding procedure wherein clusters of contiguous pixels surviving an initial gain threshold were then subjected to a cluster mass threshold based on summed pixel values. This approach significantly improved upon even the best gain-thresholding techniques. Additional analyses suggested that allowing threshold settings to vary independently for excitatory and inhibitory subfields of the STRF resulted in only marginal additional gains, at best. In summary, augmenting reverse correlation techniques with principled statistical correction choices increased prediction accuracy by over 80% for multi-unit STRFs and by over 40% for single-unit STRFs, furthering the interpretational relevance of the recovered spectrotemporal filters for auditory systems analysis.

Reconstructing the spectrotemporal modulations of real-life sounds from fMRI response patterns

Ethological views of brain functioning suggest that sound representations and computations in the auditory neural system are optimized finely to process and discriminate behaviorally relevant acoustic features and sounds (e.g., spectrotemporal modulations in the songs of zebra finches). Here, we show that modeling of neural sound representations in terms of frequency-specific spectrotemporal modulations enables accurate and specific reconstruction of real-life sounds from high-resolution functional magnetic resonance imaging (fMRI) response patterns in the human auditory cortex. Region-based analyses indicated that response patterns in separate portions of the auditory cortex are informative of distinctive sets of spectrotemporal modulations. Most relevantly, results revealed that in early auditory regions, and progressively more in surrounding regions, temporal modulations in a range relevant for speech analysis (∼2-4 Hz) were reconstructed more faithfully than other temporal modulations. In early auditory regions, this effect was frequency-dependent and only present for lower frequencies (<∼2 kHz), whereas for higher frequencies, reconstruction accuracy was higher for faster temporal modulations. Further analyses suggested that auditory cortical processing optimized for the fine-grained discrimination of speech and vocal sounds underlies this enhanced reconstruction accuracy. In sum, the present study introduces an approach to embed models of neural sound representations in the analysis of fMRI response patterns. Furthermore, it reveals that, in the human brain, even general purpose and fundamental neural processing mechanisms are shaped by the physical features of real-world stimuli that are most relevant for behavior (i.e., speech, voice).

Characteristics of spectro-temporal modulation frequency selectivity in humans

There is increasing evidence that the auditory system shows frequency selectivity for spectro-temporal modulations. A recent study of the authors has shown spectro-temporal modulation masking patterns that were in agreement with the hypothesis of spectro-temporal modulation filters in the human auditory system [Oetjen and Verhey (2015). J. Acoust. Soc. Am. 137(2), 714-723]. In the present study, that experimental data and additional data were used to model this spectro-temporal frequency selectivity. The additional data were collected to investigate to what extent the spectro-temporal modulation-frequency selectivity results from a combination of a purely temporal amplitude-modulation filter and a purely spectral amplitude-modulation filter. In contrast to the previous study, thresholds were measured for masker and target modulations with opposite directions, i.e., an upward pointing target modulation and a downward pointing masker modulation. The comparison of this data set with previous corresponding data with the same direction from target and masker modulations indicate that a specific spectro-temporal modulation filter is required to simulate all aspects of spectro-temporal modulation frequency selectivity. A model using a modified Gabor filter with a purely temporal and a purely spectral filter predicts the spectro-temporal modulation masking data.

Synchrony, connectivity, and functional similarity in auditory midbrain local circuits

The central nucleus of the inferior colliculus (ICC) contains a laminar structure that functions as an organizing substrate of ascending inputs and local processing. While topographic distributions of ICC response parameters within and across laminae have been reported, the functional micro-organization of the ICC is less well understood. For pairs of neighboring ICC neurons, we examined the nature of functional connectivity and receptive field preferences to gain a better understanding of the structure and function of local circuits. By recording from pairs of adjacent neurons and presenting pure-tone and dynamic broad-band stimulation, we estimated functional connectivity and local differences in frequency response areas (FRAs), spectrotemporal receptive fields (STRFs), nonlinear input/output functions, and single-spike information. From the cross-covariance functions we identified putative unidirectional as well as bidirectional excitatory/inhibitory interactions. STRFs of neighboring neurons strongly conserve best frequency, and moderately agree in STRF similarity, bandwidth, temporal response type, best modulation frequency, nonlinearity structure, and degree of information processing. Excitatory connectivity was stronger and temporally more precise than for inhibitory connections. Neither connection strength nor degree of synchrony correlated with receptive field parameters. The functional similarity of local pairs of ICC neurons was substantially less than for local pairs in the granular layers of primary auditory cortex (AI). These results imply that while the ICC is an obligatory nexus of ascending information, local neurons are comparatively weakly connected and exhibit considerable receptive field variability, potentially reflecting the heterogeneity of converging inputs to ICC functional zones.

Human Superior Temporal Gyrus Organization of Spectrotemporal Modulation Tuning Derived from Speech Stimuli

The human superior temporal gyrus (STG) is critical for speech perception, yet the organization of spectrotemporal processing of speech within the STG is not well understood. Here, to characterize the spatial organization of spectrotemporal processing of speech across human STG, we use high-density cortical surface field potential recordings while participants listened to natural continuous speech. While synthetic broad-band stimuli did not yield sustained activation of the STG, spectrotemporal receptive fields could be reconstructed from vigorous responses to speech stimuli. We find that the human STG displays a robust anterior-posterior spatial distribution of spectrotemporal tuning in which the posterior STG is tuned for temporally fast varying speech sounds that have relatively constant energy across the frequency axis (low spectral modulation) while the anterior STG is tuned for temporally slow varying speech sounds that have a high degree of spectral variation across the frequency axis (high spectral modulation). This work illustrates organization of spectrotemporal processing in the human STG, and illuminates processing of ethologically relevant speech signals in a region of the brain specialized for speech perception.

SIGNIFICANCE STATEMENT

Considerable evidence has implicated the human superior temporal gyrus (STG) in speech processing. However, the gross organization of spectrotemporal processing of speech within the STG is not well characterized. Here we use natural speech stimuli and advanced receptive field characterization methods to show that spectrotemporal features within speech are well organized along the posterior-to-anterior axis of the human STG. These findings demonstrate robust functional organization based on spectrotemporal modulation content, and illustrate that much of the encoded information in the STG represents the physical acoustic properties of speech stimuli.

Distinct Cortical Pathways for Music and Speech Revealed by Hypothesis-Free Voxel Decomposition

The organization of human auditory cortex remains unresolved, due in part to the small stimulus sets common to fMRI studies and the overlap of neural populations within voxels. To address these challenges, we measured fMRI responses to 165 natural sounds and inferred canonical response profiles ("components") whose weighted combinations explained voxel responses throughout auditory cortex. This analysis revealed six components, each with interpretable response characteristics despite being unconstrained by prior functional hypotheses. Four components embodied selectivity for particular acoustic features (frequency, spectrotemporal modulation, pitch). Two others exhibited pronounced selectivity for music and speech, respectively, and were not explainable by standard acoustic features. Anatomically, music and speech selectivity concentrated in distinct regions of non-primary auditory cortex. However, music selectivity was weak in raw voxel responses, and its detection required a decomposition method. Voxel decomposition identifies primary dimensions of response variation across natural sounds, revealing distinct cortical pathways for music and speech.

Neural spike-timing patterns vary with sound shape and periodicity in three auditory cortical fields

Mammals perceive a wide range of temporal cues in natural sounds, and the auditory cortex is essential for their detection and discrimination. The rat primary (A1), ventral (VAF), and caudal suprarhinal (cSRAF) auditory cortical fields have separate thalamocortical pathways that may support unique temporal cue sensitivities. To explore this, we record responses of single neurons in the three fields to variations in envelope shape and modulation frequency of periodic noise sequences. Spike rate, relative synchrony, and first-spike latency metrics have previously been used to quantify neural sensitivities to temporal sound cues; however, such metrics do not measure absolute spike timing of sustained responses to sound shape. To address this, in this study we quantify two forms of spike-timing precision, jitter, and reliability. In all three fields, we find that jitter decreases logarithmically with increase in the basis spline (B-spline) cutoff frequency used to shape the sound envelope. In contrast, reliability decreases logarithmically with increase in sound envelope modulation frequency. In A1, jitter and reliability vary independently, whereas in ventral cortical fields, jitter and reliability covary. Jitter time scales increase (A1 < VAF < cSRAF) and modulation frequency upper cutoffs decrease (A1 > VAF > cSRAF) with ventral progression from A1. These results suggest a transition from independent encoding of shape and periodicity sound cues on short time scales in A1 to a joint encoding of these same cues on longer time scales in ventral nonprimary cortices.

Natural Vocalizations in the Mammalian Inferior Colliculus are Broadly Encoded by a Small Number of Independent Multi-Units

How complex natural sounds are represented by the main converging center of the auditory midbrain, the central inferior colliculus, is an open question. We applied neural discrimination to determine the variation of detailed encoding of individual vocalizations across the best frequency gradient of the central inferior colliculus. The analysis was based on collective responses from several neurons. These multi-unit spike trains were recorded from guinea pigs exposed to a spectrotemporally rich set of eleven species-specific vocalizations. Spike trains of disparate units from the same recording were combined in order to investigate whether groups of multi-unit clusters represent the whole set of vocalizations more reliably than only one unit, and whether temporal response correlations between them facilitate an unambiguous neural representation of the vocalizations. We found a spatial distribution of the capability to accurately encode groups of vocalizations across the best frequency gradient. Different vocalizations are optimally discriminated at different locations of the best frequency gradient. Furthermore, groups of a few multi-unit clusters yield improved discrimination over only one multi-unit cluster between all tested vocalizations. However, temporal response correlations between units do not yield better discrimination. Our study is based on a large set of units of simultaneously recorded responses from several guinea pigs and electrode insertion positions. Our findings suggest a broadly distributed code for behaviorally relevant vocalizations in the mammalian inferior colliculus. Responses from a few non-interacting units are sufficient to faithfully represent the whole set of studied vocalizations with diverse spectrotemporal properties.

Periodotopy in the gerbil inferior colliculus: local clustering rather than a gradient map

Periodicities in sound waveforms are widespread, and shape important perceptual attributes of sound including rhythm and pitch. Previous studies have indicated that, in the inferior colliculus (IC), a key processing stage in the auditory midbrain, neurons tuned to different periodicities might be arranged along a periodotopic axis which runs approximately orthogonal to the tonotopic axis. Here we map out the topography of frequency and periodicity tuning in the IC of gerbils in unprecedented detail, using pure tones and different periodic sounds, including click trains, sinusoidally amplitude modulated (SAM) noise and iterated rippled noise. We found that while the tonotopic map exhibited a clear and highly reproducible gradient across all animals, periodotopic maps varied greatly across different types of periodic sound and from animal to animal. Furthermore, periodotopic gradients typically explained only about 10% of the variance in modulation tuning between recording sites. However, there was a strong local clustering of periodicity tuning at a spatial scale of ca. 0.5 mm, which also differed from animal to animal.

Idealized computational models for auditory receptive fields

We present a theory by which idealized models of auditory receptive fields can be derived in a principled axiomatic manner, from a set of structural properties to (i) enable invariance of receptive field responses under natural sound transformations and (ii) ensure internal consistency between spectro-temporal receptive fields at different temporal and spectral scales. For defining a time-frequency transformation of a purely temporal sound signal, it is shown that the framework allows for a new way of deriving the Gabor and Gammatone filters as well as a novel family of generalized Gammatone filters, with additional degrees of freedom to obtain different trade-offs between the spectral selectivity and the temporal delay of time-causal temporal window functions. When applied to the definition of a second-layer of receptive fields from a spectrogram, it is shown that the framework leads to two canonical families of spectro-temporal receptive fields, in terms of spectro-temporal derivatives of either spectro-temporal Gaussian kernels for non-causal time or a cascade of time-causal first-order integrators over the temporal domain and a Gaussian filter over the logspectral domain. For each filter family, the spectro-temporal receptive fields can be either separable over the time-frequency domain or be adapted to local glissando transformations that represent variations in logarithmic frequencies over time. Within each domain of either non-causal or time-causal time, these receptive field families are derived by uniqueness from the assumptions. It is demonstrated how the presented framework allows for computation of basic auditory features for audio processing and that it leads to predictions about auditory receptive fields with good qualitative similarity to biological receptive fields measured in the inferior colliculus (ICC) and primary auditory cortex (A1) of mammals.

Rodent auditory perception: Critical band limitations and plasticity

What do animals hear? While it remains challenging to adequately assess sensory perception in animal models, it is important to determine perceptual abilities in model systems to understand how physiological processes and plasticity relate to perception, learning, and cognition. Here we discuss hearing in rodents, reviewing previous and recent behavioral experiments querying acoustic perception in rats and mice, and examining the relation between behavioral data and electrophysiological recordings from the central auditory system. We focus on measurements of critical bands, which are psychoacoustic phenomena that seem to have a neural basis in the functional organization of the cochlea and the inferior colliculus. We then discuss how behavioral training, brain stimulation, and neuropathology impact auditory processing and perception.

Spectro-temporal modulation masking patterns reveal frequency selectivity

The present study investigated the possibility that the human auditory system demonstrates frequency selectivity to spectro-temporal amplitude modulations. Threshold modulation depth for detecting sinusoidal spectro-temporal modulations was measured using a generalized masked threshold pattern paradigm with narrowband masker modulations. Four target spectro-temporal modulations were examined, differing in their temporal and spectral modulation frequencies: a temporal modulation of -8, 8, or 16 Hz combined with a spectral modulation of 1 cycle/octave and a temporal modulation of 4 Hz combined with a spectral modulation of 0.5 cycles/octave. The temporal center frequencies of the masker modulation ranged from 0.25 to 4 times the target temporal modulation. The spectral masker-modulation center-frequencies were 0, 0.5, 1, 1.5, and 2 times the target spectral modulation. For all target modulations, the pattern of average thresholds for the eight normal-hearing listeners was consistent with the hypothesis of a spectro-temporal modulation filter. Such a pattern of modulation-frequency sensitivity was predicted on the basis of psychoacoustical data for purely temporal amplitude modulations and purely spectral amplitude modulations. An analysis of separability indicates that, for the present data set, selectivity in the spectro-temporal modulation domain can be described by a combination of a purely spectral and a purely temporal modulation filter function.

A high-density, high-channel count, multiplexed μECoG array for auditory-cortex recordings

Our understanding of the large-scale population dynamics of neural activity is limited, in part, by our inability to record simultaneously from large regions of the cortex. Here, we validated the use of a large-scale active microelectrode array that simultaneously records 196 multiplexed micro-electrocortigraphical (μECoG) signals from the cortical surface at a very high density (1,600 electrodes/cm(2)). We compared μECoG measurements in auditory cortex using a custom "active" electrode array to those recorded using a conventional "passive" μECoG array. Both of these array responses were also compared with data recorded via intrinsic optical imaging, which is a standard methodology for recording sound-evoked cortical activity. Custom active μECoG arrays generated more veridical representations of the tonotopic organization of the auditory cortex than current commercially available passive μECoG arrays. Furthermore, the cortical representation could be measured efficiently with the active arrays, requiring as little as 13.5 s of neural data acquisition. Next, we generated spectrotemporal receptive fields from the recorded neural activity on the active μECoG array and identified functional organizational principles comparable to those observed using intrinsic metabolic imaging and single-neuron recordings. This new electrode array technology has the potential for large-scale, temporally precise monitoring and mapping of the cortex, without the use of invasive penetrating electrodes.

Emergence of band-pass filtering through adaptive spiking in the owl's cochlear nucleus

In the visual, auditory, and electrosensory modalities, stimuli are defined by first- and second-order attributes. The fast time-pressure signal of a sound, a first-order attribute, is important, for instance, in sound localization and pitch perception, while its slow amplitude-modulated envelope, a second-order attribute, can be used for sound recognition. Ascending the auditory pathway from ear to midbrain, neurons increasingly show a preference for the envelope and are most sensitive to particular envelope modulation frequencies, a tuning considered important for encoding sound identity. The level at which this tuning property emerges along the pathway varies across species, and the mechanism of how this occurs is a matter of debate. In this paper, we target the transition between auditory nerve fibers and the cochlear nucleus angularis (NA). While the owl's auditory nerve fibers simultaneously encode the fast and slow attributes of a sound, one synapse further, NA neurons encode the envelope more efficiently than the auditory nerve. Using in vivo and in vitro electrophysiology and computational analysis, we show that a single-cell mechanism inducing spike threshold adaptation can explain the difference in neural filtering between the two areas. We show that spike threshold adaptation can explain the increased selectivity to modulation frequency, as input level increases in NA. These results demonstrate that a spike generation nonlinearity can modulate the tuning to second-order stimulus features, without invoking network or synaptic mechanisms.

Noise-Robust Speech Recognition Through Auditory Feature Detection and Spike Sequence Decoding

Speech recognition in noisy conditions is a major challenge for computer systems, but the human brain performs it routinely and accurately. Automatic speech recognition (ASR) systems that are inspired by neuroscience can potentially bridge the performance gap between humans and machines. We present a system for noise-robust isolated word recognition that works by decoding sequences of spikes from a population of simulated auditory feature-detecting neurons. Each neuron is trained to respond selectively to a brief spectrotemporal pattern, or feature, drawn from the simulated auditory nerve response to speech. The neural population conveys the time-dependent structure of a sound by its sequence of spikes. We compare two methods for decoding the spike sequences—one using a hidden Markov model–based recognizer, the other using a novel template-based recognition scheme. In the latter case, words are recognized by comparing their spike sequences to template sequences obtained from clean training data, using a similarity measure based on the length of the longest common sub-sequence. Using isolated spoken digits from the AURORA-2 database, we show that our combined system outperforms a state-of-the-art robust speech recognizer at low signal-to-noise ratios. Both the spike-based encoding scheme and the template-based decoding offer gains in noise robustness over traditional speech recognition methods. Our system highlights potential advantages of spike-based acoustic coding and provides a biologically motivated framework for robust ASR development.

Sensitivity to envelope-based interaural delays at high frequencies: center frequency affects the envelope rate-limitation

Sensitivity to ongoing interaural temporal disparities (ITDs) was measured using bandpass-filtered pulse trains centered at 4600, 6500, or 9200 Hz. Save for minor differences in the exact center frequencies, those target stimuli were those employed by Majdak and Laback [J. Acoust. Soc. Am. 125, 3903-3913 (2009)]. At each center frequency, threshold ITD was measured for pulse repetition rates ranging from 64 to 609 Hz. The results and quantitative predictions by a cross-correlation-based model indicated that (1) at most pulse repetition rates, threshold ITD increased with center frequency, (2) the cutoff frequency of the putative envelope low-pass filter that determines sensitivity to ITD at high envelope rates appears to be inversely related to center frequency, and (3) both outcomes were accounted for by assuming that, independent of the center frequency, the listeners' decision variable was a constant criterion change in interaural correlation of the stimuli as processed internally. The finding of an inverse relation between center frequency and the envelope rate limitation, while consistent with much prior literature, runs counter to the conclusion reached by Majdak and Laback.

Encoding of natural sounds at multiple spectral and temporal resolutions in the human auditory cortex

Functional neuroimaging research provides detailed observations of the response patterns that natural sounds (e.g. human voices and speech, animal cries, environmental sounds) evoke in the human brain. The computational and representational mechanisms underlying these observations, however, remain largely unknown. Here we combine high spatial resolution (3 and 7 Tesla) functional magnetic resonance imaging (fMRI) with computational modeling to reveal how natural sounds are represented in the human brain. We compare competing models of sound representations and select the model that most accurately predicts fMRI response patterns to natural sounds. Our results show that the cortical encoding of natural sounds entails the formation of multiple representations of sound spectrograms with different degrees of spectral and temporal resolution. The cortex derives these multi-resolution representations through frequency-specific neural processing channels and through the combined analysis of the spectral and temporal modulations in the spectrogram. Furthermore, our findings suggest that a spectral-temporal resolution trade-off may govern the modulation tuning of neuronal populations throughout the auditory cortex. Specifically, our fMRI results suggest that neuronal populations in posterior/dorsal auditory regions preferably encode coarse spectral information with high temporal precision. Vice-versa, neuronal populations in anterior/ventral auditory regions preferably encode fine-grained spectral information with low temporal precision. We propose that such a multi-resolution analysis may be crucially relevant for flexible and behaviorally-relevant sound processing and may constitute one of the computational underpinnings of functional specialization in auditory cortex.

How does the human brain analyze natural sounds? Previous functional neuroimaging research could only describe the response patterns that sounds evoke in the human brain at the level of preferential regional activations. A comprehensive account of the neural basis of human hearing, however, requires deriving computational models that are able to provide quantitative predictions of brain responses to natural sounds. Here, we make a significant step in this direction by combining functional magnetic resonance imaging (fMRI) with computational modeling. We compare competing computational models of sound representations and select the model that most accurately predicts the measured fMRI response patterns. The computational models describe the processing of three relevant properties of natural sounds: frequency, temporal modulations and spectral modulations. We find that a model that represents spectral and temporal modulations jointly and in a frequency-dependent fashion provides the best account of fMRI responses and that the functional specialization of auditory cortical fields can be partially accounted for by their modulation tuning. Our results provide insights on how natural sounds are encoded in human auditory cortex and our methodological approach constitutes an advance in the way this question can be addressed in future studies.

Primary auditory cortical responses to electrical stimulation of the thalamus

Cochlear implant electrical stimulation of the auditory system to rehabilitate deafness has been remarkably successful. Its deployment requires both an intact auditory nerve and a suitably patent cochlear lumen. When disease renders prerequisite conditions impassable, such as in neurofibromatosis type II and cochlear obliterans, alternative treatment targets are considered. Electrical stimulation of the cochlear nucleus and midbrain in humans has delivered encouraging clinical outcomes, buttressing the promise of central auditory prostheses to mitigate deafness in those who are not candidates for cochlear implantation. In this study we explored another possible implant target: the auditory thalamus. In anesthetized cats, we first presented pure tones to determine frequency preferences of thalamic and cortical sites. We then electrically stimulated tonotopically organized thalamic sites while recording from primary auditory cortical sites using a multichannel recording probe. Cathode-leading biphasic thalamic stimulation thresholds that evoked cortical responses were much lower than published accounts of cochlear and midbrain stimulation. Cortical activation dynamic ranges were similar to those reported for cochlear stimulation, but they were narrower than those found through midbrain stimulation. Our results imply that thalamic stimulation can activate auditory cortex at low electrical current levels and suggest an auditory thalamic implant may be a viable central auditory prosthesis.

Encoding of the amplitude modulation of pulsatile electrical stimulation in the feline cochlear nucleus by neurons in the inferior colliculus; effects of stimulus pulse rate

OBJECTIVES

Persons without a functional auditory nerve cannot benefit from cochlear implants, but some hearing can be restored by an auditory brainstem implant (ABI) with stimulating electrodes implanted on the surface of the cochlear nucleus (CN). Most users benefit from their ABI, but speech recognition tends to be poorer than for users of cochlear implants. Psychophysical studies suggest that poor modulation detection may contribute to the limited performance of ABI users. In a cat model, we determined how the pulse rate of the electrical stimulus applied within or on the CN affects temporal and rate encoding of amplitude modulation (AM) by neurons in the central nucleus of the inferior colliculus (ICC).

APPROACH

Stimulating microelectrodes were implanted chronically in and on the cats' CN, and multi-site recording microelectrodes were implanted chronically into the ICC. Encoding of AM pulse trains by neurons in the ICC was characterized as vector strength (VS), the synchrony of neural activity with the AM, and as the mean rate of neuronal action potentials (neuronal spike rate (NSR)).

MAIN RESULTS

For intranuclear microstimulation, encoding of AM as VS was up to 3 dB greater when stimulus pulse rate was increased from 250 to 500 pps, but only for neuronal units with low best acoustic frequencies, and when the electrical stimulation was modulated at low frequencies (10-20 Hz). For stimulation on the surface of the CN, VS was similar at 250 and 500 pps, and the dynamic range of the VS was reduced for pulse rates greater than 250 pps. Modulation depth was encoded strongly as VS when the maximum stimulus amplitude was held constant across a range of modulation depth. This 'constant maximum' protocol allows enhancement of modulation depth while preserving overall dynamic range. However, modulation depth was not encoded as strongly as NSR.

SIGNIFICANCE

The findings have implications for improved sound processors for present and future ABIs. The performance of ABIs may benefit from using pulse rates greater than those presently used in most ABIs, and by sound processing strategies that enhance the modulation depth of the electrical stimulus while preserving dynamic range.

Functional architecture of the inferior colliculus revealed with voltage-sensitive dyes

We used optical imaging with voltage-sensitive dyes to investigate the spatio-temporal dynamics of synaptically evoked activity in brain slices of the inferior colliculus (IC). Responses in transverse slices which preserve cross-frequency connections and in modified sagittal slices that preserve connections within frequency laminae were evoked by activating the lateral lemniscal tract. Comparing activity between small and large populations of cells revealed response areas in the central nucleus of the IC that were similar in magnitude but graded temporally. In transverse sections, these response areas are summed to generate a topographic response profile. Activity through the commissure to the contralateral IC required an excitation threshold that was reached when GABAergic inhibition was blocked. Within laminae, module interaction created temporal homeostasis. Diffuse activity evoked by a single lemniscal shock re-organized into distinct spatial and temporal compartments when stimulus trains were used, and generated a directional activity profile within the lamina. Using different stimulus patterns to activate subsets of microcircuits in the central nucleus of the IC, we found that localized responses evoked by low-frequency stimulus trains spread extensively when train frequency was increased, suggesting recruitment of silent microcircuits. Long stimulus trains activated a circuit specific to post-inhibitory rebound neurons. Rebound microcircuits were defined by a focal point of initiation that spread to an annular ring that oscillated between inhibition and excitation. We propose that much of the computing power of the IC is derived from local circuits, some of which are cell-type specific. These circuits organize activity within and across frequency laminae, and are critical in determining the stimulus-selectivity of auditory coding.

When and how envelope "rate-limitations" affect processing of interaural temporal disparities conveyed by high-frequency stimuli

The purpose of this chapter is to bring together historical and current findings that reveal the presence, influence, and operation of a type of envelope “rate-limitation.” The rate-limitation has been revealed in both monaural and binaural experiments. Specifically, there appears to be a low-pass envelope-filtering process that (1) functionally attenuates fluctuations of the envelope above about 150 Hz and (2) is not attributable to peripheral band-pass filtering. We show a variety of empirical outcomes and theoretical analyses that converge to demonstrate and to describe how this type of filtering constrains the processing of interaural temporal disparities (ITDs) conveyed by the envelopes of high-frequency stimuli in experiments concerning binaural detection. Included are recent behavioral and neurophysiological findings regarding how such filtering may vary with the center frequency of the stimulus.

Auditory abstraction from spectro-temporal features to coding auditory entities

The auditory system extracts behaviorally relevant information from acoustic stimuli. The average activity in auditory cortex is known to be sensitive to spectro-temporal patterns in sounds. However, it is not known whether the auditory cortex also processes more abstract features of sounds, which may be more behaviorally relevant than spectro-temporal patterns. Using recordings from three stations of the auditory pathway, the inferior colliculus (IC), the ventral division of the medial geniculate body (MGB) of the thalamus, and the primary auditory cortex (A1) of the cat in response to natural sounds, we compared the amount of information that spikes contained about two aspects of the stimuli: spectro-temporal patterns, and abstract entities present in the same stimuli such as a bird chirp, its echoes, and the ambient noise. IC spikes conveyed on average approximately the same amount of information about spectro-temporal patterns as they conveyed about abstract auditory entities, but A1 and the MGB neurons conveyed on average three times more information about abstract auditory entities than about spectro-temporal patterns. Thus, the majority of neurons in auditory thalamus and cortex coded well the presence of abstract entities in the sounds without containing much information about their spectro-temporal structure, suggesting that they are sensitive to abstract features in these sounds.

Spectrotemporal sound preferences of neighboring inferior colliculus neurons: implications for local circuitry and processing

How do local circuits in the inferior colliculus (IC) process and transform spectral and temporal sound information? Using a four-tetrode array we examined the functional properties of the IC and metrics of its micro circuitry by recording neural activity from neighboring single neurons in the cat. Spectral and temporal response preferences were compared for neurons found on the same and adjacent tetrodes (ATs), as well as across distant recording sites. We found that neighboring neurons had similar preferences while neurons recorded across distant sites were less similar. Best frequency (BF) was the most correlated parameter between neighboring neurons and BF differences exhibited unique clustering at ~0.3 octave intervals, indicative of the frequency band lamina. Other spectral and temporal parameters of the receptive fields were more similar for neighboring neurons than for those at distant sites and the receptive field similarity was larger for neurons with small differences in BF. Furthermore, correlated firing was stronger for neighboring neuron pairs and increased with proximity and decreasing BF difference. Thus, although response selectivities are quite diverse in the IC, spectral, and temporal preference within a local microcircuit are functionally quite similar. This suggests a scheme where local circuits are organized into zones that are specialized for processing distinct spectrotemporal cues.