Auditory cortex represents both pitch judgments and the corresponding acoustic cues

Joint population coding and temporal coherence link an attended talker's voice and location features in naturalistic multi-talker scenes

Listeners effortlessly extract multidimensional auditory objects, such as a localized talker, from complex acoustic scenes. However, the neural mechanisms that enable simultaneous encoding and linking of distinct sound features-such as a talker's voice and location-are not fully understood. Using invasive intracranial recordings in neurosurgical patients, we investigated how the human auditory cortex processes and integrates these features during naturalistic multi-talker scenes. We found that cortical sites exhibit a gradient of feature sensitivity, ranging from single-feature sensitive sites (responsive primarily to voice or location) to dual-feature sensitive sites (responsive to both features). At the population level, neural response patterns from both single- and dual-feature sensitive sites jointly encoded the attended talker's voice and location. Notably, single-feature sensitive sites encoded their primary feature with greater precision but also represented coarse information about the secondary feature. Sites selectively tracking a single, attended speech stream concurrently encoded both voice and location features, demonstrating a link between selective attention and feature integration. Additionally, attention selectively enhanced temporal coherence between voice- and location-sensitive sites, suggesting that temporal synchronization serves as a mechanism for linking these features. Our findings highlight two complementary neural mechanisms-joint population coding and temporal coherence- that enable the integration of voice and location features in the auditory cortex. These results provide new insights into the distributed, multidimensional nature of auditory object formation during active listening in complex environments.

SIGNIFICANCE STATEMENT

In everyday life, listeners effortlessly extract individual sound sources from complex acoustic scenes which contain multiple sound sources. Yet, how the brain links the different features of a particular sound source to each other - such as a talker's voice characteristics and location - is poorly understood. Here, we show that two neural mechanisms contribute to encoding and integrating voice and location features in multi-talker sound scenes: (1) some neuronal sites are sensitive to both voice and location and their activity patterns encode these features jointly; (2) the responses of neuronal sites that process only one sound feature - that is, location or voice - align temporally to form a stream that is segregated from the other talker.

HIGHLIGHTS

Auditory cortex exhibits a gradient of feature sensitivity, with some sites encoding only voice or location features, while others encode both simultaneously (dual-feature sensitive sites).Dual-feature sensitive sites integrate voice and location features of an attended talker with equal accuracy, providing a unified representation in multi-talker scenes.Single-feature sensitive sites primarily encode their preferred feature with high precision but also represent coarse information about other features, contributing to population-level integration.Temporal coherence selectively enhances synchronization between voice- and location-sensitive sites, providing another mechanism for integrating an auditory object's features.Multi-dimensional auditory object formation relies on complementary neural mechanisms: joint population coding and temporal coherence.

Ensemble responses of auditory midbrain neurons in the cat to speech stimuli at different signal-to-noise ratios

Originally reserved for those who are profoundly deaf, cochlear implantation is now common for people with partial hearing loss, particularly when combined with a hearing aid. This combined intervention enhances speech comprehension and sound quality when compared to electrical stimulation alone, particularly in noisy environments, but the physiological basis for the benefits is not well understood. Our long-term aim is to elucidate the underlying physiological mechanisms of this improvement, and as a first step in this process, we have investigated in normal hearing cats, the degree to which the patterns of neural activity evoked in the inferior colliculus (IC) by speech sounds in various levels of noise allows discrimination between those sounds. Neuronal responses were recorded simultaneously from 32 sites across the tonotopic axis of the IC in anaesthetised normal hearing cats (n = 7). Speech sounds were presented at 20, 40 and 60 dB SPL in quiet and with increasing levels of additive noise (signal-to-noise ratios (SNRs) -20, -15, -10, -5, 0, +5, +10, +15, +20 dB). Neural discrimination was assessed using a Euclidean measure of distance between neural responses, resulting in a function reflecting speech sound differentiation across various SNRs. Responses of IC neurons reliably encoded the speech stimuli when presented in quiet, with optimal performance when an analysis bin-width of 5-10 ms was used. Discrimination thresholds did not depend on stimulus level and were best for shorter analysis binwidths. This study sheds light on how the auditory midbrain represents speech sounds and provides baseline data with which responses to electro-acoustic speech sounds in partially deafened animals can be compared.

Temporal coherence shapes cortical responses to speech mixtures in a ferret cocktail party

Perceptual segregation of complex sounds such as speech and music simultaneously emanating from multiple sources is a remarkable ability that is common in humans and other animals alike. Unlike animal physiological experiments with simplified sounds or human investigations with spatially broad imaging techniques, this study combines insights from animal single-unit recordings with segregation of speech-like sound mixtures. Ferrets are trained to attend to a female voice and detect a target word, both in presence and absence of a concurrent equally salient male voice. Recordings are made in primary and secondary auditory cortical fields, and in frontal cortex. During task performance, representation of the female words becomes enhanced relative to the male in all, but especially in higher cortical regions. Analysis of the temporal and spectral response characteristics during task performance reveals how speech segregation gradually emerges in the auditory cortex. A computational model evaluated on the same voice mixtures replicates and extends these results to different attentional targets (attention to female or male voices). These findings underscore the role of the principle of temporal coherence whereby attention to a target voice binds together all neural responses coherently modulated with the target, thus ultimately forming and extracting a common auditory stream.

Task-specific invariant representation in auditory cortex

Categorical sensory representations are critical for many behaviors, including speech perception. In the auditory system, categorical information is thought to arise hierarchically, becoming increasingly prominent in higher-order cortical regions. The neural mechanisms that support this robust and flexible computation remain poorly understood. Here, we studied sound representations in the ferret primary and non-primary auditory cortex while animals engaged in a challenging sound discrimination task. Population-level decoding of simultaneously recorded single neurons revealed that task engagement caused categorical sound representations to emerge in non-primary auditory cortex. In primary auditory cortex, task engagement caused a general enhancement of sound decoding that was not specific to task-relevant categories. These findings are consistent with mixed selectivity models of neural disentanglement, in which early sensory regions build an overcomplete representation of the world and allow neurons in downstream brain regions to flexibly and selectively read out behaviorally relevant, categorical information.

Temporal Coherence Shapes Cortical Responses to Speech Mixtures in a Ferret Cocktail Party

Segregation of complex sounds such as speech, music and animal vocalizations as they simultaneously emanate from multiple sources (referred to as the "cocktail party problem") is a remarkable ability that is common in humans and animals alike. The neural underpinnings of this process have been extensively studied behaviorally and physiologically in non-human animals primarily with simplified sounds (tones and noise sequences). In humans, segregation experiments utilizing more complex speech mixtures are common; but physiological experiments have relied on EEG/MEG/ECoG recordings that sample activity from thousands of neurons, often obscuring the detailed processes that give rise to the observed segregation. The present study combines the insights from animal single-unit physiology with segregation of speech-like mixtures. Ferrets were trained to attend to a female voice and detect a target word, both in presence or absence of a concurrent, equally salient male voice. Single neuron recordings were obtained from primary and secondary ferret auditory cortical fields, as well as frontal cortex. During task performance, representation of the female words became more enhanced relative to those of the (distractor) male in all cortical regions, especially in the higher auditory cortical field. Analysis of the temporal and spectral response characteristics during task performance reveals how speech segregation gradually emerges in the auditory cortex. A computational model evaluated on the same voice mixtures replicates and extends these results to different attentional targets (attention to female or male voices). These findings are consistent with the temporal coherence theory whereby attention to a target voice anchors neural activity in cortical networks hence binding together channels that are coherently temporally-modulated with the target, and ultimately forming a common auditory stream.

Care, management, and use of ferrets in biomedical research

The ferret (Mustela putorius furo) is a small domesticated species of the family Mustelidae within the order Carnivora. The present article reviews and discusses the current state of knowledge about housing, care, breeding, and biomedical uses of ferrets. The management and breeding procedures of ferrets resemble those used for other carnivores. Understanding its behavior helps in the use of environmental enrichment and social housing, which promote behaviors typical of the species. Ferrets have been used in research since the beginning of the twentieth century. It is a suitable non-rodent model in biomedical research because of its hardy nature, social behavior, diet and other habits, small size, and thus the requirement of a relatively low amount of test compounds and early sexual maturity compared with dogs and non-human primates. Ferrets and humans have numerous similar anatomical, metabolic, and physiological characteristics, including the endocrine, respiratory, auditory, gastrointestinal, and immunological systems. It is one of the emerging animal models used in studies such as influenza and other infectious respiratory diseases, cystic fibrosis, lung cancer, cardiac research, gastrointestinal disorders, neuroscience, and toxicological studies. Ferrets are vulnerable to many human pathogenic organisms, like severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), because air transmission of this virus between them has been observed in the laboratory. Ferrets draw the attention of the medical community compared to rodents because they occupy a distinct niche in biomedical studies, although they possess a small representation in laboratory research.

Changes in neural readout of response magnitude during auditory streaming do not correlate with behavioral choice in the auditory cortex

A fundamental goal of the auditory system is to group stimuli from the auditory environment into a perceptual unit (i.e., "stream") or segregate the stimuli into multiple different streams. Although previous studies have clarified the psychophysical and neural mechanisms that may underlie this ability, the relationship between these mechanisms remains elusive. Here, we recorded multiunit activity (MUA) from the auditory cortex of monkeys while they participated in an auditory-streaming task consisting of interleaved low- and high-frequency tone bursts. As the streaming stimulus unfolded over time, MUA amplitude habituated; the magnitude of this habituation was correlated with the frequency difference between the tone bursts. An ideal-observer model could classify these time- and frequency-dependent changes into reports of "one stream" or "two streams" in a manner consistent with the behavioral literature. However, because classification was not modulated by the monkeys' behavioral choices, this MUA habituation may not directly reflect perceptual reports.

Dynamics and maintenance of categorical responses in primary auditory cortex during task engagement

Grouping sets of sounds into relevant categories is an important cognitive ability that enables the association of stimuli with appropriate goal-directed behavioral responses. In perceptual tasks, the primary auditory cortex (A1) assumes a prominent role by concurrently encoding both sound sensory features and task-related variables. Here, we sought to explore the role of A1 in the initiation of sound categorization, shedding light on its involvement in this cognitive process. We trained ferrets to discriminate click trains of different rates in a Go/No-Go delayed categorization task and recorded neural activity during both active behavior and passive exposure to the same sounds. Purely categorical response components were extracted and analyzed separately from sensory responses to reveal their contributions to the overall population response throughout the trials. We found that categorical activity emerged during sound presentation in the population average and was present in both active behavioral and passive states. However, upon task engagement, categorical responses to the No-Go category became suppressed in the population code, leading to an asymmetrical representation of the Go stimuli relative to the No-Go sounds and pre-stimulus baseline. The population code underwent an abrupt change at stimulus offset, with sustained responses after the Go sounds during the delay period. Notably, the categorical responses observed during the stimulus period exhibited a significant correlation with those extracted from the delay epoch, suggesting an early involvement of A1 in stimulus categorization.

Stable sound decoding despite modulated sound representation in the auditory cortex

The activity of neurons in the auditory cortex is driven by both sounds and non-sensory context. To investigate the neuronal correlates of non-sensory context, we trained head-fixed mice to perform a two-alternative-choice auditory task in which either reward or stimulus expectation (prior) was manipulated in blocks. Using two-photon calcium imaging to record populations of single neurons in the auditory cortex, we found that both stimulus and reward expectation modulated the activity of these neurons. A linear decoder trained on this population activity could decode stimuli as well or better than predicted by the animal's performance. Interestingly, the optimal decoder was stable even in the face of variable sensory representations. Neither the context nor the mouse's choice could be reliably decoded from the recorded neural activity. Our findings suggest that, in spite of modulation of auditory cortical activity by task priors, the auditory cortex does not represent sufficient information about these priors to exploit them optimally. Thus, the combination of rapidly changing sensory information with more slowly varying task information required for decisions in this task might be represented in brain regions other than the auditory cortex.

Stable sound decoding despite modulated sound representation in the auditory cortex

The activity of neurons in the auditory cortex is driven by both sounds and non-sensory context. To investigate the neuronal correlates of non-sensory context, we trained head-fixed mice to perform a two-alternative choice auditory task in which either reward or stimulus expectation (prior) was manipulated in blocks. Using two-photon calcium imaging to record populations of single neurons in auditory cortex, we found that both stimulus and reward expectation modulated the activity of these neurons. A linear decoder trained on this population activity could decode stimuli as well or better than predicted by the animal's performance. Interestingly, the optimal decoder was stable even in the face of variable sensory representations. Neither the context nor the mouse's choice could be reliably decoded from the recorded neural activity. Our findings suggest that in spite of modulation of auditory cortical activity by task priors, auditory cortex does not represent sufficient information about these priors to exploit them optimally and that decisions in this task require that rapidly changing sensory information be combined with more slowly varying task information extracted and represented in brain regions other than auditory cortex.

Functional network properties of the auditory cortex

The auditory system transforms auditory stimuli from the external environment into perceptual auditory objects. Recent studies have focused on the contribution of the auditory cortex to this transformation. Other studies have yielded important insights into the contributions of neural activity in the auditory cortex to cognition and decision-making. However, despite this important work, the relationship between auditory-cortex activity and behavior/perception has not been fully elucidated. Two of the more important gaps in our understanding are (1) the specific and differential contributions of different fields of the auditory cortex to auditory perception and behavior and (2) the way networks of auditory neurons impact and facilitate auditory information processing. Here, we focus on recent work from non-human-primate models of hearing and review work related to these gaps and put forth challenges to further our understanding of how single-unit activity and network activity in different cortical fields contribution to behavior and perception.

Explicit-memory multiresolution adaptive framework for speech and music separation

The human auditory system employs a number of principles to facilitate the selection of perceptually separated streams from a complex sound mixture. The brain leverages multi-scale redundant representations of the input and uses memory (or priors) to guide the selection of a target sound from the input mixture. Moreover, feedback mechanisms refine the memory constructs resulting in further improvement of selectivity of a particular sound object amidst dynamic backgrounds. The present study proposes a unified end-to-end computational framework that mimics these principles for sound source separation applied to both speech and music mixtures. While the problems of speech enhancement and music separation have often been tackled separately due to constraints and specificities of each signal domain, the current work posits that common principles for sound source separation are domain-agnostic. In the proposed scheme, parallel and hierarchical convolutional paths map input mixtures onto redundant but distributed higher-dimensional subspaces and utilize the concept of temporal coherence to gate the selection of embeddings belonging to a target stream abstracted in memory. These explicit memories are further refined through self-feedback from incoming observations in order to improve the system's selectivity when faced with unknown backgrounds. The model yields stable outcomes of source separation for both speech and music mixtures and demonstrates benefits of explicit memory as a powerful representation of priors that guide information selection from complex inputs.

Neuroimaging evidence for the direct role of auditory scene analysis in object perception

Auditory Scene Analysis (ASA) refers to the grouping of acoustic signals into auditory objects. Previously, we have shown that perceived musicality of auditory sequences varies with high-level organizational features. Here, we explore the neural mechanisms mediating ASA and auditory object perception. Participants performed musicality judgments on randomly generated pure-tone sequences and manipulated versions of each sequence containing low-level changes (amplitude; timbre). Low-level manipulations affected auditory object perception as evidenced by changes in musicality ratings. fMRI was used to measure neural activation to sequences rated most and least musical, and the altered versions of each sequence. Next, we generated two partially overlapping networks: (i) a music processing network (music localizer) and (ii) an ASA network (base sequences vs. ASA manipulated sequences). Using Representational Similarity Analysis, we correlated the functional profiles of each ROI to a model generated from behavioral musicality ratings as well as models corresponding to low-level feature processing and music perception. Within overlapping regions, areas near primary auditory cortex correlated with low-level ASA models, whereas right IPS was correlated with musicality ratings. Shared neural mechanisms that correlate with behavior and underlie both ASA and music perception suggests that low-level features of auditory stimuli play a role in auditory object perception.

Relative pitch representations and invariance to timbre

Information in speech and music is often conveyed through changes in fundamental frequency (f0), perceived by humans as "relative pitch". Relative pitch judgments are complicated by two facts. First, sounds can simultaneously vary in timbre due to filtering imposed by a vocal tract or instrument body. Second, relative pitch can be extracted in two ways: by measuring changes in constituent frequency components from one sound to another, or by estimating the f0 of each sound and comparing the estimates. We examined the effects of timbral differences on relative pitch judgments, and whether any invariance to timbre depends on whether judgments are based on constituent frequencies or their f0. Listeners performed up/down and interval discrimination tasks with pairs of spoken vowels, instrument notes, or synthetic tones, synthesized to be either harmonic or inharmonic. Inharmonic sounds lack a well-defined f0, such that relative pitch must be extracted from changes in individual frequencies. Pitch judgments were less accurate when vowels/instruments were different compared to when they were the same, and were biased by the associated timbre differences. However, this bias was similar for harmonic and inharmonic sounds, and was observed even in conditions where judgments of harmonic sounds were based on f0 representations. Relative pitch judgments are thus not invariant to timbre, even when timbral variation is naturalistic, and when such judgments are based on representations of f0.

Reversible Inactivation of Ferret Auditory Cortex Impairs Spatial and Nonspatial Hearing

A key question in auditory neuroscience is to what extent are brain regions functionally specialized for processing specific sound features, such as location and identity. In auditory cortex, correlations between neural activity and sounds support both the specialization of distinct cortical subfields, and encoding of multiple sound features within individual cortical areas. However, few studies have tested the contribution of auditory cortex to hearing in multiple contexts. Here we determined the role of ferret primary auditory cortex in both spatial and nonspatial hearing by reversibly inactivating the middle ectosylvian gyrus during behavior using cooling (n = 2 females) or optogenetics (n = 1 female). Optogenetic experiments used the mDLx promoter to express Channelrhodopsin-2 in GABAergic interneurons, and we confirmed both viral expression (n = 2 females) and light-driven suppression of spiking activity in auditory cortex, recorded using Neuropixels under anesthesia (n = 465 units from 2 additional untrained female ferrets). Cortical inactivation via cooling or optogenetics impaired vowel discrimination in colocated noise. Ferrets implanted with cooling loops were tested in additional conditions that revealed no deficit when identifying vowels in clean conditions, or when the temporally coincident vowel and noise were spatially separated by 180 degrees. These animals did, however, show impaired sound localization when inactivating the same auditory cortical region implicated in vowel discrimination in noise. Our results demonstrate that, as a brain region showing mixed selectivity for spatial and nonspatial features of sound, primary auditory cortex contributes to multiple forms of hearing.SIGNIFICANCE STATEMENT Neurons in primary auditory cortex are often sensitive to the location and identity of sounds. Here we inactivated auditory cortex during spatial and nonspatial listening tasks using cooling, or optogenetics. Auditory cortical inactivation impaired multiple behaviors, demonstrating a role in both the analysis of sound location and identity and confirming a functional contribution of mixed selectivity observed in neural activity. Parallel optogenetic experiments in two additional untrained ferrets linked behavior to physiology by demonstrating that expression of Channelrhodopsin-2 permitted rapid light-driven suppression of auditory cortical activity recorded under anesthesia.

Interaction of bottom-up and top-down neural mechanisms in spatial multi-talker speech perception

How the human auditory cortex represents spatially separated simultaneous talkers and how talkers' locations and voices modulate the neural representations of attended and unattended speech are unclear. Here, we measured the neural responses from electrodes implanted in neurosurgical patients as they performed single-talker and multi-talker speech perception tasks. We found that spatial separation between talkers caused a preferential encoding of the contralateral speech in Heschl's gyrus (HG), planum temporale (PT), and superior temporal gyrus (STG). Location and spectrotemporal features were encoded in different aspects of the neural response. Specifically, the talker's location changed the mean response level, whereas the talker's spectrotemporal features altered the variation of response around response's baseline. These components were differentially modulated by the attended talker's voice or location, which improved the population decoding of attended speech features. Attentional modulation due to the talker's voice only appeared in the auditory areas with longer latencies, but attentional modulation due to location was present throughout. Our results show that spatial multi-talker speech perception relies upon a separable pre-attentive neural representation, which could be further tuned by top-down attention to the location and voice of the talker.

Deep neural network models reveal interplay of peripheral coding and stimulus statistics in pitch perception

Perception is thought to be shaped by the environments for which organisms are optimized. These influences are difficult to test in biological organisms but may be revealed by machine perceptual systems optimized under different conditions. We investigated environmental and physiological influences on pitch perception, whose properties are commonly linked to peripheral neural coding limits. We first trained artificial neural networks to estimate fundamental frequency from biologically faithful cochlear representations of natural sounds. The best-performing networks replicated many characteristics of human pitch judgments. To probe the origins of these characteristics, we then optimized networks given altered cochleae or sound statistics. Human-like behavior emerged only when cochleae had high temporal fidelity and when models were optimized for naturalistic sounds. The results suggest pitch perception is critically shaped by the constraints of natural environments in addition to those of the cochlea, illustrating the use of artificial neural networks to reveal underpinnings of behavior.

Source identity shapes spatial preference in primary auditory cortex during active navigation

Information about the position of sensory objects and identifying their concurrent behavioral relevance is vital to navigate the environment. In the auditory system, spatial information is computed in the brain based on the position of the sound source relative to the observer and thus assumed to be egocentric throughout the auditory pathway. This assumption is largely based on studies conducted in either anesthetized or head-fixed and passively listening animals, thus lacking self-motion and selective listening. Yet these factors are fundamental components of natural sensing¹ that may crucially impact the nature of spatial coding and sensory object representation.² How individual objects are neuronally represented during unrestricted self-motion and active sensing remains mostly unexplored. Here, we trained gerbils on a behavioral foraging paradigm that required localization and identification of sound sources during free navigation. Chronic tetrode recordings in primary auditory cortex during task performance revealed previously unreported sensory object representations. Strikingly, the egocentric angle preference of the majority of spatially sensitive neurons changed significantly depending on the task-specific identity (outcome association) of the sound source. Spatial tuning also exhibited large temporal complexity. Moreover, we encountered egocentrically untuned neurons whose response magnitude differed between source identities. Using a neural network decoder, we show that, together, these neuronal response ensembles provide spatiotemporally co-existent information about both the egocentric location and the identity of individual sensory objects during self-motion, revealing a novel cortical computation principle for naturalistic sensing.

Multisensory-Guided Associative Learning Enhances Multisensory Representation in Primary Auditory Cortex

Sensory cortices, classically considered to represent modality-specific sensory information, are also found to engage in multisensory processing. However, how sensory processing in sensory cortices is cross-modally modulated remains an open question. Specifically, we understand little of cross-modal representation in sensory cortices in perceptual tasks and how perceptual learning modifies this process. Here, we recorded neural responses in primary auditory cortex (A1) both while freely moving rats discriminated stimuli in Go/No-Go tasks and when anesthetized. Our data show that cross-modal representation in auditory cortices varies with task contexts. In the task of an audiovisual cue being the target associating with water reward, a significantly higher proportion of auditory neurons showed a visually evoked response. The vast majority of auditory neurons, if processing auditory-visual interactions, exhibit significant multisensory enhancement. However, when the rats performed tasks with unisensory cues being the target, cross-modal inhibition, rather than enhancement, predominated. In addition, multisensory associational learning appeared to leave a trace of plastic change in A1, as a larger proportion of A1 neurons showed multisensory enhancement in anesthesia. These findings indicate that multisensory processing in principle sensory cortices is not static, and having cross-modal interaction in the task requirement can substantially enhance multisensory processing in sensory cortices.

When and How Does the Auditory Cortex Influence Subcortical Auditory Structures? New Insights About the Roles of Descending Cortical Projections

For decades, the corticofugal descending projections have been anatomically well described but their functional role remains a puzzling question. In this review, we will first describe the contributions of neuronal networks in representing communication sounds in various types of degraded acoustic conditions from the cochlear nucleus to the primary and secondary auditory cortex. In such situations, the discrimination abilities of collicular and thalamic neurons are clearly better than those of cortical neurons although the latter remain very little affected by degraded acoustic conditions. Second, we will report the functional effects resulting from activating or inactivating corticofugal projections on functional properties of subcortical neurons. In general, modest effects have been observed in anesthetized and in awake, passively listening, animals. In contrast, in behavioral tasks including challenging conditions, behavioral performance was severely reduced by removing or transiently silencing the corticofugal descending projections. This suggests that the discriminative abilities of subcortical neurons may be sufficient in many acoustic situations. It is only in particularly challenging situations, either due to the task difficulties and/or to the degraded acoustic conditions that the corticofugal descending connections bring additional abilities. Here, we propose that it is both the top-down influences from the prefrontal cortex, and those from the neuromodulatory systems, which allow the cortical descending projections to impact behavioral performance in reshaping the functional circuitry of subcortical structures. We aim at proposing potential scenarios to explain how, and under which circumstances, these projections impact on subcortical processing and on behavioral responses.

Functional characterization of human Heschl's gyrus in response to natural speech

Heschl's gyrus (HG) is a brain area that includes the primary auditory cortex in humans. Due to the limitations in obtaining direct neural measurements from this region during naturalistic speech listening, the functional organization and the role of HG in speech perception remain uncertain. Here, we used intracranial EEG to directly record neural activity in HG in eight neurosurgical patients as they listened to continuous speech stories. We studied the spatial distribution of acoustic tuning and the organization of linguistic feature encoding. We found a main gradient of change from posteromedial to anterolateral parts of HG. We also observed a decrease in frequency and temporal modulation tuning and an increase in phonemic representation, speaker normalization, speech sensitivity, and response latency. We did not observe a difference between the two brain hemispheres. These findings reveal a functional role for HG in processing and transforming simple to complex acoustic features and inform neurophysiological models of speech processing in the human auditory cortex.

Computational framework for investigating predictive processing in auditory perception

BACKGROUND

The brain tracks sound sources as they evolve in time, collecting contextual information to predict future sensory inputs. Previous work in predictive coding typically focuses on the perception of predictable stimuli, leaving the implementation of these same neural processes in more complex, real-world environments containing randomness and uncertainty up for debate.

NEW METHOD

To facilitate investigation into the perception of less tightly-controlled listening scenarios, we present a computational model as a tool to ask targeted questions about the underlying predictive processes that connect complex sensory inputs to listener behavior and neural responses. In the modeling framework, observed sound features (e.g. pitch) are tracked sequentially using Bayesian inference. Sufficient statistics are inferred from past observations at multiple time scales and used to make predictions about future observation while tracking the statistical structure of the sensory input.

RESULTS

Facets of the model are discussed in terms of their application to perceptual research, and examples taken from real-world audio demonstrate the model's flexibility to capture a variety of statistical structures along various perceptual dimensions.

COMPARISON WITH EXISTING METHODS

Previous models are often targeted toward interpreting a particular experimental paradigm (e.g., oddball paradigm), perceptual dimension (e.g., pitch processing), or task (e.g., speech segregation), thus limiting their ability to generalize to other domains. The presented model is designed as a flexible and practical tool for broad application.

CONCLUSION

The model is presented as a general framework for generating new hypotheses and guiding investigation into the neural processes underlying predictive coding of complex scenes.

Choice-related activity and neural encoding in primary auditory cortex and lateral belt during feature-selective attention

Selective attention is necessary to sift through, form a coherent percept of, and make behavioral decisions on the vast amount of information present in most sensory environments. How and where selective attention is employed in cortex and how this perceptual information then informs the relevant behavioral decisions is still not well understood. Studies probing selective attention and decision-making in visual cortex have been enlightening as to how sensory attention might work in that modality; whether or not similar mechanisms are employed in auditory attention is not yet clear. Therefore, we trained rhesus macaques on a feature-selective attention task, where they switched between reporting changes in temporal (amplitude modulation, AM) and spectral (carrier bandwidth) features of a broadband noise stimulus. We investigated how the encoding of these features by single neurons in primary (A1) and secondary (middle lateral belt, ML) auditory cortex was affected by the different attention conditions. We found that neurons in A1 and ML showed mixed selectivity to the sound and task features. We found no difference in AM encoding between the attention conditions. We found that choice-related activity in both A1 and ML neurons shifts between attentional conditions. This finding suggests that choice-related activity in auditory cortex does not simply reflect motor preparation or action and supports the relationship between reported choice-related activity and the decision and perceptual process.NEW & NOTEWORTHY We recorded from primary and secondary auditory cortex while monkeys performed a nonspatial feature attention task. Both areas exhibited rate-based choice-related activity. The manifestation of choice-related activity was attention dependent, suggesting that choice-related activity in auditory cortex does not simply reflect arousal or motor influences but relates to the specific perceptual choice.

Dissociation of task engagement and arousal effects in auditory cortex and midbrain

Both generalized arousal and engagement in a specific task influence sensory neural processing. To isolate effects of these state variables in the auditory system, we recorded single-unit activity from primary auditory cortex (A1) and inferior colliculus (IC) of ferrets during a tone detection task, while monitoring arousal via changes in pupil size. We used a generalized linear model to assess the influence of task engagement and pupil size on sound-evoked activity. In both areas, these two variables affected independent neural populations. Pupil size effects were more prominent in IC, while pupil and task engagement effects were equally likely in A1. Task engagement was correlated with larger pupil; thus, some apparent effects of task engagement should in fact be attributed to fluctuations in pupil size. These results indicate a hierarchy of auditory processing, where generalized arousal enhances activity in midbrain, and effects specific to task engagement become more prominent in cortex.

Microsecond interaural time difference discrimination restored by cochlear implants after neonatal deafness

Spatial hearing in cochlear implant (CI) patients remains a major challenge, with many early deaf users reported to have no measurable sensitivity to interaural time differences (ITDs). Deprivation of binaural experience during an early critical period is often hypothesized to be the cause of this shortcoming. However, we show that neonatally deafened (ND) rats provided with precisely synchronized CI stimulation in adulthood can be trained to lateralize ITDs with essentially normal behavioral thresholds near 50 μs. Furthermore, comparable ND rats show high physiological sensitivity to ITDs immediately after binaural implantation in adulthood. Our result that ND-CI rats achieved very good behavioral ITD thresholds, while prelingually deaf human CI patients often fail to develop a useful sensitivity to ITD raises urgent questions concerning the possibility that shortcomings in technology or treatment, rather than missing input during early development, may be behind the usually poor binaural outcomes for current CI patients.

Computational and neurophysiological principles underlying auditory perceptual decisions

A fundamental scientific goal in auditory neuroscience is identifying what mechanisms allow the brain to transform an unlabeled mixture of auditory stimuli into distinct perceptual representations. This transformation is accomplished by a complex interaction of multiple neurocomputational processes, including Gestalt grouping mechanisms, categorization, attention, and perceptual decision-making. Despite a great deal of scientific energy devoted to understanding these principles of hearing, we still do not understand either how auditory perception arises from neural activity or the causal relationship between neural activity and auditory perception. Here, we review the contributions of cortical and subcortical regions to auditory perceptual decisions with an emphasis on those studies that simultaneously measure behavior and neural activity. We also put forth challenges to the field that must be faced if we are to further our understanding of the relationship between neural activity and auditory perception.

Task rule and choice are reflected by layer-specific processing in rodent auditory cortical microcircuits

The primary auditory cortex (A1) is an essential, integrative node that encodes the behavioral relevance of acoustic stimuli, predictions, and auditory-guided decision-making. However, the realization of this integration with respect to the cortical microcircuitry is not well understood. Here, we characterize layer-specific, spatiotemporal synaptic population activity with chronic, laminar current source density analysis in Mongolian gerbils (Meriones unguiculatus) trained in an auditory decision-making Go/NoGo shuttle-box task. We demonstrate that not only sensory but also task- and choice-related information is represented in the mesoscopic neuronal population code of A1. Based on generalized linear-mixed effect models we found a layer-specific and multiplexed representation of the task rule, action selection, and the animal's behavioral options as accumulating evidence in preparation of correct choices. The findings expand our understanding of how individual layers contribute to the integrative circuit in the sensory cortex in order to code task-relevant information and guide sensory-based decision-making.

Hierarchical Encoding of Attended Auditory Objects in Multi-talker Speech Perception

Humans can easily focus on one speaker in a multi-talker acoustic environment, but how different areas of the human auditory cortex (AC) represent the acoustic components of mixed speech is unknown. We obtained invasive recordings from the primary and nonprimary AC in neurosurgical patients as they listened to multi-talker speech. We found that neural sites in the primary AC responded to individual speakers in the mixture and were relatively unchanged by attention. In contrast, neural sites in the nonprimary AC were less discerning of individual speakers but selectively represented the attended speaker. Moreover, the encoding of the attended speaker in the nonprimary AC was invariant to the degree of acoustic overlap with the unattended speaker. Finally, this emergent representation of attended speech in the nonprimary AC was linearly predictable from the primary AC responses. Our results reveal the neural computations underlying the hierarchical formation of auditory objects in human AC during multi-talker speech perception.

Emergent tuning for learned vocalizations in auditory cortex

Vocal learners use early social experience to develop auditory skills specialized for communication. However, it is unknown where in the auditory pathway neural responses become selective for vocalizations or how the underlying encoding mechanisms change with experience. We used a vocal tutoring manipulation in two species of songbird to reveal that tuning for conspecific song arises within the primary auditory cortical circuit. Neurons in the deep region of primary auditory cortex responded more to conspecific songs than to other species' songs and more to species-typical spectrotemporal modulations, but neurons in the intermediate (thalamorecipient) region did not. Moreover, birds that learned song from another species exhibited parallel shifts in selectivity and tuning toward the tutor species' songs in the deep but not the intermediate region. Our results locate a region in the auditory processing hierarchy where an experience-dependent coding mechanism aligns auditory responses with the output of a learned vocal motor behavior.

Direct electrophysiological mapping of human pitch-related processing in auditory cortex

This work sought correlates of pitch perception, defined by neural activity above the lower limit of pitch (LLP), in auditory cortical neural ensembles, and examined their topographical distribution. Local field potentials (LFPs) were recorded in eight patients undergoing invasive recordings for pharmaco-resistant epilepsy. Stimuli consisted of bursts of broadband noise followed by regular interval noise (RIN). RIN was presented at rates below and above the LLP to distinguish responses related to the regularity of the stimulus and the presence of pitch itself. LFPs were recorded from human cortical homologues of auditory core, belt, and parabelt regions using multicontact depth electrodes implanted in Heschl's gyrus (HG) and Planum Temporale (PT), and subdural grid electrodes implanted over lateral superior temporal gyrus (STG). Evoked responses corresponding to the temporal regularity of the stimulus were assessed using autocorrelation of the evoked responses, and occurred for stimuli below and above the LLP. Induced responses throughout the high gamma range (60-200 Hz) were present for pitch values above the LLP, with onset latencies of approximately 70 ms. Mapping of the induced responses onto a common brain space demonstrated variability in the topographical distribution of high gamma responses across subjects. Induced responses were present throughout the length of HG and on PT, which is consistent with previous functional neuroimaging studies. Moreover, in each subject, a region within lateral STG showed robust induced responses at pitch-evoking stimulus rates. This work suggests a distributed representation of pitch processing in neural ensembles in human homologues of core and non-core auditory cortex.

The distribution and nature of responses to broadband sounds associated with pitch in the macaque auditory cortex

The organisation of pitch-perception mechanisms in the primate cortex is controversial, in that divergent results have been obtained, ranging from a single circumscribed 'pitch centre' to systems widely distributed across auditory cortex. Possible reasons for such discrepancies include different species, recording techniques, pitch stimuli, sampling of auditory fields, and the neural metrics recorded. In the present study, we sought to bridge some of these divisions by examining activity related to pitch in both neurons and neuronal ensembles within the auditory cortex of the rhesus macaque, a primate species with similar pitch perception and auditory cortical organisation to humans. We demonstrate similar responses, in primary and non-primary auditory cortex, to two different types of broadband pitch above the macaque lower limit in both neurons and local field potential (LFP) gamma oscillations. The majority of broadband pitch responses in neurons and LFP sites did not show equivalent tuning for sine tones.

Evoked Response Strength in Primary Auditory Cortex Predicts Performance in a Spectro-Spatial Discrimination Task in Rats

The extent to which the primary auditory cortex (A1) participates in instructing animal behavior remains debated. Although multiple studies have shown A1 activity to correlate with animals' perceptual judgments (Jaramillo and Zador, 2011; Bizley et al., 2013; Rodgers and DeWeese, 2014), others have found no relationship between A1 responses and reported auditory percepts (Lemus et al., 2009; Dong et al., 2011). To address this ambiguity, we performed chronic recordings of evoked local field potentials (eLFPs) in A1 of head-fixed female rats performing a two-alternative forced-choice auditory discrimination task. Rats were presented with two interleaved sequences of pure tones from opposite sides and had to indicate the side from which the higher-frequency target stimulus was played. Animal performance closely correlated (r _rm = 0.68) with the difference between the target and distractor eLFP responses: the more the target response exceeded the distractor response, the better the animals were at identifying the side of the target frequency. Reducing the evoked response of either frequency through stimulus-specific adaptation affected performance in the expected way: target localization accuracy was degraded when the target frequency was adapted and improved when the distractor frequency was adapted. Target frequency eLFPs were stronger on hit trials than on error trials. Our results suggest that the degree to which one stimulus stands out over others within A1 activity may determine its perceptual saliency for the animals and accordingly bias their behavioral choices.SIGNIFICANCE STATEMENT The brain must continuously calibrate the saliency of sensory percepts against their relevance to the current behavioral goal. The inability to ignore irrelevant distractors characterizes a spectrum of human attentional disorders. Meanwhile, the connection between the neural underpinnings of stimulus saliency and sensory decisions remains elusive. Here, we record local field potentials in the primary auditory cortex of rats engaged in auditory discrimination to investigate how the cortical representation of target and distractor stimuli impacts behavior. We find that the amplitude difference between target- and distractor-evoked activity predicts discrimination performance (r _rm = 0.68). Specific adaptation of target or distractor shifts performance either below or above chance, respectively. It appears that recent auditory history profoundly influences stimulus saliency, biasing animals toward diametrically-opposed decisions.

Speaker-normalized sound representations in the human auditory cortex

The acoustic dimensions that distinguish speech sounds (like the vowel differences in "boot" and "boat") also differentiate speakers' voices. Therefore, listeners must normalize across speakers without losing linguistic information. Past behavioral work suggests an important role for auditory contrast enhancement in normalization: preceding context affects listeners' perception of subsequent speech sounds. Here, using intracranial electrocorticography in humans, we investigate whether and how such context effects arise in auditory cortex. Participants identified speech sounds that were preceded by phrases from two different speakers whose voices differed along the same acoustic dimension as target words (the lowest resonance of the vocal tract). In every participant, target vowels evoke a speaker-dependent neural response that is consistent with the listener's perception, and which follows from a contrast enhancement model. Auditory cortex processing thus displays a critical feature of normalization, allowing listeners to extract meaningful content from the voices of diverse speakers.

Associations between sounds and actions in early auditory cortex of nonhuman primates

An individual may need to take different actions to the same stimulus in different situations to achieve a given goal. The selection of the appropriate action hinges on the previously learned associations between stimuli, actions, and outcomes in the situations. Here, using a go/no-go paradigm and a symmetrical reward, we show that early auditory cortex of nonhuman primates represents such associations, in both the spiking activity and the local field potentials. Sound-evoked neuronal responses changed with sensorimotor associations shortly after sound onset, and the neuronal responses were largest when the sound signaled that a no-go response was required in a trial to obtain a reward. Our findings suggest that association processes take place in the auditory system and do not necessarily rely on association cortex. Thus, auditory cortex may contribute to a rapid selection of the appropriate motor responses to sounds during goal-directed behavior.

Across-species differences in pitch perception are consistent with differences in cochlear filtering

Pitch perception is critical for recognizing speech, music and animal vocalizations, but its neurobiological basis remains unsettled, in part because of divergent results across species. We investigated whether species-specific differences exist in the cues used to perceive pitch and whether these can be accounted for by differences in the auditory periphery. Ferrets accurately generalized pitch discriminations to untrained stimuli whenever temporal envelope cues were robust in the probe sounds, but not when resolved harmonics were the main available cue. By contrast, human listeners exhibited the opposite pattern of results on an analogous task, consistent with previous studies. Simulated cochlear responses in the two species suggest that differences in the relative salience of the two pitch cues can be attributed to differences in cochlear filter bandwidths. The results support the view that cross-species variation in pitch perception reflects the constraints of estimating a sound’s fundamental frequency given species-specific cochlear tuning.

State-dependent encoding of sound and behavioral meaning in a tertiary region of the ferret auditory cortex

In higher sensory cortices, there is a gradual transformation from sensation to perception and action. In the auditory system, this transformation is revealed by responses in the rostral ventral posterior auditory field (VPr), a tertiary area in the ferret auditory cortex, which shows long-term learning in trained compared to naïve animals, arising from selectively enhanced responses to behaviorally relevant target stimuli. This enhanced representation is further amplified during active performance of spectral or temporal auditory discrimination tasks. VPr also shows sustained short-term memory activity after target stimulus offset, correlated with task response timing and action. These task-related changes in auditory filter properties enable VPr neurons to quickly and nimbly switch between different responses to the same acoustic stimuli, reflecting either spectrotemporal properties, timing, or behavioral meaning of the sound. Furthermore, they demonstrate an interaction between the dynamics of short-term attention and long-term learning, as incoming sound is selectively attended, recognized, and translated into action.

Modeling and MEG evidence of early consonance processing in auditory cortex

Pitch is a fundamental attribute of auditory perception. The interaction of concurrent pitches gives rise to a sensation that can be characterized by its degree of consonance or dissonance. In this work, we propose that human auditory cortex (AC) processes pitch and consonance through a common neural network mechanism operating at early cortical levels. First, we developed a new model of neural ensembles incorporating realistic neuronal and synaptic parameters to assess pitch processing mechanisms at early stages of AC. Next, we designed a magnetoencephalography (MEG) experiment to measure the neuromagnetic activity evoked by dyads with varying degrees of consonance or dissonance. MEG results show that dissonant dyads evoke a pitch onset response (POR) with a latency up to 36 ms longer than consonant dyads. Additionally, we used the model to predict the processing time of concurrent pitches; here, consonant pitch combinations were decoded faster than dissonant combinations, in line with the experimental observations. Specifically, we found a striking match between the predicted and the observed latency of the POR as elicited by the dyads. These novel results suggest that consonance processing starts early in human auditory cortex and may share the network mechanisms that are responsible for (single) pitch processing.

A comparison of neural responses in the primary auditory cortex, amygdala, and medial prefrontal cortex of cats during auditory discrimination tasks

Discriminating biologically relevant sounds is crucial for survival. The neurophysiological mechanisms that mediate this process must register both the reward significance and the physical parameters of acoustic stimuli. Previous experiments have revealed that the primary function of the auditory cortex (AC) is to provide a neural representation of the acoustic parameters of sound stimuli. However, how the brain associates acoustic signals with reward remains unresolved. The amygdala (AMY) and medial prefrontal cortex (mPFC) play keys role in emotion and learning, but it is unknown whether AMY and mPFC neurons are involved in sound discrimination or how the roles of AMY and mPFC neurons differ from those of AC neurons. To examine this, we recorded neural activity in the primary auditory cortex (A1), AMY, and mPFC of cats while they performed a Go/No-go task to discriminate sounds with different temporal patterns. We found that the activity of A1 neurons faithfully coded the temporal patterns of sound stimuli; this activity was not affected by the cats' behavioral choices. The neural representation of stimulus patterns decreased in the AMY, but the neural activity increased when the cats were preparing to discriminate the sound stimuli and waiting for reward. Neural activity in the mPFC did not represent sound patterns, but it showed a clear association with reward and was modulated by the cats' behavioral choices. Our results indicate that the initial auditory representation in A1 is gradually transformed into a stimulus-reward association in the AMY and mPFC to ultimately generate a behavioral choice. NEW & NOTEWORTHY We compared the characteristics of neural activities of primary auditory cortex (A1), amygdala (AMY), and medial prefrontal cortex (mPFC) while cats were performing the same auditory discrimination task. Our results show that there is a gradual transformation of the neural code from a faithful temporal representation of the stimulus in A1, which is insensitive to behavioral choices, to an association with the predictive reward in AMY and mPFC, which, to some extent, is correlated with the animal's behavioral choice.

Sound identity is represented robustly in auditory cortex during perceptual constancy

Perceptual constancy requires neural representations that are selective for object identity, but also tolerant across identity-preserving transformations. How such representations arise in the brain and support perception remains unclear. Here, we study tolerant representation of sound identity in the auditory system by recording neural activity in auditory cortex of ferrets during perceptual constancy. Ferrets generalize vowel identity across variations in fundamental frequency, sound level and location, while neurons represent sound identity robustly across acoustic variations. Stimulus features are encoded with distinct time-courses in all conditions, however encoding of sound identity is delayed when animals fail to generalize and during passive listening. Neurons also encode information about task-irrelevant sound features, as well as animals’ choices and accuracy, while population decoding out-performs animals’ behavior. Our results show that during perceptual constancy, sound identity is represented robustly in auditory cortex across widely varying conditions, and behavioral generalization requires conserved timing of identity information.

Perceptual constancy requires neural representations selective for object identity, yet tolerant of identity-preserving transformations. Here, the authors show that sound identity is represented robustly in auditory cortex and that behavioral generalization requires precise timing of identity information.

Recent advances in understanding the auditory cortex

Our ability to make sense of the auditory world results from neural processing that begins in the ear, goes through multiple subcortical areas, and continues in the cortex. The specific contribution of the auditory cortex to this chain of processing is far from understood. Although many of the properties of neurons in the auditory cortex resemble those of subcortical neurons, they show somewhat more complex selectivity for sound features, which is likely to be important for the analysis of natural sounds, such as speech, in real-life listening conditions. Furthermore, recent work has shown that auditory cortical processing is highly context-dependent, integrates auditory inputs with other sensory and motor signals, depends on experience, and is shaped by cognitive demands, such as attention. Thus, in addition to being the locus for more complex sound selectivity, the auditory cortex is increasingly understood to be an integral part of the network of brain regions responsible for prediction, auditory perceptual decision-making, and learning. In this review, we focus on three key areas that are contributing to this understanding: the sound features that are preferentially represented by cortical neurons, the spatial organization of those preferences, and the cognitive roles of the auditory cortex.

The Contribution of Primary Auditory Cortex to Auditory Categorization in Behaving Monkeys

The specific contribution of core auditory cortex to auditory perception –such as categorization– remains controversial. To identify a contribution of the primary auditory cortex (A1) to perception, we recorded A1 activity while monkeys reported whether a temporal sequence of tone bursts was heard as having a “small” or “large” frequency difference. We found that A1 had frequency-tuned responses that habituated, independent of frequency content, as this auditory sequence unfolded over time. We also found that A1 firing rate was modulated by the monkeys’ reports of “small” and “large” frequency differences; this modulation correlated with their behavioral performance. These findings are consistent with the hypothesis that A1 contributes to the processes underlying auditory categorization.

Focal Suppression of Distractor Sounds by Selective Attention in Auditory Cortex

Auditory selective attention is required for parsing crowded acoustic environments, but cortical systems mediating the influence of behavioral state on auditory perception are not well characterized. Previous neurophysiological studies suggest that attention produces a general enhancement of neural responses to important target sounds versus irrelevant distractors. However, behavioral studies suggest that in the presence of masking noise, attention provides a focal suppression of distractors that compete with targets. Here, we compared effects of attention on cortical responses to masking versus non-masking distractors, controlling for effects of listening effort and general task engagement. We recorded single-unit activity from primary auditory cortex (A1) of ferrets during behavior and found that selective attention decreased responses to distractors masking targets in the same spectral band, compared with spectrally distinct distractors. This suppression enhanced neural target detection thresholds, suggesting that limited attention resources serve to focally suppress responses to distractors that interfere with target detection. Changing effort by manipulating target salience consistently modulated spontaneous but not evoked activity. Task engagement and changing effort tended to affect the same neurons, while attention affected an independent population, suggesting that distinct feedback circuits mediate effects of attention and effort in A1.

Go/No-Go task engagement enhances population representation of target stimuli in primary auditory cortex

Primary sensory cortices are classically considered to extract and represent stimulus features, while association and higher-order areas are thought to carry information about stimulus meaning. Here we show that this information can in fact be found in the neuronal population code of the primary auditory cortex (A1). A1 activity was recorded in awake ferrets while they either passively listened or actively discriminated stimuli in a range of Go/No-Go paradigms, with different sounds and reinforcements. Population-level dimensionality reduction techniques reveal that task engagement induces a shift in stimulus encoding from a sensory to a behaviorally driven representation that specifically enhances the target stimulus in all paradigms. This shift partly relies on task-engagement-induced changes in spontaneous activity. Altogether, we show that A1 population activity bears strong similarities to frontal cortex responses. These findings indicate that primary sensory cortices implement a crucial change in the structure of population activity to extract task-relevant information during behavior.

Sensory areas are thought to process stimulus information while higher-order processing occurs in association cortices. Here the authors report that during task engagement population activity in ferret primary auditory cortex shifts away from encoding stimulus features toward detection of the behaviourally relevant targets.

Contextual modulation of sound processing in the auditory cortex

In everyday acoustic environments, we navigate through a maze of sounds that possess a complex spectrotemporal structure, spanning many frequencies and exhibiting temporal modulations that differ within frequency bands. Our auditory system needs to efficiently encode the same sounds in a variety of different contexts, while preserving the ability to separate complex sounds within an acoustic scene. Recent work in auditory neuroscience has made substantial progress in studying how sounds are represented in the auditory system under different contexts, demonstrating that auditory processing of seemingly simple acoustic features, such as frequency and time, is highly dependent on co-occurring acoustic and behavioral stimuli. Through a combination of electrophysiological recordings, computational analysis and behavioral techniques, recent research identified the interactions between external spectral and temporal context of stimuli, as well as the internal behavioral state.

Incorporating behavioral and sensory context into spectro-temporal models of auditory encoding

For several decades, auditory neuroscientists have used spectro-temporal encoding models to understand how neurons in the auditory system represent sound. Derived from early applications of systems identification tools to the auditory periphery, the spectro-temporal receptive field (STRF) and more sophisticated variants have emerged as an efficient means of characterizing representation throughout the auditory system. Most of these encoding models describe neurons as static sensory filters. However, auditory neural coding is not static. Sensory context, reflecting the acoustic environment, and behavioral context, reflecting the internal state of the listener, can both influence sound-evoked activity, particularly in central auditory areas. This review explores recent efforts to integrate context into spectro-temporal encoding models. It begins with a brief tutorial on the basics of estimating and interpreting STRFs. Then it describes three recent studies that have characterized contextual effects on STRFs, emerging over a range of timescales, from many minutes to tens of milliseconds. An important theme of this work is not simply that context influences auditory coding, but also that contextual effects span a large continuum of internal states. The added complexity of these context-dependent models introduces new experimental and theoretical challenges that must be addressed in order to be used effectively. Several new methodological advances promise to address these limitations and allow the development of more comprehensive context-dependent models in the future.

Small Networks Encode Decision-Making in Primary Auditory Cortex

Sensory detection tasks enhance representations of behaviorally meaningful stimuli in primary auditory cortex (A1). However, it remains unclear how A1 encodes decision-making. Neurons in A1 layer 2/3 (L2/3) show heterogeneous stimulus selectivity and complex anatomical connectivity, and receive input from prefrontal cortex. Thus, task-related modulation of activity in A1 L2/3 might differ across subpopulations. To study the neural coding of decision-making, we used two-photon imaging in A1 L2/3 of mice performing a tone-detection task. Neural responses to targets showed attentional gain and encoded behavioral choice. To characterize network representation of behavioral choice, we analyzed functional connectivity using Granger causality, pairwise noise correlations, and neural decoding. During task performance, small groups of four to five neurons became sparsely linked, locally clustered, and rostro-caudally oriented, while noise correlations both increased and decreased. Our results suggest that sensory-based decision-making involves small neural networks driven by the sum of sensory input, attentional gain, and behavioral choice.

Category-specific neuronal activity in left and right auditory cortex and in medial geniculate body of monkeys

We address the question of whether the auditory cortex of the left and right hemisphere and the auditory thalamus are differently involved in the performance of cognitive tasks. To understand these differences on the level of single neurons we compared neuronal firing in the primary and posterior auditory cortex of the two hemispheres and in the medial geniculate body in monkeys while subjects categorized pitch relationships in tone sequences. In contrast to earlier findings in imaging studies performed on humans, we found little difference between the three brain regions in terms of the category-specificity of their neuronal responses, of tonic firing related to task components, and of decision-related firing. The differences between the results in humans and monkeys may result from the type of neuronal activity considered and how it was analyzed, from the auditory cortical fields studied, or from fundamental differences between these species.

Contribution of spiking activity in the primary auditory cortex to detection in noise

A fundamental problem in hearing is detecting a "target" stimulus (e.g., a friend's voice) that is presented with a noisy background (e.g., the din of a crowded restaurant). Despite its importance to hearing, a relationship between spiking activity and behavioral performance during such a "detection-in-noise" task has yet to be fully elucidated. In this study, we recorded spiking activity in primary auditory cortex (A1) while rhesus monkeys detected a target stimulus that was presented with a noise background. Although some neurons were modulated, the response of the typical A1 neuron was not modulated by the stimulus- and task-related parameters of our task. In contrast, we found more robust representations of these parameters in population-level activity: small populations of neurons matched the monkeys' behavioral sensitivity. Overall, these findings are consistent with the hypothesis that the sensory evidence, which is needed to solve such detection-in-noise tasks, is represented in population-level A1 activity and may be available to be read out by downstream neurons that are involved in mediating this task.NEW & NOTEWORTHY This study examines the contribution of A1 to detecting a sound that is presented with a noisy background. We found that population-level A1 activity, but not single neurons, could provide the evidence needed to make this perceptual decision.

Intonational speech prosody encoding in the human auditory cortex

Speakers of all human languages regularly use intonational pitch to convey linguistic meaning, such as to emphasize a particular word. Listeners extract pitch movements from speech and evaluate the shape of intonation contours independent of each speaker's pitch range. We used high-density electrocorticography to record neural population activity directly from the brain surface while participants listened to sentences that varied in intonational pitch contour, phonetic content, and speaker. Cortical activity at single electrodes over the human superior temporal gyrus selectively represented intonation contours. These electrodes were intermixed with, yet functionally distinct from, sites that encoded different information about phonetic features or speaker identity. Furthermore, the representation of intonation contours directly reflected the encoding of speaker-normalized relative pitch but not absolute pitch.