Behavioral origin of sound-evoked activity in mouse visual cortex

Cortical circuits for cross-modal generalization

Adapting goal-directed behaviors to changing sensory conditions is a fundamental aspect of intelligence. The brain uses abstract representations of the environment to generalize learned associations across sensory modalities. The circuit organization that mediates such cross-modal generalizations remains, however, unknown. Here, we demonstrate that mice can bidirectionally generalize sensorimotor task rules between touch and vision by using abstract representations of peri-personal space within the cortex. Using large-scale mapping in the dorsal cortex at single-cell resolution, we discovered multimodal neurons with congruent spatial representations within multiple associative areas of the dorsal and ventral streams. Optogenetic sensory substitution and systematic silencing of these associative areas revealed that a single area in the dorsal stream is necessary and sufficient for cross-modal generalization. Our results identify and comprehensively describe a cortical circuit organization that underlies an essential cognitive function, providing a structural and functional basis for abstract reasoning in the mammalian brain.

The predictive nature of spontaneous brain activity across scales and species

Emerging research suggests the brain operates as a "prediction machine," continuously anticipating sensory, motor, and cognitive outcomes. Central to this capability is the brain's spontaneous activity-ongoing internal processes independent of external stimuli. Neuroimaging and computational studies support that this activity is integral to maintaining and refining mental models of our environment, body, and behaviors, akin to generative models in computation. During rest, spontaneous activity expands the variability of potential representations, enhancing the accuracy and adaptability of these models. When performing tasks, internal models direct brain regions to anticipate sensory and motor states, optimizing performance. This review synthesizes evidence from various species, from C. elegans to humans, highlighting three key aspects of spontaneous brain activity's role in prediction: the similarity between spontaneous and task-related activity, the encoding of behavioral and interoceptive priors, and the high metabolic cost of this activity, underscoring prediction as a fundamental function of brains across species.

Auditory cortex learns to discriminate audiovisual cues through selective multisensory enhancement

Multisensory object discrimination is essential in everyday life, yet the neural mechanisms underlying this process remain unclear. In this study, we trained rats to perform a two-alternative forced-choice task using both auditory and visual cues. Our findings reveal that multisensory perceptual learning actively engages auditory cortex (AC) neurons in both visual and audiovisual processing. Importantly, many audiovisual neurons in the AC exhibited experience-dependent associations between their visual and auditory preferences, displaying a unique integration model. This model employed selective multisensory enhancement for the auditory-visual pairing guiding the contralateral choice, which correlated with improved multisensory discrimination. Furthermore, AC neurons effectively distinguished whether a preferred auditory stimulus was paired with its associated visual stimulus using this distinct integrative mechanism. Our results highlight the capability of sensory cortices to develop sophisticated integrative strategies, adapting to task demands to enhance multisensory discrimination abilities.

Motor and vestibular signals in the visual cortex permit the separation of self versus externally generated visual motion

Knowing whether we are moving or something in the world is moving around us is possibly the most critical sensory discrimination we need to perform. How the brain and, in particular, the visual system solves this motion-source separation problem is not known. Here, we find that motor, vestibular, and visual motion signals are used by the mouse primary visual cortex (VISp) to differentially represent the same visual flow information according to whether the head is stationary or experiencing passive versus active translation. During locomotion, we find that running suppresses running-congruent translation input and that translation signals dominate VISp activity when running and translation speed become incongruent. This cross-modal interaction between the motor and vestibular systems was found throughout the cortex, indicating that running and translation signals provide a brain-wide egocentric reference frame for computing the internally generated and actual speed of self when moving through and sensing the external world.

Eye Movements in Silent Visual Speech Track Unheard Acoustic Signals and Relate to Hearing Experience

Behavioral and neuroscientific studies have shown that watching a speaker's lip movements aids speech comprehension. Intriguingly, even when videos of speakers are presented silently, various cortical regions track auditory features, such as the envelope. Recently, we demonstrated that eye movements track low-level acoustic information when attentively listening to speech. In this study, we investigated whether ocular speech tracking occurs during visual speech and how it influences cortical silent speech tracking. Furthermore, we compared data from hearing individuals, congenitally deaf individuals, and those who became deaf or hard of hearing (DHH) later in life to assess how audiovisual listening experience and auditory deprivation (early vs late onset) affect neural and ocular speech tracking during silent lip-reading. Using magnetoencephalography (MEG), we examined ocular and neural speech tracking of 75 participants observing silent videos of a speaker played forward and backward. Our main finding is a clear ocular unheard speech tracking effect with dominance of <1 Hz, which was not present for lip movements. Similarly, we observed an ≤ 1.3 Hz effect of neural unheard speech tracking in temporal regions for hearing participants. Importantly, neural tracking was not directly linked to ocular tracking. Strikingly, across listening groups, deaf participants with auditory experience showed higher ocular speech tracking than hearing participants, while no ocular speech tracking effect was revealed for congenitally deaf participants in a very small sample. This study extends previous work by demonstrating the involvement of eye movements in speech processing, even in the absence of acoustic input.

Sensory origin of visually evoked activity in auditory cortex: evidence for true cross-modal processing

The meaning and functions of cross-modal sensory processing in the cortex is at the center of an ongoing debate. While some studies claim that such responses reflect genuine multisensory integration, others argue they are mere artifacts of stimulus-evoked movement or changes in internal state. We examined this issue by measuring face movements and neural activity in awake mouse primary auditory cortex (A1) and primary visual cortex (V1) during visual and auditory stimulation. Visual stimuli rarely evoked face movements, A1 responses to visual input remained robust even in the absence of movement, and optogenetic silencing of V1 reduced A1 visual responses, confirming a sensory origin of these cross-modal responses. These findings directly challenge the argument that cross- modal responses are purely movement-driven and emphasize that rather than assuming all cross-modal effects are artifactual, researchers must rigorously test each case.

Highlights

Unlike auditory stimuli, visual stimuli rarely evoke face movementsFace movements explain sound-evoked firing in VC but not visually evoked firing in ACSilencing VC suppresses visually evoked firing in ACThese results confirm a sensory (rather than motor) origin of these responses.

Visual experience orthogonalizes visual cortical stimulus responses via population code transformation

Sensory and behavioral experience can alter visual cortical stimulus coding, but the precise form of this plasticity is unclear. We measured orientation tuning in 4,000-neuron populations of mouse V1 before and after training on a visuomotor task. Changes to single-cell tuning curves appeared complex, including development of asymmetries and of multiple peaks. Nevertheless, these complex tuning curve transformations can be explained by a simple equation: a convex transformation suppressing responses to task stimuli specifically in cells responding at intermediate levels. The strength of the transformation varies across trials, suggesting a dynamic circuit mechanism rather than static synaptic plasticity. The transformation results in sparsening and orthogonalization of population codes for task stimuli. It cannot improve the performance of an optimal stimulus decoder, which is already perfect even for naive codes, but it improves the performance of a suboptimal decoder model with inductive bias as might be found in downstream readout circuits.

Stability of cross-sensory input to primary somatosensory cortex across experience

Merging information across sensory modalities is key to forming robust percepts, yet how the brain achieves this feat remains unclear. Recent studies report cross-modal influences in the primary sensory cortex, suggesting possible multisensory integration in the early stages of cortical processing. We test several hypotheses about the function of auditory influences on mouse primary somatosensory cortex (S1) using in vivo two-photon calcium imaging. We found sound-evoked spiking activity in an extremely small fraction of cells, and this sparse activity did not encode auditory stimulus identity. Moreover, S1 did not encode information about specific audio-tactile feature conjunctions. Auditory and audio-tactile stimulus encoding remained unchanged after both passive experience and reinforcement. These results suggest that while primary sensory cortex is plastic within its own modality, the influence of other modalities is remarkably stable and stimulus nonspecific.

Fast updating feedback from piriform cortex to the olfactory bulb relays multimodal identity and reward contingency signals during rule-reversal

While animals readily adjust their behavior to adapt to relevant changes in the environment, the neural pathways enabling these changes remain largely unknown. Here, using multiphoton imaging, we investigate whether feedback from the piriform cortex to the olfactory bulb supports such behavioral flexibility. To this end, we engage head-fixed male mice in a multimodal rule-reversal task guided by olfactory and auditory cues. Both odor and, surprisingly, the sound cues trigger responses in the cortical bulbar feedback axons which precede the behavioral report. Responses to the same sensory cue are strongly modulated upon changes in stimulus-reward contingency (rule-reversals). The re-shaping of individual bouton responses occurs within seconds of the rule-reversal events and is correlated with changes in behavior. Optogenetic perturbation of cortical feedback within the bulb disrupts the behavioral performance. Our results indicate that the piriform-to-olfactory bulb feedback axons carry stimulus identity and reward contingency signals which are rapidly re-formatted according to changes in the behavioral context.

Is the impact of spontaneous movements on early visual cortex species specific?

Recent studies in non-human primates do not find pronounced signals related to the animal's own body movements in the responses of neurons in the visual cortex. This is notable because such pronounced signals have been widely observed in the visual cortex of mice. Here, we discuss factors that may contribute to the differences observed between species, such as state, slow neural drift, eccentricity, and changes in retinal input. The interpretation of movement-related signals in the visual cortex also exemplifies the challenge of identifying the sources of correlated variables. Dissecting these sources is central for understanding the functional roles of movement-related signals. We suggest a functional classification of the possible sources, aimed at facilitating cross-species comparative approaches to studying the neural mechanisms of vision during natural behavior.

Inter- and Intrahemispheric Sources of Vestibular Signals to V1

Head movements are sensed by the vestibular organs. Unlike classical senses, signals from vestibular organs are not conveyed to a dedicated cortical area but are broadcast throughout the cortex. Surprisingly, the routes taken by vestibular signals to reach the cortex are still largely uncharted. Here we show that the primary visual cortex (V1) receives real-time head movement signals - direction, velocity, and acceleration - from the ipsilateral pulvinar and contralateral visual cortex. The ipsilateral pulvinar provides the main head movement signal, with a bias toward contraversive movements (e.g. clockwise movements in left V1). Conversely, the contralateral visual cortex provides head movement signals during ipsiversive movements. Crucially, head movement variables encoded in V1 are already encoded in the pulvinar, suggesting that those variables are computed subcortically. Thus, the convergence of inter- and intrahemispheric signals endows V1 with a rich representation of the animal's head movements.

Analysis methods for large-scale neuronal recordings

Simultaneous recordings from hundreds or thousands of neurons are becoming routine because of innovations in instrumentation, molecular tools, and data processing software. Such recordings can be analyzed with data science methods, but it is not immediately clear what methods to use or how to adapt them for neuroscience applications. We review, categorize, and illustrate diverse analysis methods for neural population recordings and describe how these methods have been used to make progress on longstanding questions in neuroscience. We review a variety of approaches, ranging from the mathematically simple to the complex, from exploratory to hypothesis-driven, and from recently developed to more established methods. We also illustrate some of the common statistical pitfalls in analyzing large-scale neural data.

Comparative Validation of Scintillator Materials for X-Ray-Mediated Neuronal Control in the Deep Brain

When exposed to X-rays, scintillators emit visible luminescence. X-ray-mediated optogenetics employs scintillators for remotely activating light-sensitive proteins in biological tissue through X-ray irradiation. This approach offers advantages over traditional optogenetics, allowing for deeper tissue penetration and wireless control. Here, we assessed the short-term safety and efficacy of candidate scintillator materials for neuronal control. Our analyses revealed that lead-free halide scintillators, such as Cs3Cu2I5, exhibited significant cytotoxicity within 24 h and induced neuroinflammatory effects when injected into the mouse brain. In contrast, cerium-doped gadolinium aluminum gallium garnet (Ce:GAGG) nanoparticles showed no detectable cytotoxicity within the same period, and injection into the mouse brain did not lead to observable neuroinflammation over four weeks. Electrophysiological recordings in the cerebral cortex of awake mice showed that X-ray-induced radioluminescence from Ce:GAGG nanoparticles reliably activated 45% of the neuronal population surrounding the implanted particles, a significantly higher activation rate than europium-doped GAGG (Eu:GAGG) microparticles, which activated only 10% of neurons. Furthermore, we established the cell-type specificity of this technique by using Ce:GAGG nanoparticles to selectively stimulate midbrain dopamine neurons. This technique was applied to freely behaving mice, allowing for wireless modulation of place preference behavior mediated by midbrain dopamine neurons. These findings highlight the unique suitability of Ce:GAGG nanoparticles for X-ray-mediated optogenetics. The deep tissue penetration, short-term safety, wireless neuronal control, and cell-type specificity of this system offer exciting possibilities for diverse neuroscience applications and therapeutic interventions.

The Neural and Computational Architecture of Feedback Dynamics in Mouse Cortex during Stimulus Report

Conscious reportability of visual input is associated with a bimodal neural response in the primary visual cortex (V1): an early-latency response coupled to stimulus features and a late-latency response coupled to stimulus report or detection. This late wave of activity, central to major theories of consciousness, is thought to be driven by the prefrontal cortex (PFC), responsible for "igniting" it. Here we analyzed two electrophysiological studies in mice performing different stimulus detection tasks and characterized neural activity profiles in three key cortical regions: V1, posterior parietal cortex (PPC), and PFC. We then developed a minimal network model, constrained by known connectivity between these regions, reproducing the spatiotemporal propagation of visual- and report-related activity. Remarkably, while PFC was indeed necessary to generate report-related activity in V1, this occurred only through the mediation of PPC. PPC, and not PFC, had the final veto in enabling the report-related late wave of V1 activity.

Centering cognitive neuroscience on task demands and generalization

Cognitive neuroscience seeks generalizable theories explaining the relationship between behavioral, physiological and mental states. In pursuit of such theories, we propose a theoretical and empirical framework that centers on understanding task demands and the mutual constraints they impose on behavior and neural activity. Task demands emerge from the interaction between an agent's sensory impressions, goals and behavior, which jointly shape the activity and structure of the nervous system on multiple spatiotemporal scales. Understanding this interaction requires multitask studies that vary more than one experimental component (for example, stimuli and instructions) combined with dense behavioral and neural sampling and explicit testing for generalization across tasks and data modalities. By centering task demands rather than mental processes that tasks are assumed to engage, this framework paves the way for the discovery of new generalizable concepts unconstrained by existing taxonomies, and moves cognitive neuroscience toward an action-oriented, dynamic and integrated view of the brain.

Stability of cross-sensory input to primary somatosensory cortex across experience

Merging information from across sensory modalities is key to forming robust, disambiguated percepts of the world, yet how the brain achieves this feat remains unclear. Recent observations of cross-modal influences in primary sensory cortical areas have suggested that multisensory integration may occur in the earliest stages of cortical processing, but the role of these responses is still poorly understood. We address these questions by testing several hypotheses about the possible functions served by auditory influences on the barrel field of mouse primary somatosensory cortex (S1) using in vivo 2-photon calcium imaging. We observed sound-evoked spiking activity in a small fraction of cells overall, and moreover that this sparse activity was insufficient to encode auditory stimulus identity; few cells responded preferentially to one sound or another, and a linear classifier trained to decode auditory stimuli from population activity performed barely above chance. Moreover S1 did not encode information about specific audio-tactile feature conjunctions that we tested. Our ability to decode auditory audio-tactile stimuli from neural activity remained unchanged after both passive experience and reinforcement. Collectively, these results suggest that while a primary sensory cortex is highly plastic with regard to its own modality, the influence of other modalities are remarkably stable and play a largely stimulus-non-specific role.

Decoding dynamic visual scenes across the brain hierarchy

Understanding the computational mechanisms that underlie the encoding and decoding of environmental stimuli is a crucial investigation in neuroscience. Central to this pursuit is the exploration of how the brain represents visual information across its hierarchical architecture. A prominent challenge resides in discerning the neural underpinnings of the processing of dynamic natural visual scenes. Although considerable research efforts have been made to characterize individual components of the visual pathway, a systematic understanding of the distinctive neural coding associated with visual stimuli, as they traverse this hierarchical landscape, remains elusive. In this study, we leverage the comprehensive Allen Visual Coding-Neuropixels dataset and utilize the capabilities of deep learning neural network models to study neural coding in response to dynamic natural visual scenes across an expansive array of brain regions. Our study reveals that our decoding model adeptly deciphers visual scenes from neural spiking patterns exhibited within each distinct brain area. A compelling observation arises from the comparative analysis of decoding performances, which manifests as a notable encoding proficiency within the visual cortex and subcortical nuclei, in contrast to a relatively reduced encoding activity within hippocampal neurons. Strikingly, our results unveil a robust correlation between our decoding metrics and well-established anatomical and functional hierarchy indexes. These findings corroborate existing knowledge in visual coding related to artificial visual stimuli and illuminate the functional role of these deeper brain regions using dynamic stimuli. Consequently, our results suggest a novel perspective on the utility of decoding neural network models as a metric for quantifying the encoding quality of dynamic natural visual scenes represented by neural responses, thereby advancing our comprehension of visual coding within the complex hierarchy of the brain.

Pan-cortical 2-photon mesoscopic imaging and neurobehavioral alignment in awake, behaving mice

The flow of neural activity across the neocortex during active sensory discrimination is constrained by task-specific cognitive demands, movements, and internal states. During behavior, the brain appears to sample from a broad repertoire of activation motifs. Understanding how these patterns of local and global activity are selected in relation to both spontaneous and task-dependent behavior requires in-depth study of densely sampled activity at single neuron resolution across large regions of cortex. In a significant advance toward this goal, we developed procedures to record mesoscale 2-photon Ca2+ imaging data from two novel in vivo preparations that, between them, allow for simultaneous access to nearly all 0f the mouse dorsal and lateral neocortex. As a proof of principle, we aligned neural activity with both behavioral primitives and high-level motifs to reveal the existence of large populations of neurons that coordinated their activity across cortical areas with spontaneous changes in movement and/or arousal. The methods we detail here facilitate the identification and exploration of widespread, spatially heterogeneous neural ensembles whose activity is related to diverse aspects of behavior.

Auditory discrimination learning differentially modulates neural representation in auditory cortex subregions and inter-areal connectivity

Changes in sound-evoked responses in the auditory cortex (ACtx) occur during learning, but how learning alters neural responses in different ACtx subregions and changes their interactions is unclear. To address these questions, we developed an automated training and widefield imaging system to longitudinally track the neural activity of all mouse ACtx subregions during a tone discrimination task. We find that responses in primary ACtx are highly informative of learned stimuli and behavioral outcomes throughout training. In contrast, representations of behavioral outcomes in the dorsal posterior auditory field, learned stimuli in the dorsal anterior auditory field, and inter-regional correlations between primary and higher-order areas are enhanced with training. Moreover, ACtx response changes vary between stimuli, and such differences display lag synchronization with the learning rate. These results indicate that learning alters functional connections between ACtx subregions, inducing region-specific modulations by propagating behavioral information from primary to higher-order areas.

Spontaneous movements and their impact on neural activity fluctuate with latent engagement states

Existing work demonstrates that animals alternate between engaged and disengaged states during perceptual decision-making. To understand the neural signature of these states, we performed cortex-wide measurements of neural activity in mice making auditory decisions. The trial-averaged magnitude of neural activity was similar in the two states. However, the trial-to-trial variance in neural activity was higher during disengagement. To understand this increased variance, we trained separate linear encoding models on neural data from each state. The models demonstrated that although task variables and task-aligned movements impacted neural activity similarly during the two states, movements that are independent of task events explained more variance during disengagement. Behavioral analyses uncovered that during disengagement, movements become uncoupled to task events. Taken together, these results argue that the neural signature of disengagement, though obscured in trial-averaged neural activity, is evident in trial-to-trial variability driven by changing patterns of spontaneous movements.

Distributed representations of prediction error signals across the cortical hierarchy are synergistic

A relevant question concerning inter-areal communication in the cortex is whether these interactions are synergistic. Synergy refers to the complementary effect of multiple brain signals conveying more information than the sum of each isolated signal. Redundancy, on the other hand, refers to the common information shared between brain signals. Here, we dissociated cortical interactions encoding complementary information (synergy) from those sharing common information (redundancy) during prediction error (PE) processing. We analyzed auditory and frontal electrocorticography (ECoG) signals in five common awake marmosets performing two distinct auditory oddball tasks and investigated to what extent event-related potentials (ERP) and broadband (BB) dynamics encoded synergistic and redundant information about PE processing. The information conveyed by ERPs and BB signals was synergistic even at lower stages of the hierarchy in the auditory cortex and between auditory and frontal regions. Using a brain-constrained neural network, we simulated the synergy and redundancy observed in the experimental results and demonstrated that the emergence of synergy between auditory and frontal regions requires the presence of strong, long-distance, feedback, and feedforward connections. These results indicate that distributed representations of PE signals across the cortical hierarchy can be highly synergistic.

Cortical cellular encoding of thermotactile integration

Recent evidence suggests that primary sensory cortical regions play a role in the integration of information from multiple sensory modalities. How primary cortical neurons integrate different sources of sensory information is unclear, partly because non-primary sensory input to a cortical sensory region is often weak or modulatory. To address this question, we take advantage of the robust representation of thermal (cooling) and tactile stimuli in mouse forelimb primary somatosensory cortex (fS1). Using a thermotactile detection task, we show that the perception of threshold-level cool or tactile information is enhanced when they are presented simultaneously, compared with presentation alone. To investigate the cortical cellular correlates of thermotactile integration, we performed in vivo extracellular recordings from fS1 in awake resting and anesthetized mice during unimodal and bimodal stimulation of the forepaw. Unimodal stimulation evoked thermal- or tactile- specific excitatory and inhibitory responses of fS1 neurons. The most prominent features of combined thermotactile stimulation are the recruitment of unimodally silent fS1 neurons, non-linear integration features, and response dynamics that favor longer response durations with additional spikes. Together, we identify quantitative and qualitative changes in cortical encoding that may underlie the improvement in perception of thermotactile surfaces during haptic exploration.

Sound elicits stereotyped facial movements that provide a sensitive index of hearing abilities in mice

Sound elicits rapid movements of muscles in the face, ears, and eyes that protect the body from injury and trigger brain-wide internal state changes. Here, we performed quantitative facial videography from mice resting atop a piezoelectric force plate and observed that broadband sounds elicited rapid and stereotyped facial twitches. Facial motion energy (FME) adjacent to the whisker array was 30 dB more sensitive than the acoustic startle reflex and offered greater inter-trial and inter-animal reliability than sound-evoked pupil dilations or movement of other facial and body regions. FME tracked the low-frequency envelope of broadband sounds, providing a means to study behavioral discrimination of complex auditory stimuli, such as speech phonemes in noise. Approximately 25% of layer 5-6 units in the auditory cortex (ACtx) exhibited firing rate changes during facial movements. However, FME facilitation during ACtx photoinhibition indicated that sound-evoked facial movements were mediated by a midbrain pathway and modulated by descending corticofugal input. FME and auditory brainstem response (ABR) thresholds were closely aligned after noise-induced sensorineural hearing loss, yet FME growth slopes were disproportionately steep at spared frequencies, reflecting a central plasticity that matched commensurate changes in ABR wave 4. Sound-evoked facial movements were also hypersensitive in Ptchd1 knockout mice, highlighting the use of FME for identifying sensory hyper-reactivity phenotypes after adult-onset hyperacusis and inherited deficiencies in autism risk genes. These findings present a sensitive and integrative measure of hearing while also highlighting that even low-intensity broadband sounds can elicit a complex mixture of auditory, motor, and reafferent somatosensory neural activity.

Dynamic modulation of mouse thalamocortical visual activity by salient sounds

Visual responses of the primary visual cortex (V1) are altered by sound. Sound-driven behavioral arousal suggests that, in addition to direct inputs from the primary auditory cortex (A1), multiple other sources may shape V1 responses to sound. Here, we show in anesthetized mice that sound (white noise, ≥70dB) drives a biphasic modulation of V1 visually driven gamma-band activity, comprising fast-transient inhibitory and slow, prolonged excitatory (A1-independent) arousal-driven components. An analogous yet quicker modulation of the visual response also occurred earlier in the visual pathway, at the level of the dorsolateral geniculate nucleus (dLGN), where sound transiently inhibited the early phasic visual response and subsequently induced a prolonged increase in tonic spiking activity and gamma rhythmicity. Our results demonstrate that sound-driven modulations of visual activity are not exclusive to V1 and suggest that thalamocortical inputs from the dLGN to V1 contribute to shaping V1 visual response to sound.

Auditory cortex conveys non-topographic sound localization signals to visual cortex

Spatiotemporally congruent sensory stimuli are fused into a unified percept. The auditory cortex (AC) sends projections to the primary visual cortex (V1), which could provide signals for binding spatially corresponding audio-visual stimuli. However, whether AC inputs in V1 encode sound location remains unknown. Using two-photon axonal calcium imaging and a speaker array, we measured the auditory spatial information transmitted from AC to layer 1 of V1. AC conveys information about the location of ipsilateral and contralateral sound sources to V1. Sound location could be accurately decoded by sampling AC axons in V1, providing a substrate for making location-specific audiovisual associations. However, AC inputs were not retinotopically arranged in V1, and audio-visual modulations of V1 neurons did not depend on the spatial congruency of the sound and light stimuli. The non-topographic sound localization signals provided by AC might allow the association of specific audiovisual spatial patterns in V1 neurons.

Lightning Pose: improved animal pose estimation via semi-supervised learning, Bayesian ensembling, and cloud-native open-source tools

Contemporary pose estimation methods enable precise measurements of behavior via supervised deep learning with hand-labeled video frames. Although effective in many cases, the supervised approach requires extensive labeling and often produces outputs that are unreliable for downstream analyses. Here, we introduce "Lightning Pose," an efficient pose estimation package with three algorithmic contributions. First, in addition to training on a few labeled video frames, we use many unlabeled videos and penalize the network whenever its predictions violate motion continuity, multiple-view geometry, and posture plausibility (semi-supervised learning). Second, we introduce a network architecture that resolves occlusions by predicting pose on any given frame using surrounding unlabeled frames. Third, we refine the pose predictions post-hoc by combining ensembling and Kalman smoothing. Together, these components render pose trajectories more accurate and scientifically usable. We release a cloud application that allows users to label data, train networks, and predict new videos directly from the browser.

The selfish network: how the brain preserves behavioral function through shifts in neuronal network state

Neuronal networks possess the ability to regulate their activity states in response to disruptions. How and when neuronal networks turn from physiological into pathological states, leading to the manifestation of neuropsychiatric disorders, remains largely unknown. Here, we propose that neuronal networks intrinsically maintain network stability even at the cost of neuronal loss. Despite the new stable state being potentially maladaptive, neural networks may not reverse back to states associated with better long-term outcomes. These maladaptive states are often associated with hyperactive neurons, marking the starting point for activity-dependent neurodegeneration. Transitions between network states may occur rapidly, and in discrete steps rather than continuously, particularly in neurodegenerative disorders. The self-stabilizing, metastable, and noncontinuous characteristics of these network states can be mathematically described as attractors. Maladaptive attractors may represent a distinct pathophysiological entity that could serve as a target for new therapies and for fostering resilience.

Triple dissociation of visual, auditory and motor processing in mouse primary visual cortex

Primary sensory cortices respond to crossmodal stimuli-for example, auditory responses are found in primary visual cortex (V1). However, it remains unclear whether these responses reflect sensory inputs or behavioral modulation through sound-evoked body movement. We address this controversy by showing that sound-evoked activity in V1 of awake mice can be dissociated into auditory and behavioral components with distinct spatiotemporal profiles. The auditory component began at approximately 27 ms, was found in superficial and deep layers and originated from auditory cortex. Sound-evoked orofacial movements correlated with V1 neural activity starting at approximately 80-100 ms and explained auditory frequency tuning. Visual, auditory and motor activity were expressed by different laminar profiles and largely segregated subsets of neuronal populations. During simultaneous audiovisual stimulation, visual representations remained dissociable from auditory-related and motor-related activity. This three-fold dissociability of auditory, motor and visual processing is central to understanding how distinct inputs to visual cortex interact to support vision.

Rodents' visual gamma as a biomarker of pathological neural conditions

Neural gamma oscillations (indicatively 30-100 Hz) are ubiquitous: they are associated with a broad range of functions in multiple cortical areas and across many animal species. Experimental and computational works established gamma rhythms as a global emergent property of neuronal networks generated by the balanced and coordinated interaction of excitation and inhibition. Coherently, gamma activity is strongly influenced by the alterations of synaptic dynamics which are often associated with pathological neural dysfunctions. We argue therefore that these oscillations are an optimal biomarker for probing the mechanism of cortical dysfunctions. Gamma oscillations are also highly sensitive to external stimuli in sensory cortices, especially the primary visual cortex (V1), where the stimulus dependence of gamma oscillations has been thoroughly investigated. Gamma manipulation by visual stimuli tuning is particularly easy in rodents, which have become a standard animal model for investigating the effects of network alterations on gamma oscillations. Overall, gamma in the rodents' visual cortex offers an accessible probe on dysfunctional information processing in pathological conditions. Beyond vision-related dysfunctions, alterations of gamma oscillations in rodents were indeed also reported in neural deficits such as migraine, epilepsy and neurodegenerative or neuropsychiatric conditions such as Alzheimer's, schizophrenia and autism spectrum disorders. Altogether, the connections between visual cortical gamma activity and physio-pathological conditions in rodent models underscore the potential of gamma oscillations as markers of neuronal (dys)functioning.

Visual Stimuli Modulate Local Field Potentials But Drive No High-Frequency Activity in Human Auditory Cortex

Neuroimaging studies suggest cross-sensory visual influences in human auditory cortices (ACs). Whether these influences reflect active visual processing in human ACs, which drives neuronal firing and concurrent broadband high-frequency activity (BHFA; >70 Hz), or whether they merely modulate sound processing is still debatable. Here, we presented auditory, visual, and audiovisual stimuli to 16 participants (7 women, 9 men) with stereo-EEG depth electrodes implanted near ACs for presurgical monitoring. Anatomically normalized group analyses were facilitated by inverse modeling of intracranial source currents. Analyses of intracranial event-related potentials (iERPs) suggested cross-sensory responses to visual stimuli in ACs, which lagged the earliest auditory responses by several tens of milliseconds. Visual stimuli also modulated the phase of intrinsic low-frequency oscillations and triggered 15-30 Hz event-related desynchronization in ACs. However, BHFA, a putative correlate of neuronal firing, was not significantly increased in ACs after visual stimuli, not even when they coincided with auditory stimuli. Intracranial recordings demonstrate cross-sensory modulations, but no indication of active visual processing in human ACs.

Pan-cortical 2-photon mesoscopic imaging and neurobehavioral alignment in awake, behaving mice

The flow of neural activity across the neocortex during active sensory discrimination is constrained by task-specific cognitive demands, movements, and internal states. During behavior, the brain appears to sample from a broad repertoire of activation motifs. Understanding how these patterns of local and global activity are selected in relation to both spontaneous and task-dependent behavior requires in-depth study of densely sampled activity at single neuron resolution across large regions of cortex. In a significant advance toward this goal, we developed procedures to record mesoscale 2-photon Ca2+ imaging data from two novel in vivo preparations that, between them, allow simultaneous access to nearly all of the mouse dorsal and lateral neocortex. As a proof of principle, we aligned neural activity with both behavioral primitives and high-level motifs to reveal the existence of large populations of neurons that coordinated their activity across cortical areas with spontaneous changes in movement and/or arousal. The methods we detail here facilitate the identification and exploration of widespread, spatially heterogeneous neural ensembles whose activity is related to diverse aspects of behavior.

Brain-state-dependent constraints on claustrocortical communication and function

Neural activity in the claustrum has been associated with a range of vigilance states, yet the activity patterns and efficacy of synaptic communication of identified claustrum neurons have not been thoroughly determined. Here, we show that claustrum neurons projecting to the retrosplenial cortex are most active during synchronized cortical states such as non-rapid eye movement (NREM) sleep and are suppressed during increased cortical desynchronization associated with arousal, movement, and REM sleep. The efficacy of claustrocortical signaling is increased during NREM and diminished during movement due in part to increased cholinergic tone. Finally, claustrum activation during NREM sleep enhances memory consolidation through the phase resetting of cortical delta waves. Therefore, claustrocortical communication is constrained to function most effectively during cognitive processes associated with synchronized cortical states, such as memory consolidation.

Layer 1 neocortex: Gating and integrating multidimensional signals

Layer 1 (L1) of the neocortex acts as a nexus for the collection and processing of widespread information. By integrating ascending inputs with extensive top-down activity, this layer likely provides critical information regulating how the perception of sensory inputs is reconciled with expectation. This is accomplished by sorting, directing, and integrating the complex network of excitatory inputs that converge onto L1. These signals are combined with neuromodulatory afferents and gated by the wealth of inhibitory interneurons that either are embedded within L1 or send axons from other cortical layers. Together, these interactions dynamically calibrate information flow throughout the neocortex. This review will primarily focus on L1 within the primary sensory cortex and will use these insights to understand L1 in other cortical areas.

Sensory and behavioral modulation of thalamic head-direction cells

Head-direction (HD) neurons are thought to exclusively encode directional heading. In awake mice, we found that sensory stimuli evoked robust short-latency responses in thalamic HD cells, but not in non-HD neurons. The activity of HD cells, but not that of non-HD neurons, was tightly correlated to brain-state fluctuations and dynamically modulated during social interactions. These data point to a new role for the thalamic compass in relaying sensory and behavioral-state information.

Hypersensitivity to Distractors in Fragile X Syndrome from Loss of Modulation of Cortical VIP Interneurons

Attention deficit is one of the most prominent and disabling symptoms in Fragile X syndrome (FXS). Hypersensitivity to sensory stimuli contributes to attention difficulties by overwhelming and/or distracting affected individuals, which disrupts activities of daily living at home and learning at school. We find that auditory or visual distractors selectively impair visual discrimination performance in humans and mice with FXS but not in typically developing controls. In both species, males and females were examined. Vasoactive intestinal polypeptide (VIP) neurons were significantly modulated by incorrect responses in the poststimulus period during early distractor trials in WT mice, consistent with their known role as error signals. Strikingly, however, VIP cells from Fmr1 -/- mice showed little modulation in error trials, and this correlated with their poor performance on the distractor task. Thus, VIP interneurons and their reduced modulatory influence on pyramidal cells could be a potential therapeutic target for attentional difficulties in FXS.SIGNIFICANCE STATEMENT Sensory hypersensitivity, impulsivity, and persistent inattention are among the most consistent clinical features of FXS, all of which impede daily functioning and create barriers to learning. However, the neural mechanisms underlying sensory over-reactivity remain elusive. To overcome a significant challenge in translational FXS research we demonstrate a compelling alignment of sensory over-reactivity in both humans with FXS and Fmr1 -/- mice (the principal animal model of FXS) using a novel analogous distractor task. Two-photon microscopy in mice revealed that lack of modulation by VIP cells contributes to susceptibility to distractors. Implementing research efforts we describe here can help identify dysfunctional neural mechanisms associated not only with sensory issues but broader impairments, including those in learning and cognition.

Recent strategies for neural dynamics observation at a larger scale and wider scope

The tremendous technical progress in neuroscience offers opportunities to observe a more minor or/and broader dynamic picture of the brain. Moreover, the large-scale neural activity of individual neurons enables the dissection of detailed mechanistic links between neural populations and behaviors. To measure neural activity in-vivo, multi-neuron recording, and neuroimaging techniques are employed and developed to acquire more neurons. The tools introduced concurrently recorded dozens to hundreds of neurons in the coordinated brain regions and elucidated the neuronal ensembles from a massive population perspective of diverse neurons at cellular resolution. In particular, the increasing spatiotemporal resolution of neuronal monitoring across the whole brain dramatically facilitates our understanding of additional nervous system functions in health and disease. Here, we will introduce state-of-the-art neuroscience tools involving large-scale neural population recording and the long-range connections spanning multiple brain regions. Their synergic effects provide to clarify the controversial circuitry underlying neuroscience. These challenging neural tools present a promising outlook for the fundamental dynamic interplay across levels of synaptic cellular, circuit organization, and brain-wide. Hence, more observations of neural dynamics will provide more clues to elucidate brain functions and push forward innovative technology at the intersection of neural engineering disciplines. We hope this review will provide insight into the use or development of recent neural techniques considering spatiotemporal scales of brain observation.

Activity in primate visual cortex is minimally driven by spontaneous movements

Organisms process sensory information in the context of their own moving bodies, an idea referred to as embodiment. This idea is important for developmental neuroscience, robotics and systems neuroscience. The mechanisms supporting embodiment are unknown, but a manifestation could be the observation in mice of brain-wide neuromodulation, including in the primary visual cortex, driven by task-irrelevant spontaneous body movements. We tested this hypothesis in macaque monkeys (Macaca mulatta), a primate model for human vision, by simultaneously recording visual cortex activity and facial and body movements. We also sought a direct comparison using an analogous approach to those used in mouse studies. Here we found that activity in the primate visual cortex (V1, V2 and V3/V3A) was associated with the animals' own movements, but this modulation was largely explained by the impact of the movements on the retinal image, that is, by changes in visual input. These results indicate that visual cortex in primates is minimally driven by spontaneous movements and may reflect species-specific sensorimotor strategies.

Potentiation of cholinergic and corticofugal inputs to the lateral amygdala in threat learning

The amygdala, cholinergic basal forebrain, and higher-order auditory cortex (HO-AC) regulate brain-wide plasticity underlying auditory threat learning. Here, we perform multi-regional extracellular recordings and optical measurements of acetylcholine (ACh) release to characterize the development of discriminative plasticity within and between these brain regions as mice acquire and recall auditory threat memories. Spiking responses are potentiated for sounds paired with shock (CS+) in the lateral amygdala (LA) and optogenetically identified corticoamygdalar projection neurons, although not in neighboring HO-AC units. Spike- or optogenetically triggered local field potentials reveal enhanced corticofugal-but not corticopetal-functional coupling between HO-AC and LA during threat memory recall that is correlated with pupil-indexed memory strength. We also note robust sound-evoked ACh release that rapidly potentiates for the CS+ in LA but habituates across sessions in HO-AC. These findings highlight a distributed and cooperative plasticity in LA inputs as mice learn to reappraise neutral stimuli as possible threats.

Short-term auditory priming in freely-moving mice

Priming, a change in the mental processing of a stimulus as a result of prior encounter with a related stimulus, has been observed repeatedly and studied extensively in humans. Yet currently, there is no behavioral model of short-term priming in lab animals, precluding research on the neurobiological basis of priming. Here, we describe an auditory discrimination paradigm for studying response priming in freely moving mice. We find a priming effect in success rate in all mice tested on the task. In contrast, we do not find a priming effect in response times. Compared to non-primed discrimination trials, the addition of incongruent prime stimuli reduces success rate more than congruent prime stimuli, suggesting a cognitive mechanism based on differential interference. The results establish the short-term priming phenomenon in rodents, and the paradigm opens the door to studying the cellular-network basis of priming.

Top-down input modulates visual context processing through an interneuron-specific circuit

Visual stimuli that deviate from the current context elicit augmented responses in the primary visual cortex (V1). These heightened responses, known as "deviance detection," require local inhibition in the V1 and top-down input from the anterior cingulate area (ACa). Here, we investigated the mechanisms by which the ACa and V1 interact to support deviance detection. Local field potential recordings in mice during an oddball paradigm showed that ACa-V1 synchrony peaks in the theta/alpha band (≈10 Hz). Two-photon imaging in the V1 revealed that mainly pyramidal neurons exhibited deviance detection, while contextually redundant stimuli increased vasoactive intestinal peptide (VIP)-positive interneuron (VIP) activity and decreased somatostatin-positive interneuron (SST) activity. Optogenetic drive of ACa-V1 inputs at 10 Hz activated V1-VIPs but inhibited V1-SSTs, mirroring the dynamics present during the oddball paradigm. Chemogenetic inhibition of V1-VIPs disrupted Aca-V1 synchrony and deviance detection in the V1. These results outline temporal and interneuron-specific mechanisms of top-down modulation that support visual context processing.

How 'visual' is the visual cortex? The interactions between the visual cortex and other sensory, motivational and motor systems as enabling factors for visual perception

The definition of the visual cortex is primarily based on the evidence that lesions of this area impair visual perception. However, this does not exclude that the visual cortex may process more information than of retinal origin alone, or that other brain structures contribute to vision. Indeed, research across the past decades has shown that non-visual information, such as neural activity related to reward expectation and value, locomotion, working memory and other sensory modalities, can modulate primary visual cortical responses to retinal inputs. Nevertheless, the function of this non-visual information is poorly understood. Here we review recent evidence, coming primarily from studies in rodents, arguing that non-visual and motor effects in visual cortex play a role in visual processing itself, for instance disentangling direct auditory effects on visual cortex from effects of sound-evoked orofacial movement. These findings are placed in a broader framework casting vision in terms of predictive processing under control of frontal, reward- and motor-related systems. In contrast to the prevalent notion that vision is exclusively constructed by the visual cortical system, we propose that visual percepts are generated by a larger network-the extended visual system-spanning other sensory cortices, supramodal areas and frontal systems. This article is part of the theme issue 'Decision and control processes in multisensory perception'.

Robust multisensory deviance detection in the mouse parietal associative area

Context modulates how information is processed in the mammalian brain. For example, brain responses are amplified to contextually unusual stimuli. This phenomenon, known as "deviance detection,"1,2 is well documented in early, primary sensory cortex, where large responses are generated to simple stimuli that deviate from their context in low-order properties, such as line orientation, size, or pitch.2,3,4,5 However, the extent to which neural deviance detection manifests (1) in broader cortical networks and (2) to simple versus complex stimuli, which deviate only in their higher-order, multisensory properties, is not known. Consistent with a predictive processing framework,6,7 we hypothesized that deviance detection manifests in a hierarchical manner across cortical networks,8,9 emerging later and further downstream when stimulus deviance is complex. To test this, we examined brain responses of awake mice to simple unisensory deviants (e.g., visual line gratings, deviating from context in their orientation alone) versus complex multisensory deviants (i.e., audiovisual pairs, deviating from context only in their audiovisual pairing but not visual or auditory content alone). We find that mouse parietal associative area-a higher cortical region-displays robust multisensory deviance detection. In contrast, primary visual cortex exhibits strong unisensory visual deviance detection but weaker multisensory deviance detection. These results suggest that deviance detection signals in the cortex may be conceptualized as "prediction errors," which are primarily fed forward-or downstream-in cortical networks.6,7.

Fast updating feedback from piriform cortex to the olfactory bulb relays multimodal reward contingency signals during rule-reversal

While animals readily adjust their behavior to adapt to relevant changes in the environment, the neural pathways enabling these changes remain largely unknown. Here, using multiphoton imaging, we investigated whether feedback from the piriform cortex to the olfactory bulb supports such behavioral flexibility. To this end, we engaged head-fixed mice in a multimodal rule-reversal task guided by olfactory and auditory cues. Both odor and, surprisingly, the sound cues triggered cortical bulbar feedback responses which preceded the behavioral report. Responses to the same sensory cue were strongly modulated upon changes in stimulus-reward contingency (rule reversals). The re-shaping of individual bouton responses occurred within seconds of the rule-reversal events and was correlated with changes in the behavior. Optogenetic perturbation of cortical feedback within the bulb disrupted the behavioral performance. Our results indicate that the piriform-to-olfactory bulb feedback carries reward contingency signals and is rapidly re-formatted according to changes in the behavioral context.

The spatial and temporal structure of neural activity across the fly brain

What are the spatial and temporal scales of brainwide neuronal activity? We used swept, confocally-aligned planar excitation (SCAPE) microscopy to image all cells in a large volume of the brain of adult Drosophila with high spatiotemporal resolution while flies engaged in a variety of spontaneous behaviors. This revealed neural representations of behavior on multiple spatial and temporal scales. The activity of most neurons correlated (or anticorrelated) with running and flailing over timescales that ranged from seconds to a minute. Grooming elicited a weaker global response. Significant residual activity not directly correlated with behavior was high dimensional and reflected the activity of small clusters of spatially organized neurons that may correspond to genetically defined cell types. These clusters participate in the global dynamics, indicating that neural activity reflects a combination of local and broadly distributed components. This suggests that microcircuits with highly specified functions are provided with knowledge of the larger context in which they operate.

Neural mechanisms underlying uninstructed orofacial movements during reward-based learning behaviors

During reward-based learning tasks, animals make orofacial movements that globally influence brain activity at the timings of reward expectation and acquisition. These orofacial movements are not explicitly instructed and typically appear along with goal-directed behaviors. Here, we show that reinforcing optogenetic stimulation of dopamine neurons in the ventral tegmental area (oDAS) in mice is sufficient to induce orofacial movements in the whiskers and nose without accompanying goal-directed behaviors. Pavlovian conditioning with a sensory cue and oDAS elicited cue-locked and oDAS-aligned orofacial movements, which were distinguishable by a machine-learning model. Inhibition or knockout of dopamine D1 receptors in the nucleus accumbens inhibited oDAS-induced motion but spared cue-locked motion, suggesting differential regulation of these two types of orofacial motions. In contrast, inactivation of the whisker primary motor cortex (wM1) abolished both types of orofacial movements. We found specific neuronal populations in wM1 representing either oDAS-aligned or cue-locked whisker movements. Notably, optogenetic stimulation of wM1 neurons successfully replicated these two types of movements. Our results thus suggest that accumbal D1-receptor-dependent and -independent neuronal signals converge in the wM1 for facilitating distinct uninstructed orofacial movements during a reward-based learning task.

Mouse frontal cortex mediates additive multisensory decisions

The brain can combine auditory and visual information to localize objects. However, the cortical substrates underlying audiovisual integration remain uncertain. Here, we show that mouse frontal cortex combines auditory and visual evidence; that this combination is additive, mirroring behavior; and that it evolves with learning. We trained mice in an audiovisual localization task. Inactivating frontal cortex impaired responses to either sensory modality, while inactivating visual or parietal cortex affected only visual stimuli. Recordings from >14,000 neurons indicated that after task learning, activity in the anterior part of frontal area MOs (secondary motor cortex) additively encodes visual and auditory signals, consistent with the mice's behavioral strategy. An accumulator model applied to these sensory representations reproduced the observed choices and reaction times. These results suggest that frontal cortex adapts through learning to combine evidence across sensory cortices, providing a signal that is transformed into a binary decision by a downstream accumulator.

A novel task to investigate vibrotactile detection in mice

Throughout the last decades, understanding the neural mechanisms of sensory processing has been a key objective for neuroscientists. Many studies focused on uncovering the microcircuit-level architecture of somatosensation using the rodent whisker system as a model. Although these studies have significantly advanced our understanding of tactile processing, the question remains to what extent the whisker system can provide results translatable to the human somatosensory system. To address this, we developed a restrained vibrotactile detection task involving the limb system in mice. A vibrotactile stimulus was delivered to the hindlimb of head-fixed mice, who were trained to perform a Go/No-go detection task. Mice were able to learn this task with satisfactory performance and with reasonably short training times. In addition, the task we developed is versatile, as it can be combined with diverse neuroscience methods. Thus, this study introduces a novel task to study the neuron-level mechanisms of tactile processing in a system other than the more commonly studied whisker system.

Sound Improves Neuronal Encoding of Visual Stimuli in Mouse Primary Visual Cortex

In everyday life, we integrate visual and auditory information in routine tasks such as navigation and communication. While concurrent sound can improve visual perception, the neuronal correlates of audiovisual integration are not fully understood. Specifically, it remains unclear whether neuronal firing patters in the primary visual cortex (V1) of awake animals demonstrate similar sound-induced improvement in visual discriminability. Furthermore, presentation of sound is associated with movement in the subjects, but little is understood about whether and how sound-associated movement affects audiovisual integration in V1. Here, we investigated how sound and movement interact to modulate V1 visual responses in awake, head-fixed mice and whether this interaction improves neuronal encoding of the visual stimulus. We presented visual drifting gratings with and without simultaneous auditory white noise to awake mice while recording mouse movement and V1 neuronal activity. Sound modulated activity of 80% of light-responsive neurons, with 95% of neurons increasing activity when the auditory stimulus was present. A generalized linear model (GLM) revealed that sound and movement had distinct and complementary effects of the neuronal visual responses. Furthermore, decoding of the visual stimulus from the neuronal activity was improved with sound, an effect that persisted even when controlling for movement. These results demonstrate that sound and movement modulate visual responses in complementary ways, improving neuronal representation of the visual stimulus. This study clarifies the role of movement as a potential confound in neuronal audiovisual responses and expands our knowledge of how multimodal processing is mediated at a neuronal level in the awake brain.SIGNIFICANCE STATEMENT Sound and movement are both known to modulate visual responses in the primary visual cortex; however, sound-induced movement has largely remained unaccounted for as a potential confound in audiovisual studies in awake animals. Here, authors found that sound and movement both modulate visual responses in an important visual brain area, the primary visual cortex, in distinct, yet complementary ways. Furthermore, sound improved encoding of the visual stimulus even when accounting for movement. This study reconciles contrasting theories on the mechanism underlying audiovisual integration and asserts the primary visual cortex as a key brain region participating in tripartite sensory interactions.

Multimodal mismatch responses in associative but not primary visual cortex support hierarchical predictive coding in cortical networks

A key function of the mammalian neocortex is to process sensory data in the context of current and past stimuli. Primary sensory cortices, such as V1, respond weakly to stimuli that typical in their context but strongly to novel stimuli, an effect known as "deviance detection". How deviance detection occurs in associative cortical regions that are downstream of V1 is not well-understood. Here we investigated parietal associative area (PTLp) responses to auditory, visual, and audio-visual mismatches with two-photon calcium imaging and local field potential recordings. We employed basic unisensory auditory and visual oddball paradigms as well as a novel multisensory oddball paradigm, involving typical parings (VaAc or VbAd) presented at p=.88 with rare "deviant" pairings (e.g. VaAd or VbAc) presented at p=.12. We found that PTLp displayed robust deviance detection responses to auditory-visual mismatches, both in individual neurons and in population theta and gamma-band oscillations. In contrast, V1 neurons displayed deviance detection only to visual deviants in a unisensory context, but not to auditory or auditory-visual mismatches. Taken together, these results accord with a predictive processing framework for cortical responses, wherein modality specific prediction errors (i.e. deviance detection responses) are computed in functionally specified cortical areas and feed-forward to update higher brain regions.