Stimulus relevance modulates contrast adaptation in visual cortex

Visual routines for detecting causal interactions are tuned to motion direction

Detecting causal relations structures our perception of events in the world. Here, we determined for visual interactions whether generalized (i.e. feature-invariant) or specialized (i.e. feature-selective) visual routines underlie the perception of causality. To this end, we applied a visual adaptation protocol to assess the adaptability of specific features in classical launching events of simple geometric shapes. We asked observers to report whether they observed a launch or a pass in ambiguous test events (i.e. the overlap between two discs varied from trial to trial). After prolonged exposure to causal launch events (the adaptor) defined by a particular set of features (i.e. a particular motion direction, motion speed, or feature conjunction), observers were less likely to see causal launches in subsequent ambiguous test events than before adaptation. Crucially, adaptation was contingent on the causal impression in launches as demonstrated by a lack of adaptation in non-causal control events. We assessed whether this negative aftereffect transfers to test events with a new set of feature values that were not presented during adaptation. Processing in specialized (as opposed to generalized) visual routines predicts that the transfer of visual adaptation depends on the feature similarity of the adaptor and the test event. We show that the negative aftereffects do not transfer to unadapted launch directions but do transfer to launch events of different speeds. Finally, we used colored discs to assign distinct feature-based identities to the launching and the launched stimulus. We found that the adaptation transferred across colors if the test event had the same motion direction as the adaptor. In summary, visual adaptation allowed us to carve out a visual feature space underlying the perception of causality and revealed specialized visual routines that are tuned to a launch's motion direction.

Adaptation shapes the representational geometry in mouse V1 to efficiently encode the environment

Sensory adaptation dynamically changes neural responses as a function of previous stimuli, profoundly impacting perception. The response changes induced by adaptation have been characterized in detail in individual neurons and at the population level after averaging across trials. However, it is not clear how adaptation modifies the aspects of the representations that relate more directly to the ability to perceive stimuli, such as their geometry and the noise structure in individual trials. To address this question, we recorded from a population of neurons in the mouse visual cortex and presented one stimulus (an oriented grating) more frequently than the others. We then analyzed these data in terms of representational geometry and studied the ability of a linear decoder to discriminate between similar visual stimuli based on the single-trial population responses. Surprisingly, the discriminability of stimuli near the adaptor increased, even though the responses of individual neurons to these stimuli decreased. Similar changes were observed in artificial neural networks trained to reconstruct the visual stimulus under metabolic constraints. We conclude that the paradoxical effects of adaptation are consistent with the efficient coding framework, allowing the brain to improve the representation of frequent stimuli while limiting the associated metabolic cost.

Stability of cross-sensory input to primary somatosensory cortex across experience

Merging information across sensory modalities is key to forming robust percepts, yet how the brain achieves this feat remains unclear. Recent studies report cross-modal influences in the primary sensory cortex, suggesting possible multisensory integration in the early stages of cortical processing. We test several hypotheses about the function of auditory influences on mouse primary somatosensory cortex (S1) using in vivo two-photon calcium imaging. We found sound-evoked spiking activity in an extremely small fraction of cells, and this sparse activity did not encode auditory stimulus identity. Moreover, S1 did not encode information about specific audio-tactile feature conjunctions. Auditory and audio-tactile stimulus encoding remained unchanged after both passive experience and reinforcement. These results suggest that while primary sensory cortex is plastic within its own modality, the influence of other modalities is remarkably stable and stimulus nonspecific.

Temporal regularities shape perceptual decisions and striatal dopamine signals

Perceptual decisions should depend on sensory evidence. However, such decisions are also influenced by past choices and outcomes. These choice history biases may reflect advantageous strategies to exploit temporal regularities of natural environments. However, it is unclear whether and how observers can adapt their choice history biases to different temporal regularities, to exploit the multitude of temporal correlations that exist in nature. Here, we show that male mice adapt their perceptual choice history biases to different temporal regularities of visual stimuli. This adaptation was slow, evolving over hundreds of trials across several days. It occurred alongside a fast non-adaptive choice history bias, limited to a few trials. Both fast and slow trial history effects are well captured by a normative reinforcement learning algorithm with multi-trial belief states, comprising both current trial sensory and previous trial memory states. We demonstrate that dorsal striatal dopamine tracks predictions of the model and behavior, suggesting that striatal dopamine reports reward predictions associated with adaptive choice history biases. Our results reveal the adaptive nature of perceptual choice history biases and shed light on their underlying computational principles and neural correlates.

Stability of cross-sensory input to primary somatosensory cortex across experience

Merging information from across sensory modalities is key to forming robust, disambiguated percepts of the world, yet how the brain achieves this feat remains unclear. Recent observations of cross-modal influences in primary sensory cortical areas have suggested that multisensory integration may occur in the earliest stages of cortical processing, but the role of these responses is still poorly understood. We address these questions by testing several hypotheses about the possible functions served by auditory influences on the barrel field of mouse primary somatosensory cortex (S1) using in vivo 2-photon calcium imaging. We observed sound-evoked spiking activity in a small fraction of cells overall, and moreover that this sparse activity was insufficient to encode auditory stimulus identity; few cells responded preferentially to one sound or another, and a linear classifier trained to decode auditory stimuli from population activity performed barely above chance. Moreover S1 did not encode information about specific audio-tactile feature conjunctions that we tested. Our ability to decode auditory audio-tactile stimuli from neural activity remained unchanged after both passive experience and reinforcement. Collectively, these results suggest that while a primary sensory cortex is highly plastic with regard to its own modality, the influence of other modalities are remarkably stable and play a largely stimulus-non-specific role.

Learning enhances encoding of time and temporal surprise in mouse primary sensory cortex

Primary sensory cortex has long been believed to play a straightforward role in the initial processing of sensory information. Yet, the superficial layers of cortex overall are sparsely active, even during sensory stimulation; additionally, cortical activity is influenced by other modalities, task context, reward, and behavioral state. Our study demonstrates that reinforcement learning dramatically alters representations among longitudinally imaged neurons in superficial layers of mouse primary somatosensory cortex. Learning an object detection task recruits previously unresponsive neurons, enlarging the neuronal population sensitive to touch and behavioral choice. Cortical responses decrease upon repeated stimulus presentation outside of the behavioral task. Moreover, training improves population encoding of the passage of time, and unexpected deviations in trial timing elicit even stronger responses than touches do. In conclusion, the superficial layers of sensory cortex exhibit a high degree of learning-dependent plasticity and are strongly modulated by non-sensory but behaviorally-relevant features, such as timing and surprise.

Activity in the superficial layers of the sensory cortex is believed to be largely driven by incoming sensory stimuli. Here the authors demonstrate how learning changes neural responses to sensations according to both behavioral relevance and timing, suggesting a high degree of non-sensory modulation.

Selective enhancement of neural coding in V1 underlies fine-discrimination learning in tree shrew

Visual discrimination improves with training, a phenomenon that is thought to reflect plastic changes in the responses of neurons in primary visual cortex (V1). However, the identity of the neurons that undergo change, the nature of the changes, and the consequences of these changes for other visual behaviors remain unclear. We used chronic in vivo 2-photon calcium imaging to monitor the responses of neurons in the V1 of tree shrews learning a Go/No-Go fine orientation discrimination task. We observed increases in neural population measures of discriminability for task-relevant stimuli that correlate with performance and depend on a select subset of neurons with preferred orientations that include the rewarded stimulus and nearby orientations biased away from the non-rewarded stimulus. Learning is accompanied by selective enhancement in the response of these neurons to the rewarded stimulus that further increases their ability to discriminate the task stimuli. These changes persist outside of the trained task and predict observed enhancement and impairment in performance of other discriminations, providing evidence for selective and persistent learning-induced plasticity in the V1, with significant consequences for perception.

Not so spontaneous: Multi-dimensional representations of behaviors and context in sensory areas

Sensory areas are spontaneously active in the absence of sensory stimuli. This spontaneous activity has long been studied; however, its functional role remains largely unknown. Recent advances in technology, allowing large-scale neural recordings in the awake and behaving animal, have transformed our understanding of spontaneous activity. Studies using these recordings have discovered high-dimensional spontaneous activity patterns, correlation between spontaneous activity and behavior, and dissimilarity between spontaneous and sensory-driven activity patterns. These findings are supported by evidence from developing animals, where a transition toward these characteristics is observed as the circuit matures, as well as by evidence from mature animals across species. These newly revealed characteristics call for the formulation of a new role for spontaneous activity in neural sensory computation.

Existing function in primary visual cortex is not perturbed by new skill acquisition of a non-matched sensory task

Acquisition of new skills has the potential to disturb existing network function. To directly assess whether previously acquired cortical function is altered during learning, mice were trained in an abstract task in which selected activity patterns were rewarded using an optical brain-computer interface device coupled to primary visual cortex (V1) neurons. Excitatory neurons were longitudinally recorded using 2-photon calcium imaging. Despite significant changes in local neural activity during task performance, tuning properties and stimulus encoding assessed outside of the trained context were not perturbed. Similarly, stimulus tuning was stable in neurons that remained responsive following a different, visual discrimination training task. However, visual discrimination training increased the rate of representational drift. Our results indicate that while some forms of perceptual learning may modify the contribution of individual neurons to stimulus encoding, new skill learning is not inherently disruptive to the quality of stimulus representation in adult V1.

Recruitment of frontal sensory circuits during visual discrimination

A long-range circuit linking the medial frontal cortex to the primary visual cortex (V1) has been proposed to mediate visual selective attention in mice during visually guided behavior. Here, we use in vivo two-photon functional imaging to measure the endogenous activity of axons of A24b/M2 neurons from this region projecting to layer 1 of V1 (A24b/M2-V1_axons) in mice either passively viewing stimuli or performing a go/no-go visually guided task. We observe that while A24b/M2-V1_axons are recruited under these conditions, this is not linked to enhancement of neural or behavioral measures of sensory coding. Instead, A24b/M2-V1_axon activity is associated with licking behavior, modulated by reward, and biased toward the sensory cortical hemisphere representing the stimulus currently being discriminated.

•

Endogenous A24b/M2-V1 axon activity elevation does not enhance sensory processing

•

A24b/M2-V1 axon activity is correlated with licking

•

A24b/M2-V1 axon lick-correlated activity is modulated by reward

Opposite forms of adaptation in mouse visual cortex are controlled by distinct inhibitory microcircuits

Sensory processing in the cortex adapts to the history of stimulation but the mechanisms are not understood. Imaging the primary visual cortex of mice we find here that an increase in stimulus contrast is not followed by a simple decrease in gain of pyramidal cells; as many cells increase gain to improve detection of a subsequent decrease in contrast. Depressing and sensitizing forms of adaptation also occur in different types of interneurons (PV, SST and VIP) and the net effect within individual pyramidal cells reflects the balance of PV inputs, driving depression, and a subset of SST interneurons driving sensitization. Changes in internal state associated with locomotion increase gain across the population of pyramidal cells while maintaining the balance between these opposite forms of plasticity, consistent with activation of both VIP->SST and SST->PV disinhibitory pathways. These results reveal how different inhibitory microcircuits adjust the gain of pyramidal cells signalling changes in stimulus strength.

The authors describe the role of inhibitory microcircuits in the visual cortex of mice in adaptation to contrast. They show how external stimuli and internal state interact to adjust processing in the visual cortex.

A cortical circuit for audio-visual predictions

Learned associations between stimuli in different sensory modalities can shape the way we perceive these stimuli. However, it is not well understood how these interactions are mediated or at what level of the processing hierarchy they occur. Here we describe a neural mechanism by which an auditory input can shape visual representations of behaviorally relevant stimuli through direct interactions between auditory and visual cortices in mice. We show that the association of an auditory stimulus with a visual stimulus in a behaviorally relevant context leads to experience-dependent suppression of visual responses in primary visual cortex (V1). Auditory cortex axons carry a mixture of auditory and retinotopically matched visual input to V1, and optogenetic stimulation of these axons selectively suppresses V1 neurons that are responsive to the associated visual stimulus after, but not before, learning. Our results suggest that cross-modal associations can be communicated by long-range cortical connections and that, with learning, these cross-modal connections function to suppress responses to predictable input.

The contribution of low contrast-preferring neurons to information representation in the primary visual cortex after learning

Animals exhibit improved perception of lower-contrast visual objects after training. We explored this neuronal mechanism using multiple single-unit recordings from deep layers of the primary visual cortex (V1) of trained rats during orientation discrimination. We found that the firing rates of a subset of neurons increased by reducing luminance contrast, being at least above basal activities at low contrast. These low contrast–preferring neurons were rare during passive viewing without training or anesthesia after training. They fired more frequently in correct-choice than incorrect-choice trials. At single-neuron and population levels, they efficiently represented low-contrast orientations. Following training, in addition to generally enhanced excitation, the phase synchronization of spikes to beta oscillations at high contrast was stronger in putative inhibitory than excitatory neurons. The change in excitation-inhibition balance might contribute to low-contrast preference. Thus, low-contrast preference in V1 activity is strengthened in an experience-dependent manner, which may contribute to low-contrast visual discrimination.

The generation of cortical novelty responses through inhibitory plasticity

Animals depend on fast and reliable detection of novel stimuli in their environment. Neurons in multiple sensory areas respond more strongly to novel in comparison to familiar stimuli. Yet, it remains unclear which circuit, cellular, and synaptic mechanisms underlie those responses. Here, we show that spike-timing-dependent plasticity of inhibitory-to-excitatory synapses generates novelty responses in a recurrent spiking network model. Inhibitory plasticity increases the inhibition onto excitatory neurons tuned to familiar stimuli, while inhibition for novel stimuli remains low, leading to a network novelty response. The generation of novelty responses does not depend on the periodicity but rather on the distribution of presented stimuli. By including tuning of inhibitory neurons, the network further captures stimulus-specific adaptation. Finally, we suggest that disinhibition can control the amplification of novelty responses. Therefore, inhibitory plasticity provides a flexible, biologically plausible mechanism to detect the novelty of bottom-up stimuli, enabling us to make experimentally testable predictions.

Active dendrites enable strong but sparse inputs to determine orientation selectivity

The dendrites of neocortical pyramidal neurons are excitable. However, it is unknown how synaptic inputs engage nonlinear dendritic mechanisms during sensory processing in vivo, and how they in turn influence action potential output. Here, we provide a quantitative account of the relationship between synaptic inputs, nonlinear dendritic events, and action potential output. We developed a detailed pyramidal neuron model constrained by in vivo dendritic recordings. We drive this model with realistic input patterns constrained by sensory responses measured in vivo and connectivity measured in vitro. We show mechanistically that under realistic conditions, dendritic Na+ and NMDA spikes are the major determinants of neuronal output in vivo. We demonstrate that these dendritic spikes can be triggered by a surprisingly small number of strong synaptic inputs, in some cases even by single synapses. We predict that dendritic excitability allows the 1% strongest synaptic inputs of a neuron to control the tuning of its output. Active dendrites therefore allow smaller subcircuits consisting of only a few strongly connected neurons to achieve selectivity for specific sensory features.

Rethinking Single Neuron Electrical Compartmentalization: Dendritic Contributions to Network Computation In Vivo

Decades of experimental and theoretical work support a now well-established theory that active dendritic processing contributes to the computational power of individual neurons. This theory is based on the high degree of electrical compartmentalization observed in the dendrites of single neurons in ex vivo preparations. Compartmentalization allows dendrites to conduct semi-independent operations on their inputs before final integration and output at the axon, producing a "network-in-a-neuron." However, recent in vivo functional imaging experiments in mouse cortex have reported surprisingly little evidence for strong dendritic compartmentalization. In this review, we contextualize these new findings and discuss their impact on the future of the field. Specifically, we consider how highly coordinated, and thus less compartmentalized, activity in soma and dendrites can contribute to cortical computations including nonlinear mixed selectivity, prediction/expectation, multiplexing, and credit assignment.

Interocular velocity cues elicit vergence eye movements in mice

We stabilize the dynamic visual world on our retina by moving our eyes in response to motion signals. Coordinated movements between the two eyes are characterized as version when both eyes move in the same direction and vergence when the two eyes move in opposite directions. Vergence eye movements are necessary to track objects in three dimensions. In primates they can be elicited by intraocular differences in either spatial signals (disparity) or velocity, requiring the integration of left and right eye inputs. Whether mice are capable of similar behaviors is not known. To address this issue, we measured vergence eye movements in mice using a stereoscopic stimulus known to elicit vergence eye movements in primates. We found that mice also exhibit vergence eye movements, although at a low gain and that the primary driver of these vergence eye movements is interocular motion. Spatial disparity cues alone are ineffective. We also found that the vergence eye movements we observed in mice were robust to silencing visual cortex and to manipulations that disrupt the normal development of binocularity in visual cortex. A sublinear combination of motor commands driven by monocular signals is sufficient to account for our results.NEW & NOTEWORTHY The visual system integrates signals from the left and right eye to generate a representation of the world in depth. The binocular integration of signals may be observed from the coordinated vergence eye movements elicited by object motion in depth. We explored the circuits and signals responsible for these vergence eye movements in rodent and find these vergence eye movements are generated by a comparison of the motion and not spatial visual signals.

Feedback generates a second receptive field in neurons of the visual cortex

Animals sense the environment through pathways that link sensory organs to the brain. In the visual system, these feedforward pathways define the classical feedforward receptive field (ffRF), the area in space in which visual stimuli excite a neuron1. The visual system also uses visual context-the visual scene surrounding a stimulus-to predict the content of the stimulus2, and accordingly, neurons have been identified that are excited by stimuli outside their ffRF3-8. However, the mechanisms that generate excitation to stimuli outside the ffRF are unclear. Here we show that feedback projections onto excitatory neurons in the mouse primary visual cortex generate a second receptive field that is driven by stimuli outside the ffRF. The stimulation of this feedback receptive field (fbRF) elicits responses that are slower and are delayed in comparison with those resulting from the stimulation of the ffRF. These responses are preferentially reduced by anaesthesia and by silencing higher visual areas. Feedback inputs from higher visual areas have scattered receptive fields relative to their putative targets in the primary visual cortex, which enables the generation of the fbRF. Neurons with fbRFs are located in cortical layers that receive strong feedback projections and are absent in the main input layer, which is consistent with a laminar processing hierarchy. The observation that large, uniform stimuli-which cover both the fbRF and the ffRF-suppress these responses indicates that the fbRF and the ffRF are mutually antagonistic. Whereas somatostatin-expressing inhibitory neurons are driven by these large stimuli, inhibitory neurons that express parvalbumin and vasoactive intestinal peptide have mutually antagonistic fbRF and ffRF, similar to excitatory neurons. Feedback projections may therefore enable neurons to use context to estimate information that is missing from the ffRF and to report differences in stimulus features across visual space, regardless of whether excitation occurs inside or outside the ffRF. By complementing the ffRF, the fbRF that we identify here could contribute to predictive processing.

Reward Association Enhances Stimulus-Specific Representations in Primary Visual Cortex

The potential for neuronal representations of external stimuli to be modified by previous experience is critical for efficient sensory processing and improved behavioral outcomes. To investigate how repeated exposure to a visual stimulus affects its representation in mouse primary visual cortex (V1), we performed two-photon calcium imaging of layer 2/3 neurons and assessed responses before, during, and after the presentation of a repetitive stimulus over 5 consecutive days. We found a stimulus-specific enhancement of the neuronal representation of the repetitively presented stimulus when it was associated with a reward. This was observed both after mice actively learned a rewarded task and when the reward was randomly received. Stimulus-specific enhanced representation resulted both from neurons gaining selectivity and from increased response reliability in previously selective neurons. In the absence of reward, there was either no change in stimulus representation or a decreased representation when the stimulus was viewed at a fixed temporal frequency. Pairing a second stimulus with a reward led to a similar enhanced representation and increased discriminability between the equally rewarded stimuli. Single-neuron responses showed that separate subpopulations discriminated between the two rewarded stimuli depending on whether the stimuli were displayed in a virtual environment or viewed on a single screen. We suggest that reward-associated responses enable the generalization of enhanced stimulus representation across these V1 subpopulations. We propose that this dynamic regulation of visual processing based on the behavioral relevance of sensory input ultimately enhances and stabilizes the representation of task-relevant features while suppressing responses to non-relevant stimuli.

Control of visual adaptation depends upon task

The visual system optimizes its functioning for a given environment through processes collectively called adaptation. It is currently unknown, however, whether adaptation is affected by the particular task the observer performs within that environment. Two experiments tested whether this is the case. Observers adapted to high contrast grating patterns, and the decay of adaptation was measured using a version of the tilt-aftereffect, while they performed two different secondary tasks. One task involved judging the luminance of a small circular spot at fixation, and was expected to be unaffected by adaptation. The other secondary task involved judging a low contrast grating, and adaptation was expected to make this task difficult by reducing the visibility of the grating. Identical displays containing both a fixation spot and a grating were used for both tasks. Tilt-aftereffects were smaller when subjects concurrently performed the grating task than when they performed the fixation task. These results suggest that the control of adaptation, in this case its decay, is sensitive to the nature of the task the observer is performing. Adaptation may attempt to optimize vision with respect to many different criteria simultaneously; task is likely one of the criteria included in this process.

Audio-visual experience strengthens multisensory assemblies in adult mouse visual cortex

We experience the world through multiple senses simultaneously. To better understand mechanisms of multisensory processing we ask whether inputs from two senses (auditory and visual) can interact and drive plasticity in neural-circuits of the primary visual cortex (V1). Using genetically-encoded voltage and calcium indicators, we find coincident audio-visual experience modifies both the supra and subthreshold response properties of neurons in L2/3 of mouse V1. Specifically, we find that after audio-visual pairing, a subset of multimodal neurons develops enhanced auditory responses to the paired auditory stimulus. This cross-modal plasticity persists over days and is reflected in the strengthening of small functional networks of L2/3 neurons. We find V1 processes coincident auditory and visual events by strengthening functional associations between feature specific assemblies of multimodal neurons during bouts of sensory driven co-activity, leaving a trace of multisensory experience in the cortical network.

Embodied gestalts: Unstable visual phenomena become stable when they are stimuli for competitive action selection

An animal's environment is rich with affordances. Different possible actions are specified by visual information while competing for dominance over neural dynamics. Affordance competition models account for this in terms of winner-takes-all cross-inhibition dynamics. Multistable phenomena also reveal how the visual system deals with ambiguity. Their key property is spontaneous instability, in forms such as alternating dominance in binocular rivalry. Theoretical models of self-inhibition or self-organized instability posit that the instability is tied to some kind of neural adaptation and that its functional significance is to enable flexible perceptual transitions. We hypothesized that the two perspectives are interlinked. Spontaneous instability is an intrinsic property of perceptual systems, but it is revealed when they are stripped from the constraints of possibilities for action. To test this, we compared a multistable gestalt phenomenon against its embodied version and estimated the neural adaptation and competition parameters of an affordance transition dynamic model. Wertheimer's (Zeitschrift fur Psychologie 61, 161-265, 1912) optimal (β) and pure (φ) forms of apparent motion from a stroboscopic point-light display were endowed with action relevance by embedding the display in a visual object-tracking task. Thus, each mode was complemented by its action, because each perceptual mode uniquely enabled different ways of tracking the target. Perceptual judgment of the traditional apparent motion exhibited spontaneous instabilities, in the form of earlier switching when the frame rate was changed stepwise. In contrast, the embodied version exhibited hysteresis, consistent with affordance transition studies. Consistent with our predictions, the parameter for competition between modes in the affordance transition model increased, and the parameter for self-inhibition vanished.

Neuronal Adaptation Reveals a Suboptimal Decoding of Orientation Tuned Populations in the Mouse Visual Cortex

Sensory information is encoded by populations of cortical neurons. Yet, it is unknown how this information is used for even simple perceptual choices such as discriminating orientation. To determine the computation underlying this perceptual choice, we took advantage of the robust visual adaptation in mouse primary visual cortex (V1). We first designed a stimulus paradigm in which we could vary the degree of neuronal adaptation measured in V1 during an orientation discrimination task. We then determined how adaptation affects task performance for mice of both sexes and tested which neuronal computations are most consistent with the behavioral results given the adapted population responses in V1. Despite increasing the reliability of the population representation of orientation among neurons, and improving the ability of a variety of optimal decoders to discriminate target from distractor orientations, adaptation increases animals' behavioral thresholds. Decoding the animals' choice from neuronal activity revealed that this unexpected effect on behavior could be explained by an overreliance of the perceptual choice circuit on target preferring neurons and a failure to appropriately discount the activity of neurons that prefer the distractor. Consistent with this all-positive computation, we find that animals' task performance is susceptible to subtle perturbations of distractor orientation and optogenetic suppression of neuronal activity in V1. This suggests that to solve this task the circuit has adopted a suboptimal and task-specific computation that discards important task-related information.SIGNIFICANCE STATEMENT A major goal in systems neuroscience is to understand how sensory signals are used to guide behavior. This requires determining what information in sensory cortical areas is used, and how it is combined, by downstream perceptual choice circuits. Here we demonstrate that when performing a go/no-go orientation discrimination task, mice suboptimally integrate signals from orientation tuned visual cortical neurons. While they appropriately positively weight target-preferring neurons, they fail to negatively weight distractor-preferring neurons. We propose that this all-positive computation may be adopted because of its simple learning rules and faster processing, and may be a common approach to perceptual decision-making when task conditions allow.

Evidence for a central component in adaptation to chromatic light

Adaptation to environmental light allows our visual system to compensate for dynamic changes in the visual environment for avoiding everyday hazards (e.g., misreading traffic lights) and for accurate reaching. We investigated the hypothesis that adaptation to coloured light is achieved not only via photoreceptors in the retina and monocular contrast adaptation, but also by a binocular process that may occur at the level of the cerebral cortex. In the present study, to determine the role of higher-order cortical binocular processes in adaptation to coloured light, participants were adapted to chromatic light such that the duration of adaptation during monocular processing differed from that during binocular processing. A dichoptic device was used to adapt each eye independently. The extent of after-effects, measured as the distance between the neutral points before and after adaptation to coloured light, depended on the duration of adaptation not only at the monocular level but also at a higher cortical level downstream from binocular fusion. Thus, contrast adaptation to coloured light occurs on at least two levels; it is a result of monocular processes at one level and binocular processes at the other, and each type of process exhibits different temporal characteristics. The results of this study suggest a significant cortical role in adaptation to changes in lighting conditions or the optical environment, including the effects of age on the eye, and the necessity of further investigation to clarify the functional connection between chromatic adaptation by photoreceptors and chromatic adaptation by cortical systems.

Prediction error and repetition suppression have distinct effects on neural representations of visual information

Predictive coding theories argue that recent experience establishes expectations in the brain that generate prediction errors when violated. Prediction errors provide a possible explanation for repetition suppression, where evoked neural activity is attenuated across repeated presentations of the same stimulus. The predictive coding account argues repetition suppression arises because repeated stimuli are expected, whereas non-repeated stimuli are unexpected and thus elicit larger neural responses. Here, we employed electroencephalography in humans to test the predictive coding account of repetition suppression by presenting sequences of visual gratings with orientations that were expected either to repeat or change in separate blocks of trials. We applied multivariate forward modelling to determine how orientation selectivity was affected by repetition and prediction. Unexpected stimuli were associated with significantly enhanced orientation selectivity, whereas selectivity was unaffected for repeated stimuli. Our results suggest that repetition suppression and expectation have separable effects on neural representations of visual feature information.

Predictive Processing: A Canonical Cortical Computation

This perspective describes predictive processing as a computational framework for understanding cortical function in the context of emerging evidence, with a focus on sensory processing. We discuss how the predictive processing framework may be implemented at the level of cortical circuits and how its implementation could be falsified experimentally. Lastly, we summarize the general implications of predictive processing on cortical function in healthy and diseased states.

The Impact of Visual Cues, Reward, and Motor Feedback on the Representation of Behaviorally Relevant Spatial Locations in Primary Visual Cortex

The integration of visual stimuli and motor feedback is critical for successful visually guided navigation. These signals have been shown to shape neuronal activity in the primary visual cortex (V1), in an experience-dependent manner. Here, we examined whether visual, reward, and self-motion-related inputs are integrated in order to encode behaviorally relevant locations in V1 neurons. Using a behavioral task in a virtual environment, we monitored layer 2/3 neuronal activity as mice learned to locate a reward along a linear corridor. With learning, a subset of neurons became responsive to the expected reward location. Without a visual cue to the reward location, both behavioral and neuronal responses relied on self-motion-derived estimations. However, when visual cues were available, both neuronal and behavioral responses were driven by visual information. Therefore, a population of V1 neurons encode behaviorally relevant spatial locations, based on either visual cues or on self-motion feedback when visual cues are absent.

Action and learning shape the activity of neuronal circuits in the visual cortex

•

Arousal and locomotion modulate neuronal activity in primary visual cortex.

•

Neurons in primary visual cortex respond to visuomotor mismatch.

•

Experience shapes neuronal responses to familiar stimuli, reward and object location.

•

Neuronal representations of visual stimuli are modulated according to the behavioural relevance of the stimuli.

•

Neuromodulatory, top-down and thalamocortical inputs convey arousal-related and motor-related signals to primary visual cortex.

Nonsensory variables strongly influence neuronal activity in the adult mouse primary visual cortex. Neuronal responses to visual stimuli are modulated by behavioural state, such as arousal and motor activity, and are shaped by experience. This dynamic process leads to neural representations in the visual cortex that reflect stimulus familiarity, expectations of reward and object location, and mismatch between self-motion and visual-flow. The recent development of genetic tools and recording techniques in awake behaving mice has enabled the investigation of the circuit mechanisms underlying state-dependent and experience-dependent neuronal representations in primary visual cortex. These neuronal circuits involve neuromodulatory, top-down cortico-cortical and thalamocortical pathways. The functions of nonsensory signals at this early stage of visual information processing are now beginning to be unravelled.

Determining Excitatory and Inhibitory Neuronal Activity from Multimodal fMRI Data Using a Generative Hemodynamic Model

Hemodynamic responses, in general, and the blood oxygenation level-dependent (BOLD) fMRI signal, in particular, provide an indirect measure of neuronal activity. There is strong evidence that the BOLD response correlates well with post-synaptic changes, induced by changes in the excitatory and inhibitory (E-I) balance between active neuronal populations. Typical BOLD responses exhibit transients, such as the early-overshoot and post-stimulus undershoot, that can be linked to transients in neuronal activity, but they can also result from vascular uncoupling between cerebral blood flow (CBF) and venous cerebral blood volume (venous CBV). Recently, we have proposed a novel generative hemodynamic model of the BOLD signal within the dynamic causal modeling framework, inspired by physiological observations, called P-DCM (Havlicek et al., 2015). We demonstrated the generative model's ability to more accurately model commonly observed neuronal and vascular transients in single regions but also effective connectivity between multiple brain areas (Havlicek et al., 2017b). In this paper, we additionally demonstrate the versatility of the generative model to jointly explain dynamic relationships between neuronal and hemodynamic physiological variables underlying the BOLD signal using multi-modal data. For this purpose, we utilized three distinct data-sets of experimentally induced responses in the primary visual areas measured in human, cat, and monkey brain, respectively: (1) CBF and BOLD responses; (2) CBF, total CBV, and BOLD responses (Jin and Kim, 2008); and (3) positive and negative neuronal and BOLD responses (Shmuel et al., 2006). By fitting the generative model to the three multi-modal experimental data-sets, we showed that the presence or absence of dynamic features in the BOLD signal is not an unambiguous indication of presence or absence of those features on the neuronal level. Nevertheless, the generative model that takes into account the dynamics of the physiological mechanisms underlying the BOLD response allowed dissociating neuronal from vascular transients and deducing excitatory and inhibitory neuronal activity time-courses from BOLD data alone and from multi-modal data.

A Sensorimotor Circuit in Mouse Cortex for Visual Flow Predictions

The cortex is organized as a hierarchical processing structure. Feedback from higher levels of the hierarchy, known as top-down signals, have been shown to be involved in attentional and contextual modulation of sensory responses. Here we argue that top-down input to the primary visual cortex (V1) from A24b and the adjacent secondary motor cortex (M2) signals a prediction of visual flow based on motor output. A24b/M2 sends a dense and topographically organized projection to V1 that targets most neurons in layer 2/3. By imaging the activity of A24b/M2 axons in V1 of mice learning to navigate a 2D virtual environment, we found that their activity was strongly correlated with locomotion and resulting visual flow feedback in an experience-dependent manner. When mice were trained to navigate a left-right inverted virtual environment, correlations of neural activity with behavior reversed to match visual flow. These findings are consistent with a predictive coding interpretation of visual processing.