Distinguishing externally from saccade-induced motion in visual cortex

A reduced ability to discriminate social from non-social touch at the circuit level may underlie social avoidance in autism

Social touch is critical for communication to impart emotions and intentions. However, certain autistic individuals experience aversion to social touch. Here, we used Neuropixels probes to record neural responses to social vs. non-social interactions in somatosensory cortex, tail of striatum, and basolateral amygdala. We find that wild type mice show aversion to repeated presentations of an inanimate object but not of another mouse. Cortical neurons are modulated especially by touch context (social vs. object), while striatal neurons change their preference depending on whether mice could choose or not to interact. In contrast, Fmr1 knockout (KO) mice, a model of autism, find social and non-social interactions equally aversive, especially at close proximity, and their cortical/striatal neurons are less able to discriminate social valence. A linear model shows that the encoding of certain avoidance/aversive behaviors in cortical neuron activity differed between genotypes. Thus, a reduced capacity to represent social stimuli at the circuit level may underlie social avoidance in autism.

Motor and vestibular signals in the visual cortex permit the separation of self versus externally generated visual motion

Knowing whether we are moving or something in the world is moving around us is possibly the most critical sensory discrimination we need to perform. How the brain and, in particular, the visual system solves this motion-source separation problem is not known. Here, we find that motor, vestibular, and visual motion signals are used by the mouse primary visual cortex (VISp) to differentially represent the same visual flow information according to whether the head is stationary or experiencing passive versus active translation. During locomotion, we find that running suppresses running-congruent translation input and that translation signals dominate VISp activity when running and translation speed become incongruent. This cross-modal interaction between the motor and vestibular systems was found throughout the cortex, indicating that running and translation signals provide a brain-wide egocentric reference frame for computing the internally generated and actual speed of self when moving through and sensing the external world.

The Effect of Microsaccades in the Primary Visual Cortex: Increased Synchronization in the Fovea during a Two-Phase Response Modulation

Our eyes are never still. Even when we attempt to fixate, the visual gaze is never motionless, as we continuously perform miniature oculomotor movements termed as fixational eye movements. The fastest eye movements during the fixation epochs are termed microsaccades (MSs) that are leading to continual motion of the visual input, affecting mainly neurons in the fovea. Yet our vision appears to be stable. To explain this gap, previous studies suggested the existence of an extraretinal input (ERI) into the visual cortex that can account for the motion and produce visual stability. Here, we investigated the existence of an ERI to V1 fovea in macaque monkeys (male) while they performed spontaneous MSs, during fixation. We used voltage-sensitive dye imaging (VSDI) to measure and characterize at high spatiotemporal resolution the influence of MSs on neural population activity, in the foveal region of the primary visual cortex (V1). Microsaccades, performed over a blank screen, induced a two-phase response modulation: an early suppression followed by an enhancement. A correlation analysis revealed a widespread foveal increase in neural synchronization, peaking around ∼100 ms after MS onset. Next, we investigated the MS effects in the presence of a small visual stimulus and found that this modulation was different from the blank condition yet both modulations coexisted in the fovea. Finally, the VSD response to an external motion of the fixation point could not explain the MS modulation. These results support an ERI that may be involved in visual stabilization already at the level of V1.

Long-term intravital subcellular imaging with confocal scanning light-field microscopy

Long-term observation of subcellular dynamics in living organisms is limited by background fluorescence originating from tissue scattering or dense labeling. Existing confocal approaches face an inevitable tradeoff among parallelization, resolution and phototoxicity. Here we present confocal scanning light-field microscopy (csLFM), which integrates axially elongated line-confocal illumination with the rolling shutter in scanning light-field microscopy (sLFM). csLFM enables high-fidelity, high-speed, three-dimensional (3D) imaging at near-diffraction-limit resolution with both optical sectioning and low phototoxicity. By simultaneous 3D excitation and detection, the excitation intensity can be reduced below 1 mW mm-2, with 15-fold higher signal-to-background ratio over sLFM. We imaged subcellular dynamics over 25,000 timeframes in optically challenging environments in different species, such as migrasome delivery in mouse spleen, retractosome generation in mouse liver and 3D voltage imaging in Drosophila. Moreover, csLFM facilitates high-fidelity, large-scale neural recording with reduced crosstalk, leading to high orientation selectivity to visual stimuli, similar to two-photon microscopy, which aids understanding of neural coding mechanisms.

A thalamic hub-and-spoke network enables visual perception during action by coordinating visuomotor dynamics

For accurate perception and motor control, an animal must distinguish between sensory experiences elicited by external stimuli and those elicited by its own actions. The diversity of behaviors and their complex influences on the senses make this distinction challenging. Here, we uncover an action-cue hub that coordinates motor commands with visual processing in the brain's first visual relay. We show that the ventral lateral geniculate nucleus (vLGN) acts as a corollary discharge center, integrating visual translational optic flow signals with motor copies from saccades, locomotion and pupil dynamics. The vLGN relays these signals to correct action-specific visual distortions and to refine perception, as shown for the superior colliculus and in a depth-estimation task. Simultaneously, brain-wide vLGN projections drive corrective actions necessary for accurate visuomotor control. Our results reveal an extended corollary discharge architecture that refines early visual transformations and coordinates actions via a distributed hub-and-spoke network to enable visual perception during action.

Interdigitating Modules for Visual Processing During Locomotion and Rest in Mouse V1

Layer 1 of V1 has been shown to receive locomotion-related signals from the dorsal lateral geniculate (dLGN) and lateral posterior (LP) thalamic nuclei (Roth et al., 2016). Inputs from the dLGN terminate in M2+ patches while inputs from LP target M2- interpatches (D'Souza et al., 2019) suggesting that motion related signals are processed in distinct networks. Here, we investigated by calcium imaging in head-fixed awake mice whether L2/3 neurons underneath L1 M2+ and M2- modules are differentially activated by locomotion, and whether distinct networks of feedback connections from higher cortical areas to L1 may contribute to these differences. We found that strongly locomotion-modulated cell clusters during visual stimulation were aligned with M2- interpatches, while weakly modulated cells clustered under M2+ patches. Unlike M2+ patch cells, pairs of M2- interpatch cells showed increased correlated variability of calcium transients when the sites in the visuotopic map were far apart, suggesting that activity is integrated across large parts of the visual field. Pathway tracing further suggests that strong locomotion modulation in L2/3 M2- interpatch cells of V1 relies on looped, like-to-like networks between apical dendrites of MOs-, PM- and RSP-projecting neurons and feedback input from these areas to L1. M2- interpatches receive strong inputs from SST neurons, suggesting that during locomotion these interneurons influence the firing of specific subnetworks by controlling the excitability of apical dendrites in M2- interpatches.

Eye saccades align optic flow with retinal specializations during object pursuit in freely moving ferrets

During prey pursuit, how eye rotations, such as saccades, enable continuous tracking of erratically moving targets while enabling an animal to navigate through the environment is unknown. To better understand this, we measured head and eye rotations in freely running ferrets during pursuit behavior. By also tracking the target and all environmental features, we reconstructed the animal's visual fields and their relationship to retinal structures. In the reconstructed visual fields, the target position clustered on and around the high-acuity retinal area location, the area centralis, and surprisingly, this cluster was not significantly shifted by digital removal of either eye saccades, exclusively elicited when the ferrets made turns, or head rotations that were tightly synchronized with the saccades. Here, we show that, while the saccades did not fixate the moving target with the area centralis, they instead aligned the area centralis with the intended direction of travel. This also aligned the area centralis with features of the optic flow pattern, such as flow direction and focus of expansion, used for navigation by many species. While saccades initially rotated the eyes in the same direction as the head turn, saccades were followed by eye rotations countering the ongoing head rotation, which reduced image blur and limited information loss across the visual field during head turns. As we measured the same head and eye rotational relationship in freely moving tree shrews, rats, and mice, we suggest that these saccades and counter-rotations are a generalized mechanism enabling mammals to navigate complex environments during pursuit.

Learning to segment self-generated from externally caused optic flow through sensorimotor mismatch circuits

Efficient sensory detection requires the capacity to ignore task-irrelevant information, for example when optic flow patterns created by egomotion need to be disentangled from object perception. To investigate how this is achieved in the visual system, predictive coding with sensorimotor mismatch detection is an attractive starting point. Indeed, experimental evidence for sensorimotor mismatch signals in early visual areas exists, but it is not understood how they are integrated into cortical networks that perform input segmentation and categorization. Our model advances a biologically plausible solution by extending predictive coding models with the ability to distinguish self-generated from externally caused optic flow. We first show that a simple three neuron circuit produces experience-dependent sensorimotor mismatch responses, in agreement with calcium imaging data from mice. This microcircuit is then integrated into a neural network with two generative streams. The motor-to-visual stream consists of parallel microcircuits between motor and visual areas and learns to spatially predict optic flow resulting from self-motion. The second stream bidirectionally connects a motion-selective higher visual area (mHVA) to V1, assigning a crucial role to the abundant feedback connections to V1: the maintenance of a generative model of externally caused optic flow. In the model, area mHVA learns to segment moving objects from the background, and facilitates object categorization. Based on shared neurocomputational principles across species, the model also maps onto primate visual cortex. Our work extends Hebbian predictive coding to sensorimotor settings, in which the agent actively moves - and learns to predict the consequences of its own movements.

Saccade execution increases the preview effect with faces: An EEG and eye-tracking coregistration study

Under naturalistic viewing conditions, humans conduct about three to four saccadic eye movements per second. These dynamics imply that in real life, humans rarely see something completely new; there is usually a preview of the upcoming foveal input from extrafoveal regions of the visual field. In line with results from the field of reading research, we have shown with EEG and eye-tracking coregistration that an extrafoveal preview also affects postsaccadic visual object processing and facilitates discrimination. Here, we ask whether this preview effect in the fixation-locked N170, and in manual responses to the postsaccadic target face (tilt discrimination), requires saccade execution. Participants performed a gaze-contingent experiment in which extrafoveal face images could change their orientation during a saccade directed to them. In a control block, participants maintained stable gaze throughout the experiment and the extrafoveal face reappeared foveally after a simulated saccade latency. Compared with this no-saccade condition, the neural and the behavioral preview effects were much larger in the saccade condition. We also found shorter first fixation durations after an invalid preview, which is in contrast to reading studies. We interpret the increased preview effect under saccade execution as the result of the additional sensorimotor processes that come with gaze behavior compared with visual perception under stable fixation. In addition, our findings call into question whether EEG studies with fixed gaze capture key properties and dynamics of active, natural vision.

Is the impact of spontaneous movements on early visual cortex species specific?

Recent studies in non-human primates do not find pronounced signals related to the animal's own body movements in the responses of neurons in the visual cortex. This is notable because such pronounced signals have been widely observed in the visual cortex of mice. Here, we discuss factors that may contribute to the differences observed between species, such as state, slow neural drift, eccentricity, and changes in retinal input. The interpretation of movement-related signals in the visual cortex also exemplifies the challenge of identifying the sources of correlated variables. Dissecting these sources is central for understanding the functional roles of movement-related signals. We suggest a functional classification of the possible sources, aimed at facilitating cross-species comparative approaches to studying the neural mechanisms of vision during natural behavior.

Inter- and Intrahemispheric Sources of Vestibular Signals to V1

Head movements are sensed by the vestibular organs. Unlike classical senses, signals from vestibular organs are not conveyed to a dedicated cortical area but are broadcast throughout the cortex. Surprisingly, the routes taken by vestibular signals to reach the cortex are still largely uncharted. Here we show that the primary visual cortex (V1) receives real-time head movement signals - direction, velocity, and acceleration - from the ipsilateral pulvinar and contralateral visual cortex. The ipsilateral pulvinar provides the main head movement signal, with a bias toward contraversive movements (e.g. clockwise movements in left V1). Conversely, the contralateral visual cortex provides head movement signals during ipsiversive movements. Crucially, head movement variables encoded in V1 are already encoded in the pulvinar, suggesting that those variables are computed subcortically. Thus, the convergence of inter- and intrahemispheric signals endows V1 with a rich representation of the animal's head movements.

Building egocentric models of local space from retinal input

Determining the location of objects relative to ourselves is essential for interacting with the world. Neural activity in the retina is used to form a vision-independent model of the local spatial environment relative to the body. For example, when an animal navigates through a forest, it rapidly shifts its gaze to identify the position of important objects, such as a tree obstructing its path. This seemingly trivial behavior belies a sophisticated neural computation. Visual information entering the brain in a retinocentric reference frame must be transformed into an egocentric reference frame to guide motor planning and action. This, in turn, allows the animal to extract the location of the tree and plan a path around it. In this review, we explore the anatomical, physiological, and computational implementation of retinocentric-to-egocentric reference frame transformations - a research area undergoing rapid progress stimulated by an ever-expanding molecular, physiological, and computational toolbox for probing neural circuits. We begin by summarizing evidence for retinocentric and egocentric reference frames in the brains of diverse organisms, from insects to primates. Next, we cover how distance estimation contributes to creating a three-dimensional representation of local space. We then review proposed implementations of reference frame transformations across different biological and artificial neural networks. Finally, we discuss how an internal egocentric model of the environment is maintained independently of the sensory inputs from which it is derived. By comparing findings across a variety of nervous systems and behaviors, we aim to inspire new avenues for investigating the neural basis of reference frame transformation, a canonical computation critical for modeling the external environment and guiding goal-directed behavior.

Sensory History Drives Adaptive Neural Geometry in LP/Pulvinar-Prefrontal Cortex Circuits

Prior expectations guide attention and support perceptual filtering for efficient processing during decision-making. Here we show that during a visual discrimination task, mice adaptively use prior stimulus history to guide ongoing choices by estimating differences in evidence between consecutive trials (| Δ Dir |). The thalamic lateral posterior (LP)/pulvinar nucleus provides robust inputs to the Anterior Cingulate Cortex (ACC), which has been implicated in selective attention and predictive processing, but the function of the LP-ACC projection is unknown. We found that optogenetic manipulations of LP-ACC axons disrupted animals' ability to effectively estimate and use information across stimulus history, leading to | Δ Dir |-dependent ipsilateral biases. Two-photon calcium imaging of LP-ACC axons revealed an engagement-dependent low-dimensional organization of stimuli along a curved manifold. This representation was scaled by | Δ Dir | in a manner that emphasized greater deviations from prior evidence. Thus, our work identifies the LP-ACC pathway as essential for selecting and evaluating stimuli relative to prior evidence to guide decisions.

Mesoscale neuronal granular trial variability in vivo illustrated by nonlinear recurrent network in silico

Large-scale neural recording with single-neuron resolution has revealed the functional complexity of the neural systems. However, even under well-designed task conditions, the cortex-wide network exhibits highly dynamic trial variability, posing challenges to the conventional trial-averaged analysis. To study mesoscale trial variability, we conducted a comparative study between fluorescence imaging of layer-2/3 neurons in vivo and network simulation in silico. We imaged up to 40,000 cortical neurons' triggered responses by deep brain stimulus (DBS). And we build an in silico network to reproduce the biological phenomena we observed in vivo. We proved the existence of ineluctable trial variability and found it influenced by input amplitude and range. Moreover, we demonstrated that a spatially heterogeneous coding community accounts for more reliable inter-trial coding despite single-unit trial variability. A deeper understanding of trial variability from the perspective of a dynamical system may lead to uncovering intellectual abilities such as parallel coding and creativity.

Inability to pursue nonrigid motion produces instability of spatial perception

Vision generates a stable representation of space by combining retinal input with internal predictions about the visual consequences of eye movements. We report a type of nonrigid motion that disrupts the connection between eye movements and perception, causing visual instability. This motion is accurately perceived during fixation, but it cannot be pursued. Catch-up saccades are accurately directed to the moving target but the motion stimulus appears to jump in space with each saccade. Our results reveal four major findings about perception and the visuomotor system: (i) Pursuit fails for certain types of motion; (ii) pursuit and catch-up saccades are independently controlled; (iii) prediction of saccade consequences is independent from saccade control; and (iv) the visual stability of moving objects relies on similar motion mechanisms as pursuit.

Computational model of layer 2/3 in mouse primary visual cortex explains observed visuomotor mismatch response

Activity in layer 2/3 of the mouse primary visual cortex has been shown to depend both on visual input and the mouse's locomotion. Moreover, this activity is altered by a mismatch between the observed visual flow and the predicted visual flow from locomotion. Here, I present a simple computational model that explains previously reported recordings from layer 2/3 neurons in mice. In my model, layer 2/3 encodes the velocity difference between the estimate from visual flow and the prediction from locomotion using a neural population code. Moreover, I describe a hypothesized mechanism for how the brain may carry out computations of variables encoded in population codes. This mechanism may point to a general principle for computing any mathematical function in the brain.

Neural substrates for saccadic modulation of visual representations in mouse superior colliculus

How do sensory systems account for stimuli generated by natural behavior? We addressed this question by examining how an ethologically relevant class of saccades modulates visual representations in the mouse superior colliculus (SC), a key region for sensorimotor integration. We quantified saccadic modulation by recording SC responses to visual probes presented at stochastic saccade-probe latencies. Saccades significantly impacted population representations of the probes, with early enhancement that began prior to saccades and pronounced suppression for several hundred milliseconds following saccades, independent of units' visual response properties or directional tuning. To determine the cause of saccadic modulation, we presented fictive saccades that simulated the visual experience during saccades without motor output. Some units exhibited similar modulation by fictive and real saccades, suggesting a sensory-driven origin of saccadic modulation, while others had dissimilar modulation, indicating a motor contribution. These findings advance our understanding of the neural basis of natural visual coding.

Widespread receptive field remapping in early primate visual cortex

Predictive remapping of receptive fields (RFs) is thought to be one of the critical mechanisms for enforcing perceptual stability during eye movements. While RF remapping has been observed in several cortical areas, its role in early visual cortex and its consequences on the tuning properties of neurons have been poorly understood. Here, we track remapping RFs in hundreds of neurons from visual area V2 while subjects perform a cued saccade task. We find that remapping is widespread in area V2 across neurons from all recorded cortical layers and cell types. Furthermore, our results suggest that remapping RFs not only maintain but also transiently enhance their feature selectivity due to untuned suppression. Taken together, these findings shed light on the dynamics and prevalence of remapping in the early visual cortex, forcing us to revise current models of perceptual stability during saccadic eye movements.

A sensory-motor theory of the neocortex

Recent neurophysiological and neuroanatomical studies suggest a close interaction between sensory and motor processes across the neocortex. Here, I propose that the neocortex implements active predictive coding (APC): each cortical area estimates both latent sensory states and actions (including potentially abstract actions internal to the cortex), and the cortex as a whole predicts the consequences of actions at multiple hierarchical levels. Feedback from higher areas modulates the dynamics of state and action networks in lower areas. I show how the same APC architecture can explain (1) how we recognize an object and its parts using eye movements, (2) why perception seems stable despite eye movements, (3) how we learn compositional representations, for example, part-whole hierarchies, (4) how complex actions can be planned using simpler actions, and (5) how we form episodic memories of sensory-motor experiences and learn abstract concepts such as a family tree. I postulate a mapping of the APC model to the laminar architecture of the cortex and suggest possible roles for cortico-cortical and cortico-subcortical pathways.

A failure to discriminate social from non-social touch at the circuit level may underlie social avoidance in autism

Social touch is critical for communication and to impart emotions and intentions. However, certain autistic individuals experience aversion to social touch, especially when it is unwanted. We used a novel social touch assay and Neuropixels probes to compare neural responses to social vs. non-social interactions in three relevant brain regions: vibrissal somatosensory cortex, tail of striatum, and basolateral amygdala. We find that wild type (WT) mice showed aversion to repeated presentations of an inanimate object but not of another mouse. Cortical neurons cared most about touch context (social vs. object) and showed a preference for social interactions, while striatal neurons changed their preference depending on whether mice could choose or not to interact. Amygdalar and striatal neurons were preferentially modulated by forced object touch, which was the most aversive. In contrast, the Fmr1 knockout (KO) model of autism found social and non-social interactions equally aversive and displayed more aversive facial expressions to social touch when it invaded their personal space. Importantly, when Fmr1 KO mice could choose to interact, neurons in all three regions did not discriminate social valence. Thus, a failure to differentially encode social from non-social stimuli at the circuit level may underlie social avoidance in autism.

Natural visual behavior and active sensing in the mouse

In the natural world, animals use vision for a wide variety of behaviors not reflected in most laboratory paradigms. Although mice have low-acuity vision, they use their vision for many natural behaviors, including predator avoidance, prey capture, and navigation. They also perform active sensing, moving their head and eyes to achieve behavioral goals and acquire visual information. These aspects of natural vision result in visual inputs and corresponding behavioral outputs that are outside the range of conventional vision studies but are essential aspects of visual function. Here, we review recent studies in mice that have tapped into natural behavior and active sensing to reveal the computational logic of neural circuits for vision.

High-speed eye tracking based on a synchronized imaging mechanism by a dual-ring infrared lighting source

It is a challenge for conventional monocular-camera single-light source eye-tracking methods to achieve high-speed eye tracking. In this work, a dual-ring infrared lighting source was designed to achieve bright and dark pupils in high speed. The eye-tracking method used a dual-ring infrared lighting source and synchronized triggers for the even and odd camera frames to capture bright and dark pupils. A pupillary corneal reflex was calculated by the center coordinates of the Purkinje spot and the pupil. A map function was established to map the relationship between the pupillary corneal reflex and gaze spots. The gaze coordinate was calculated based on the mapping function. The average detection time of each gaze spot was 3.76 ms.

Behind mouse eyes: The function and control of eye movements in mice

The mouse visual system has become the most popular model to study the cellular and circuit mechanisms of sensory processing. However, the importance of eye movements only started to be appreciated recently. Eye movements provide a basis for predictive sensing and deliver insights into various brain functions and dysfunctions. A plethora of knowledge on the central control of eye movements and their role in perception and behaviour arose from work on primates. However, an overview of various eye movements in mice and a comparison to primates is missing. Here, we review the eye movement types described to date in mice and compare them to those observed in primates. We discuss the central neuronal mechanisms for their generation and control. Furthermore, we review the mounting literature on eye movements in mice during head-fixed and freely moving behaviours. Finally, we highlight gaps in our understanding and suggest future directions for research.

A mechanosensory feedback that uncouples external and self-generated sensory responses in the olfactory cortex

Sampling behaviors have sensory consequences that can hinder perceptual stability. In olfaction, sniffing affects early odor encoding, mimicking a sudden change in odor concentration. We examined how the inhalation speed affects the representation of odor concentration in the main olfactory cortex. Neurons combine the odor input with a global top-down signal preceding the sniff and a mechanosensory feedback generated by the air passage through the nose during inhalation. Still, the population representation of concentration is remarkably sniff invariant. This is because the mechanosensory and olfactory responses are uncorrelated within and across neurons. Thus, faster odor inhalation and an increase in concentration change the cortical activity pattern in distinct ways. This encoding strategy affords tolerance to potential concentration fluctuations caused by varying inhalation speeds. Since mechanosensory reafferences are widespread across sensory systems, the coding scheme described here may be a canonical strategy to mitigate the sensory ambiguities caused by movements.

Modular horizontal network within mouse primary visual cortex

Interactions between feedback connections from higher cortical areas and local horizontal connections within primary visual cortex (V1) were shown to play a role in contextual processing in different behavioral states. Layer 1 (L1) is an important part of the underlying network. This cell-sparse layer is a target of feedback and local inputs, and nexus for contacts onto apical dendrites of projection neurons in the layers below. Importantly, L1 is a site for coupling inputs from the outside world with internal information. To determine whether all of these circuit elements overlap in L1, we labeled the horizontal network within mouse V1 with anterograde and retrograde viral tracers. We found two types of local horizontal connections: short ones that were tangentially limited to the representation of the point image, and long ones which reached beyond the receptive field center, deep into its surround. The long connections were patchy and terminated preferentially in M2 muscarinic acetylcholine receptor-negative (M2-) interpatches. Anterogradely labeled inputs overlapped in M2-interpatches with apical dendrites of retrogradely labeled L2/3 and L5 cells, forming module-selective loops between topographically distant locations. Previous work showed that L1 of M2-interpatches receive inputs from the lateral posterior thalamic nucleus (LP) and from a feedback network from areas of the medial dorsal stream, including the secondary motor cortex. Together, these findings suggest that interactions in M2-interpatches play a role in processing visual inputs produced by object-and self-motion.

The pulvinar as a hub of visual processing and cortical integration

The pulvinar nucleus of the thalamus is a crucial component of the visual system and plays significant roles in sensory processing and cognitive integration. The pulvinar's extensive connectivity with cortical regions allows for bidirectional communication, contributing to the integration of sensory information across the visual hierarchy. Recent findings underscore the pulvinar's involvement in attentional modulation, feature binding, and predictive coding. In this review, we highlight recent advances in clarifying the pulvinar's circuitry and function. We discuss the contributions of the pulvinar to signal modulation across the global cortical network and place these findings within theoretical frameworks of cortical processing, particularly the global neuronal workspace (GNW) theory and predictive coding.

Layer 1 neocortex: Gating and integrating multidimensional signals

Layer 1 (L1) of the neocortex acts as a nexus for the collection and processing of widespread information. By integrating ascending inputs with extensive top-down activity, this layer likely provides critical information regulating how the perception of sensory inputs is reconciled with expectation. This is accomplished by sorting, directing, and integrating the complex network of excitatory inputs that converge onto L1. These signals are combined with neuromodulatory afferents and gated by the wealth of inhibitory interneurons that either are embedded within L1 or send axons from other cortical layers. Together, these interactions dynamically calibrate information flow throughout the neocortex. This review will primarily focus on L1 within the primary sensory cortex and will use these insights to understand L1 in other cortical areas.

A dynamic sequence of visual processing initiated by gaze shifts

Animals move their head and eyes as they explore the visual scene. Neural correlates of these movements have been found in rodent primary visual cortex (V1), but their sources and computational roles are unclear. We addressed this by combining head and eye movement measurements with neural recordings in freely moving mice. V1 neurons responded primarily to gaze shifts, where head movements are accompanied by saccadic eye movements, rather than to head movements where compensatory eye movements stabilize gaze. A variety of activity patterns followed gaze shifts and together these formed a temporal sequence that was absent in darkness. Gaze-shift responses resembled those evoked by sequentially flashed stimuli, suggesting a large component corresponds to onset of new visual input. Notably, neurons responded in a sequence that matches their spatial frequency bias, consistent with coarse-to-fine processing. Recordings in freely gazing marmosets revealed a similar sequence following saccades, also aligned to spatial frequency preference. Our results demonstrate that active vision in both mice and marmosets consists of a dynamic temporal sequence of neural activity associated with visual sampling.

A synaptic corollary discharge signal suppresses midbrain visual processing during saccade-like locomotion

In motor control, the brain not only sends motor commands to the periphery, but also generates concurrent internal signals known as corollary discharge (CD) that influence sensory information processing around the time of movement. CD signals are important for identifying sensory input arising from self-motion and to compensate for it, but the underlying mechanisms remain unclear. Using whole-cell patch clamp recordings from neurons in the zebrafish optic tectum, we discovered an inhibitory synaptic signal, temporally locked to spontaneous and visually driven locomotion. This motor-related inhibition was appropriately timed to counteract visually driven excitatory input arising from the fish's own motion, and transiently suppressed tectal spiking activity. High-resolution calcium imaging revealed localized motor-related signals in the tectal neuropil and the upstream torus longitudinalis, suggesting that CD enters the tectum via this pathway. Together, our results show how visual processing is suppressed during self-motion by motor-related phasic inhibition. This may help explain perceptual saccadic suppression observed in many species.

Recent strategies for neural dynamics observation at a larger scale and wider scope

The tremendous technical progress in neuroscience offers opportunities to observe a more minor or/and broader dynamic picture of the brain. Moreover, the large-scale neural activity of individual neurons enables the dissection of detailed mechanistic links between neural populations and behaviors. To measure neural activity in-vivo, multi-neuron recording, and neuroimaging techniques are employed and developed to acquire more neurons. The tools introduced concurrently recorded dozens to hundreds of neurons in the coordinated brain regions and elucidated the neuronal ensembles from a massive population perspective of diverse neurons at cellular resolution. In particular, the increasing spatiotemporal resolution of neuronal monitoring across the whole brain dramatically facilitates our understanding of additional nervous system functions in health and disease. Here, we will introduce state-of-the-art neuroscience tools involving large-scale neural population recording and the long-range connections spanning multiple brain regions. Their synergic effects provide to clarify the controversial circuitry underlying neuroscience. These challenging neural tools present a promising outlook for the fundamental dynamic interplay across levels of synaptic cellular, circuit organization, and brain-wide. Hence, more observations of neural dynamics will provide more clues to elucidate brain functions and push forward innovative technology at the intersection of neural engineering disciplines. We hope this review will provide insight into the use or development of recent neural techniques considering spatiotemporal scales of brain observation.

Spatial correspondence in relative space regulates serial dependence

Our perception is often attracted to what we have seen before, a phenomenon called 'serial dependence.' Serial dependence can help maintain a stable perception of the world, given the statistical regularity in the environment. If serial dependence serves this presumed utility, it should be pronounced when consecutive elements share the same identity when multiple elements spatially shift across successive views. However, such preferential serial dependence between identity-matching elements in dynamic situations has never been empirically tested. Here, we hypothesized that serial dependence between consecutive elements is modulated more effectively by the spatial correspondence in relative space than by that in absolute space because spatial correspondence in relative coordinates can warrant identity matching invariantly to changes in absolute coordinates. To test this hypothesis, we developed a task where two targets change positions in unison between successive views. We found that serial dependence was substantially modulated by the correspondence in relative coordinates, but not by that in absolute coordinates. Moreover, such selective modulation by the correspondence in relative space was also observed even for the serial dependence defined by previous non-target elements. Our findings are consistent with the view that serial dependence subserves object-based perceptual stabilization over time in dynamic situations.

Saccade-Responsive Visual Cortical Neurons Do Not Exhibit Distinct Visual Response Properties

Rapid saccadic eye movements are used by animals to sample different parts of the visual scene. Previous work has investigated neural correlates of these saccades in visual cortical areas such as V1; however, how saccade-responsive neurons are distributed across visual areas, cell types, and cortical layers has remained unknown. Through analyzing 818 1 h experimental sessions from the Allen Brain Observatory, we present a large-scale analysis of saccadic behaviors in head-fixed mice and their neural correlates. We find that saccade-responsive neurons are present across visual cortex, but their distribution varies considerably by transgenically defined cell type, cortical area, and cortical layer. We also find that saccade-responsive neurons do not exhibit distinct visual response properties from the broader neural population, suggesting that the saccadic responses of these neurons are likely not predominantly visually driven. These results provide insight into the roles played by different cell types within a broader, distributed network of sensory and motor interactions.

The perceptual consequences and neurophysiology of eye blinks

A hand passing in front of a camera produces a large and obvious disruption of a video. Yet the closure of the eyelid during a blink, which lasts for hundreds of milliseconds and occurs thousands of times per day, typically goes unnoticed. What are the neural mechanisms that mediate our uninterrupted visual experience despite frequent occlusion of the eyes? Here, we review the existing literature on the neurophysiology, perceptual consequences, and behavioral dynamics of blinks. We begin by detailing the kinematics of the eyelid that define a blink. We next discuss the ways in which blinks alter visual function by occluding the pupil, decreasing visual sensitivity, and moving the eyes. Then, to anchor our understanding, we review the similarities between blinks and other actions that lead to reductions in visual sensitivity, such as saccadic eye movements. The similarity between these two actions has led to suggestions that they share a common neural substrate. We consider the extent of overlap in their neural circuits and go on to explain how recent findings regarding saccade suppression cast doubt on the strong version of the shared mechanism hypothesis. We also evaluate alternative explanations of how blink-related processes modulate neural activity to maintain visual stability: a reverberating corticothalamic loop to maintain information in the face of lid closure; and a suppression of visual transients related to lid closure. Next, we survey the many areas throughout the brain that contribute to the execution of, regulation of, or response to blinks. Regardless of the underlying mechanisms, blinks drastically attenuate our visual abilities, yet these perturbations fail to reach awareness. We conclude by outlining opportunities for future work to better understand how the brain maintains visual perception in the face of eye blinks. Future work will likely benefit from incorporating theories of perceptual stability, neurophysiology, and novel behavior paradigms to address issues central to our understanding of natural visual behavior and for the clinical rehabilitation of active vision.

Interactions between rodent visual and spatial systems during navigation

Many behaviours that are critical for animals to survive and thrive rely on spatial navigation. Spatial navigation, in turn, relies on internal representations about one's spatial location, one's orientation or heading direction and the distance to objects in the environment. Although the importance of vision in guiding such internal representations has long been recognized, emerging evidence suggests that spatial signals can also modulate neural responses in the central visual pathway. Here, we review the bidirectional influences between visual and navigational signals in the rodent brain. Specifically, we discuss reciprocal interactions between vision and the internal representations of spatial position, explore the effects of vision on representations of an animal's heading direction and vice versa, and examine how the visual and navigational systems work together to assess the relative distances of objects and other features. Throughout, we consider how technological advances and novel ethological paradigms that probe rodent visuo-spatial behaviours allow us to advance our understanding of how brain areas of the central visual pathway and the spatial systems interact and enable complex behaviours.

Cortical Integration of Vestibular and Visual Cues for Navigation, Visual Processing, and Perception

Despite increasing evidence of its involvement in several key functions of the cerebral cortex, the vestibular sense rarely enters our consciousness. Indeed, the extent to which these internal signals are incorporated within cortical sensory representation and how they might be relied upon for sensory-driven decision-making, during, for example, spatial navigation, is yet to be understood. Recent novel experimental approaches in rodents have probed both the physiological and behavioral significance of vestibular signals and indicate that their widespread integration with vision improves both the cortical representation and perceptual accuracy of self-motion and orientation. Here, we summarize these recent findings with a focus on cortical circuits involved in visual perception and spatial navigation and highlight the major remaining knowledge gaps. We suggest that vestibulo-visual integration reflects a process of constant updating regarding the status of self-motion, and access to such information by the cortex is used for sensory perception and predictions that may be implemented for rapid, navigation-related decision-making.

Not everything, not everywhere, not all at once: a study of brain-wide encoding of movement

Activity related to movement is found throughout sensory and motor regions of the brain. However, it remains unclear how movement-related activity is distributed across the brain and whether systematic differences exist between brain areas. Here, we analyzed movement related activity in brain-wide recordings containing more than 50,000 neurons in mice performing a decision-making task. Using multiple techniques, from markers to deep neural networks, we find that movement-related signals were pervasive across the brain, but systematically differed across areas. Movement-related activity was stronger in areas closer to the motor or sensory periphery. Delineating activity in terms of sensory- and motor-related components revealed finer scale structures of their encodings within brain areas. We further identified activity modulation that correlates with decision-making and uninstructed movement. Our work charts out a largescale map of movement encoding and provides a roadmap for dissecting different forms of movement and decision-making related encoding across multi-regional neural circuits.

Semantic novelty modulates neural responses to visual change across the human brain

Our continuous visual experience in daily life is dominated by change. Previous research has focused on visual change due to stimulus motion, eye movements or unfolding events, but not their combined impact across the brain, or their interactions with semantic novelty. We investigate the neural responses to these sources of novelty during film viewing. We analyzed intracranial recordings in humans across 6328 electrodes from 23 individuals. Responses associated with saccades and film cuts were dominant across the entire brain. Film cuts at semantic event boundaries were particularly effective in the temporal and medial temporal lobe. Saccades to visual targets with high visual novelty were also associated with strong neural responses. Specific locations in higher-order association areas showed selectivity to either high or low-novelty saccades. We conclude that neural activity associated with film cuts and eye movements is widespread across the brain and is modulated by semantic novelty.

Vision: Neuronal mechanisms enabling stable perception

Eye movements cause rapid motion of the retinal image, potentially confusable with external motion. A recent study shows that neurons in mouse primary visual cortex distinguish self-generated from external motion by combining sensory input with saccade-related signals from the thalamic pulvinar nucleus.