Corollary Discharge and Oculomotor Proprioception: Cortical Mechanisms for Spatially Accurate Vision

Brain representations of motion and position in the double-drift illusion

In the 'double-drift' illusion, local motion within a window moving in the periphery of the visual field alters the window's perceived path. The illusion is strong even when the eyes track a target whose motion matches the window so that the stimulus remains stable on the retina. This implies that the illusion involves the integration of retinal signals with non-retinal eye-movement signals. To identify where in the brain this integration occurs, we measured BOLD fMRI responses in visual cortex while subjects experienced the double-drift illusion. We then used a combination of univariate and multivariate decoding analyses to identify (1) which brain areas were sensitive to the illusion and (2) whether these brain areas contained information about the illusory stimulus trajectory. We identified a number of cortical areas that responded more strongly during the illusion than a control condition that was matched for low-level stimulus properties. Only in area hMT+ was it possible to decode the illusory trajectory. We additionally performed a number of important controls that rule out possible low-level confounds. Concurrent eye tracking confirmed that subjects accurately tracked the moving target; we were unable to decode the illusion trajectory using eye position measurements recorded during fMRI scanning, ruling out explanations based on differences in oculomotor behavior. Our results provide evidence for a perceptual representation in human visual cortex that incorporates extraretinal information.

Primacy of vision shapes behavioral strategies and neural substrates of spatial navigation in marmoset hippocampus

The role of the hippocampus in spatial navigation has been primarily studied in nocturnal mammals, such as rats, that lack many adaptations for daylight vision. Here we demonstrate that during 3D navigation, the common marmoset, a new world primate adapted to daylight, predominantly uses rapid head-gaze shifts for visual exploration while remaining stationary. During active locomotion marmosets stabilize the head, in contrast to rats that use low-velocity head movements to scan the environment as they locomote. Pyramidal neurons in the marmoset hippocampus CA3/CA1 regions predominantly show mixed selectivity for 3D spatial view, head direction, and place. Exclusive place selectivity is scarce. Inhibitory interneurons are predominantly mixed selective for angular head velocity and translation speed. Finally, we found theta phase resetting of local field potential oscillations triggered by head-gaze shifts. Our findings indicate that marmosets adapted to their daylight ecological niche by modifying exploration/navigation strategies and their corresponding hippocampal specializations.

Oculomotor routines for perceptual judgments

In everyday life we frequently make simple visual judgments about object properties, for example, how big or wide is a certain object? Our goal is to test whether there are also task-specific oculomotor routines that support perceptual judgments, similar to the well-established exploratory routines for haptic perception. In a first study, observers saw different scenes with two objects presented in a photorealistic virtual reality environment. Observers were asked to judge which of two objects was taller or wider while gaze was tracked. All tasks were performed with the same set of virtual objects in the same scenes, so that we can compare spatial characteristics of exploratory gaze behavior to quantify oculomotor routines for each task. Width judgments showed fixations around the center of the objects with larger horizontal spread. In contrast, for height judgments, gaze was shifted toward the top of the objects with larger vertical spread. These results suggest specific strategies in gaze behavior that presumably are used for perceptual judgments. To test the causal link between oculomotor behavior and perception, in a second study, observers could freely gaze at the object or we introduced a gaze-contingent setup forcing observers to fixate specific positions on the object. Discrimination performance was similar between free-gaze and the gaze-contingent conditions for width and height judgments. These results suggest that although gaze is adapted for different tasks, performance seems to be based on a perceptual strategy, independent of potential cues that can be provided by the oculomotor system.

Saccade execution increases the preview effect with faces: An EEG and eye-tracking coregistration study

Under naturalistic viewing conditions, humans conduct about three to four saccadic eye movements per second. These dynamics imply that in real life, humans rarely see something completely new; there is usually a preview of the upcoming foveal input from extrafoveal regions of the visual field. In line with results from the field of reading research, we have shown with EEG and eye-tracking coregistration that an extrafoveal preview also affects postsaccadic visual object processing and facilitates discrimination. Here, we ask whether this preview effect in the fixation-locked N170, and in manual responses to the postsaccadic target face (tilt discrimination), requires saccade execution. Participants performed a gaze-contingent experiment in which extrafoveal face images could change their orientation during a saccade directed to them. In a control block, participants maintained stable gaze throughout the experiment and the extrafoveal face reappeared foveally after a simulated saccade latency. Compared with this no-saccade condition, the neural and the behavioral preview effects were much larger in the saccade condition. We also found shorter first fixation durations after an invalid preview, which is in contrast to reading studies. We interpret the increased preview effect under saccade execution as the result of the additional sensorimotor processes that come with gaze behavior compared with visual perception under stable fixation. In addition, our findings call into question whether EEG studies with fixed gaze capture key properties and dynamics of active, natural vision.

Mice require proprioception to establish long-term visuospatial memory

Because the retina moves constantly, the retinotopic representation of the visual world is spatially inaccurate and the brain must transform this spatially inaccurate retinal signal to a spatially accurate signal usable for perception and action. One of the salient discoveries of modern neuroscience is the role of the hippocampus in establishing gaze-independent, long-term visuospatial memories. The rat hippocampus has neurons which report the animal's position in space regardless of its angle of gaze. Rats with hippocampal lesions are unable to find the location of an escape platform hidden in a pool of opaque fluid, the Morris Water Maze (MWM) based on the visual aspects of their surrounding environment. Here we show that the representation of proprioception in the dysgranular zone of primary somatosensory cortex is equivalently necessary for mice to learn the location of the hidden platform, presumably because without it they cannot create a long-term gaze-independent visuospatial representation of their environment from the retinal signal. They have no trouble finding the platform when it is marked by a flag, and they have no motor or vestibular deficits.

Dynamic saccade context triggers more stable object-location binding

Our visual systems rapidly perceive and integrate information about object identities and locations. There is long-standing debate about how we achieve world-centered (spatiotopic) object representations across eye movements, with many studies reporting persistent retinotopic (eye-centered) effects even for higher-level object-location binding. But these studies are generally conducted in fairly static experimental contexts. Might spatiotopic object-location binding only emerge in more dynamic saccade contexts? In the present study, we investigated this using the Spatial Congruency Bias paradigm in healthy adults. In the static (single saccade) context, we found purely retinotopic binding, as before. However, robust spatiotopic binding emerged in the dynamic (multiple frequent saccades) context. We further isolated specific factors that modulate retinotopic and spatiotopic binding. Our results provide strong evidence that dynamic saccade context can trigger more stable object-location binding in ecologically-relevant spatiotopic coordinates, perhaps via a more flexible brain state which accommodates improved visual stability in the dynamic world.

Cortical maps as a fundamental neural substrate for visual representation

Visual perception is the product of serial hierarchical processing, parallel processing, and remapping on a dynamic network involving several topographically organized cortical visual areas. Here, we will focus on the topographical organization of cortical areas and the different kinds of visual maps found in the primate brain. We will interpret our findings in light of a broader representational framework for perception. Based on neurophysiological data, our results do not support the notion that vision can be explained by a strict representational model, where the objective visual world is faithfully represented in our brain. On the contrary, we find strong evidence that vision is an active and constructive process from the very initial stages taking place in the eye and from the very initial stages of our development. A constructive interplay between perceptual and motor systems (e.g., during saccadic eye movements) is actively learnt from early infancy and ultimately provides our fluid stable visual perception of the world.

Inferring visual space from ultra-fine extra-retinal knowledge of gaze position

It has long been debated how humans resolve fine details and perceive a stable visual world despite the incessant fixational motion of their eyes. Current theories assume these processes to rely solely on the visual input to the retina, without contributions from motor and/or proprioceptive sources. Here we show that contrary to this widespread assumption, the visual system has access to high-resolution extra-retinal knowledge of fixational eye motion and uses it to deduce spatial relations. Building on recent advances in gaze-contingent display control, we created a spatial discrimination task in which the stimulus configuration was entirely determined by oculomotor activity. Our results show that humans correctly infer geometrical relations in the absence of spatial information on the retina and accurately combine high-resolution extraretinal monitoring of gaze displacement with retinal signals. These findings reveal a sensory-motor strategy for encoding space, in which fine oculomotor knowledge is used to interpret the fixational input to the retina.

SPORTS REHABILITATION IN ATHLETES WITH MENISCAL LESIONS BASED ON ELECTROACUPUNCTURE ASSOCIATED WITH SPORTS THERAPY

ABSTRACT Introduction: Meniscal injury is a common condition that can lead to disability due to pain and proprioceptive failure, requiring immediate attention. Combination therapies involve advanced approaches aiming to accelerate rehabilitation in athletes, and electroacupuncture presents therapeutic benefits, although there is still no evidence of its combination with sports therapy. Objective: This paper analyzes the performance of sports rehabilitation in athletes with meniscal lesions using electroacupuncture combined with sports therapy. Methods: The intervention in the control group was based on a traditional range of motion work, muscle strength, proprioceptive training, and other exercise therapies, while the experimental group received a 30 min electro-acupuncture protocol three times a week for four consecutive weeks. The surrogate data (gender, age, disease course, location) are the same. Before treatment, joint activity, muscle strength, total joint scale score of the LYSHOLM questionnaire, and other observational indices were measured during the 6th and 12th week of treatment. The non-parametric statistical method and T-test were used to analyze the changes of each index before and after treatment. After 12 weeks of treatment, the difference between the experimental group and the combination before treatment was significant. Results: The treatment effect of the experimental group was significantly better than the control group. Conclusion: The effect of sports rehabilitation of athletes with meniscus injury based on electroacupuncture combined with sports therapy showed high resolutive application value, indicating an alternative for non-surgical treatment in knee meniscus injuries. Level of evidence II; Therapeutic studies - investigation of treatment outcomes.

Sleep, plasticity, and sensory neurodevelopment

A defining feature of early infancy is the immense neural plasticity that enables animals to develop a brain that is functionally integrated with a growing body. Early infancy is also defined as a period dominated by sleep. Here, we describe three conceptual frameworks that vary in terms of whether and how they incorporate sleep as a factor in the activity-dependent development of sensory and sensorimotor systems. The most widely accepted framework is exemplified by the visual system where retinal waves seemingly occur independent of sleep-wake states. An alternative framework is exemplified by the sensorimotor system where sensory feedback from sleep-specific movements activates the brain. We prefer a third framework that encompasses the first two but also captures the diverse ways in which sleep modulates activity-dependent development throughout the nervous system. Appreciation of the third framework will spur progress toward a more comprehensive and cohesive understanding of both typical and atypical neurodevelopment.

Spatiotopic and retinotopic memory in the context of natural images

Neural responses throughout the visual cortex encode stimulus location in a retinotopic (i.e., eye-centered) reference frame, and memory for stimulus position is most precise in retinal coordinates. Yet visual perception is spatiotopic: objects are perceived as stationary, even though eye movements cause frequent displacement of their location on the retina. Previous studies found that, after a single saccade, memory of retinotopic locations is more accurate than memory of spatiotopic locations. However, it is not known whether various aspects of natural viewing affect the retinotopic reference frame advantage. We found that the retinotopic advantage may in part depend on a retinal afterimage, which can be effectively nullified through backwards masking. Moreover, in the presence of natural scenes, spatiotopic memory is more accurate than retinotopic memory, but only when subjects are provided sufficient time to process the scene before the eye movement. Our results demonstrate that retinotopic memory is not always more accurate than spatiotopic memory and that the fidelity of memory traces in both reference frames are sensitive to the presence of contextual cues.

Cortical neural dynamics unveil the rhythm of natural visual behavior in marmosets

Numerous studies have shown that the visual system consists of functionally distinct ventral and dorsal streams; however, its exact spatial-temporal dynamics during natural visual behavior remain to be investigated. Here, we report cerebral neural dynamics during active visual exploration recorded by an electrocorticographic array covering the entire lateral surface of the marmoset cortex. We found that the dorsal stream was activated before the primary visual cortex with saccades and followed by the alteration of suppression and activation signals along the ventral stream. Similarly, the signal that propagated from the dorsal to ventral visual areas was accompanied by a travelling wave of low frequency oscillations. Such signal dynamics occurred at an average of 220 ms after saccades, which corresponded to the timing when whole-brain activation returned to background levels. We also demonstrated that saccades could occur at any point of signal flow, indicating the parallel computation of motor commands. Overall, this study reveals the neural dynamics of active vision, which are efficiently linked to the natural rhythms of visual exploration.

The macaque ventral intraparietal area has expanded into three homologue human parietal areas

The macaque ventral intraparietal area (VIP) in the fundus of the intraparietal sulcus has been implicated in a diverse range of sensorimotor and cognitive functions such as motion processing, multisensory integration, processing of head peripersonal space, defensive behavior, and numerosity coding. Here, we exhaustively review macaque VIP function, cytoarchitectonics, and anatomical connectivity and integrate it with human studies that have attempted to identify a potential human VIP homologue. We show that human VIP research has consistently identified three, rather than one, bilateral parietal areas that each appear to subsume some, but not all, of the macaque area's functionality. Available evidence suggests that this human "VIP complex" has evolved as an expansion of the macaque area, but that some precursory specialization within macaque VIP has been previously overlooked. The three human areas are dominated, roughly, by coding the head or self in the environment, visual heading direction, and the peripersonal environment around the head, respectively. A unifying functional principle may be best described as prediction in space and time, linking VIP to state estimation as a key parietal sensorimotor function. VIP's expansive differentiation of head and self-related processing may have been key in the emergence of human bodily self-consciousness.

Mixed selectivity coding of sensory and motor social signals in the thalamus of a weakly electric fish

High-level neural activity often exhibits mixed selectivity to multivariate signals. How such representations arise and modulate natural behavior is poorly understood. We addressed this question in weakly electric fish, whose social behavior is relatively low dimensional and can be easily reproduced in the laboratory. We report that the preglomerular complex, a thalamic region exclusively connecting midbrain with pallium, implements a mixed selectivity strategy to encode interactions related to courtship and rivalry. We discuss how this code enables the pallial recurrent networks to control social behavior, including dominance in male-male competition and female mate selection. Notably, response latency analysis and computational modeling suggest that corollary discharge from premotor regions is implicated in flagging outgoing communications and thereby disambiguating self- versus non-self-generated signals. These findings provide new insights into the neural substrates of social behavior, multi-dimensional neural representation, and its role in perception and decision making.

Unreferenced spatial localization under monocular and dichoptic viewing conditions

Knowledge of eye position in the brain is critical for localization of objects in space. To investigate the accuracy and precision of eye position feedback in an unreferenced environment, subjects with normal ocular alignment attempted to localize briefly presented targets during monocular and dichoptic viewing. In the task, subjects' used a computer mouse to position a response disk at the remembered location of the target. Under dichoptic viewing (with red (right eye)-green (left eye) glasses), target and response disks were presented to the same or alternate eyes, leading to four conditions [green target-green response cue (LL), green-red (LR), red-green (RL), and red-red (RR)]. Time interval between target and response disks was varied and localization errors were the difference between the estimated and real positions of the target disk. Overall, the precision of spatial localization (variance across trials) became progressively worse with time. Under dichoptic viewing, localization errors were significantly greater for alternate-eye trials as compared to same-eye trials and were correlated to the average phoria of each subject. Our data suggests that during binocular dissociation, spatial localization may be achieved by combining a reliable versional efference copy signal with a proprioceptive signal that is unreliable perhaps because it is from the wrong eye or is too noisy.

Recent understanding of binocular vision in the natural environment with clinical implications

Technological advances in recent decades have allowed us to measure both the information available to the visual system in the natural environment and the rich array of behaviors that the visual system supports. This review highlights the tasks undertaken by the binocular visual system in particular and how, for much of human activity, these tasks differ from those considered when an observer fixates a static target on the midline. The everyday motor and perceptual challenges involved in generating a stable, useful binocular percept of the environment are discussed, together with how these challenges are but minimally addressed by much of current clinical interpretation of binocular function. The implications for new technology, such as virtual reality, are also highlighted in terms of clinical and basic research application.

Interocular suppression in primary visual cortex in strabismus: impact of staggering the presentation of stimuli to the eyes

Diplopia (double vision) in strabismus is prevented by suppression of the image emanating from one eye. In a recent study conducted in two macaques raised with exotropia (an outward ocular deviation) but having normal acuity in each eye, simultaneous display of stimuli to each eye did not induce suppression in V1 neurons. Puzzled by this negative result, we have modified our protocol to display stimuli in a staggered sequence, rather than simultaneously. Additional recordings were made in the same two macaques, following two paradigms. In trial type 1, the receptive field in one eye was stimulated with a sine-wave grating while the other eye was occluded. After 5 s, the occluder was removed and the neuron was stimulated for another 5 s. The effect of uncovering the eye, which potentially exposed the animal to diplopia, was quantified by the peripheral retinal interaction index (PRII). In trial type 2, the receptive field in the fixating eye was stimulated with a grating during binocular viewing. After 5 s, a second grating appeared in the receptive field of the nonfixating eye. The impact of the second grating, which had the potential to generate visual confusion, was quantified by the receptive field interaction index (RFII). For 82 units, the mean PRII was 0.48 ± 0.05 (0.50 = no suppression) and the mean RFII was 0.46 ± 0.08 (0.50 = no suppression). These values suggest mild suppression, but the modest decline in spike rate registered during the second epoch of visual stimulation might have been due to neuronal adaptation, rather than interocular suppression. In a few instances neurons showed unequivocal suppression, but overall, these recordings did not support the contention that staggered stimulus presentation is more effective than simultaneous stimulus presentation at evoking interocular suppression in V1 neurons.NEW & NOTEWORTHY In strabismus, double vision is prevented by interocular suppression. It has been reported that inhibition of neuronal firing in the primary visual cortex occurs only when stimuli are presented sequentially, rather than simultaneously. However, these recordings in alert macaques raised with exotropia showed, with rare exceptions, little evidence to support the concept that staggered stimulus presentation is more effective at inducing interocular suppression of V1 neurons.

A change in perspective: The interaction of saccadic and pursuit eye movements in oculomotor control and perception

Due to the close relationship between oculomotor behavior and visual processing, eye movements have been studied in many different areas of research over the last few decades. While these studies have brought interesting insights, specialization within each research area comes at the potential cost of a narrow and isolated view of the oculomotor system. In this review, we want to expand this perspective by looking at the interactions between the two most important types of voluntary eye movements: saccades and pursuit. Recent evidence indicates multiple interactions and shared signals at the behavioral and neurophysiological level for oculomotor control and for visual perception during pursuit and saccades. Oculomotor control seems to be based on shared position- and velocity-related information, which leads to multiple behavioral interactions and synergies. The distinction between position- and velocity-related information seems to be also present at the neurophysiological level. In addition, visual perception seems to be based on shared efferent signals about upcoming eye positions and velocities, which are to some degree independent of the actual oculomotor response. This review suggests an interactive perspective on the oculomotor system, based mainly on different types of sensory input, and less so on separate subsystems for saccadic or pursuit eye movements.

Oculomotor corollary discharge signaling is related to repetitive behavior in children with autism spectrum disorder

Corollary discharge (CD) signals are "copies" of motor signals sent to sensory regions that allow animals to adjust sensory consequences of self-generated actions. Autism spectrum disorder (ASD) is characterized by sensory and motor deficits, which may be underpinned by altered CD signaling. We evaluated oculomotor CD using the blanking task, which measures the influence of saccades on visual perception, in 30 children with ASD and 35 typically developing (TD) children. Participants were instructed to make a saccade to a visual target. Upon saccade initiation, the presaccadic target disappeared and reappeared to the left or right of the original position. Participants indicated the direction of the jump. With intact CD, participants can make accurate perceptual judgements. Otherwise, participants may use saccade landing site as a proxy of the presaccadic target and use it to inform perception. We used multilevel modeling to examine the influence of saccade landing site on trans-saccadic perceptual judgements. We found that, compared with TD participants, children with ASD were more sensitive to target displacement and less reliant on saccade landing site when spatial uncertainty of the post-saccadic target was high. This pattern was driven by ASD participants with less severe restricted and repetitive behaviors. These results suggest a relationship between altered CD signaling and core ASD symptoms.

Visual Remapping

Our visual system is fundamentally retinotopic. When viewing a stable scene, each eye movement shifts object features and locations on the retina. Thus, sensory representations must be updated, or remapped, across saccades to align presaccadic and postsaccadic inputs. The earliest remapping studies focused on anticipatory, presaccadic shifts of neuronal spatial receptive fields. Over time, it has become clear that there are multiple forms of remapping and that different forms of remapping may be mediated by different neural mechanisms. This review attempts to organize the various forms of remapping into a functional taxonomy based on experimental data and ongoing debates about forward versus convergent remapping, presaccadic versus postsaccadic remapping, and spatial versus attentional remapping. We integrate findings from primate neurophysiological, human neuroimaging and behavioral, and computational modeling studies. We conclude by discussing persistent open questions related to remapping, with specific attention to binding of spatial and featural information during remapping and speculations about remapping's functional significance. Expected final online publication date for the Annual Review of Vision Science, Volume 7 is September 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

The extrafoveal preview paradigm as a measure of predictive, active sampling in visual perception

A key feature of visual processing in humans is the use of saccadic eye movements to look around the environment. Saccades are typically used to bring relevant information, which is glimpsed with extrafoveal vision, into the high-resolution fovea for further processing. With the exception of some unusual circumstances, such as the first fixation when walking into a room, our saccades are mainly guided based on this extrafoveal preview. In contrast, the majority of experimental studies in vision science have investigated "passive" behavioral and neural responses to suddenly appearing and often temporally or spatially unpredictable stimuli. As reviewed here, a growing number of studies have investigated visual processing of objects under more natural viewing conditions in which observers move their eyes to a stationary stimulus, visible previously in extrafoveal vision, during each trial. These studies demonstrate that the extrafoveal preview has a profound influence on visual processing of objects, both for behavior and neural activity. Starting from the preview effect in reading research we follow subsequent developments in vision research more generally and finally argue that taking such evidence seriously leads to a reconceptualization of the nature of human visual perception that incorporates the strong influence of prediction and action on sensory processing. We review theoretical perspectives on visual perception under naturalistic viewing conditions, including theories of active vision, active sensing, and sampling. Although the extrafoveal preview paradigm has already provided useful information about the timing of, and potential mechanisms for, the close interaction of the oculomotor and visual systems while reading and in natural scenes, the findings thus far also raise many new questions for future research.

Interocular Suppression in Primary Visual Cortex in Strabismus

People with strabismus acquired during childhood do not experience diplopia (double vision). To investigate how perception of the duplicate image is suppressed, we raised two male monkeys with alternating exotropia by disinserting the medial rectus muscle in each eye at age four weeks. Once the animals were mature, they were brought to the laboratory and trained to fixate a small spot while recordings were made in primary visual cortex (V1). Drifting gratings were presented to the receptive fields of 500 single neurons for eight interleaved conditions: (1) right eye monocular; (2) left eye monocular; (3) right eye's field, right eye fixating; (4) right eye's field, left eye fixating; (5) left eye's field, right eye fixating; (6) left eye's field, left eye fixating; (7) both eyes' fields, right eye fixating; (8) both eyes' fields, left eye fixating. As expected, ocular dominance histograms showed a monocular bias compared with normal animals, but many cells could still be driven via both eyes. Overall, neuronal responses were not affected by switches in ocular fixation. Individual neurons exhibited binocular interactions, but mean population indices indicated no net interocular suppression or facilitation. Even neurons located in cortex with reduced cytochrome oxidase (CO) activity, representing portions of the nasal visual field where perception is suppressed during binocular viewing, showed no net inhibition. These data indicate that V1 neurons do not appear to reflect strabismic suppression and therefore the elimination of diplopia is likely to be mediated at a higher cortical level.SIGNIFICANCE STATEMENT In patients with strabismus, images fall on non-corresponding points in the two retinas. Only one image is perceived, because signals emanating from the other eye that convey the duplicate image are suppressed. The benefit is that diplopia is prevented, but the penalty is that the visual feedback required to adjust eye muscle tone to realign the globes is eliminated. Here, we report the first electrophysiological recordings from the primary visual cortex (V1) in awake monkeys raised with strabismus. The experiments were designed to reveal how perception of double images is avoided.

Spatiotemporal Content of Saccade Transients

Humans use rapid gaze shifts, known as saccades, to explore visual scenes. These movements yield abrupt luminance changes on the retina, which elicit robust neural discharges at fixation onsets. Yet little is known about the spatial content of saccade transients. Here, we show that saccades redistribute spatial information within the temporal range of retinal sensitivity following two distinct regimes: saccade modulations counterbalance (whiten) the spectral density of natural scenes at low spatial frequencies and follow the external power distribution at higher frequencies. This redistribution is a consequence of saccade dynamics, particularly the speed/amplitude/duration relation known as the main sequence. It resembles the redistribution resulting from inter-saccadic eye drifts, revealing a continuum in the modulations given by different eye movements, with oculomotor transitions primarily acting by regulating the bandwidth of whitening. Our findings suggest important computational roles for saccade transients in the establishment of spatial representations and lead to testable predictions about their consequences for visual functions and encoding mechanisms.

Perisaccadic remapping: What? How? Why?

About 25 years ago, the discovery of receptive field (RF) remapping in the parietal cortex of nonhuman primates revealed that visual RFs, widely assumed to have a fixed retinotopic organization, can change position before every saccade. Measuring such changes can be deceptively difficult. As a result, studies that followed have generated a fascinating but somewhat confusing picture of the phenomenon. In this review, we describe how observations of RF remapping depend on the spatial and temporal sampling of visual RFs and saccade directions. Further, we summarize some of the theories of how remapping might occur in neural circuitry. Finally, based on neurophysiological and psychophysical observations, we discuss the ways in which remapping information might facilitate computations in downstream brain areas.

The Contribution of Oculomotor Functions to Rates of Visual Information Processing in Younger and Older Adults

Oculomotor functions are established surrogate measures of visual attention shifting and rate of information processing, however, the temporal characteristics of saccades and fixations have seldom been compared in healthy educated samples of younger and older adults. Thus, the current study aimed to compare duration of eye movement components in younger (18-25 years) and older (50-81 years) adults during text reading and during object/alphanumeric Rapid Automatic Naming (RAN) tasks. The current study also aimed to examine the contribution of oculomotor functions to threshold time needed for accurate performance on visually-driven cognitive tasks (Inspection Time [IT] and Change Detection [CD]). Results showed that younger adults fixated on individual stimuli for significantly longer than the older participants, while older adults demonstrated significantly longer saccade durations than the younger group. Results also demonstrated that older adults required longer threshold durations (i.e., performed slower) on the visually-driven cognitive tasks, however, the age-group time difference on the CD task was eradicated when the effects of saccade duration were covaried. Thus, these results suggest that age-related cognitive decline is also related to increased duration of saccades and hence, highlights the need to dissociate the age-related motor constraints on the temporal aspects of oculomotor function from visuo-cognitive speed of processing.

Intra-saccadic displacement sensitivity after a lesion to the posterior parietal cortex

Visual perception is introspectively stable and continuous across eye movements. It has been hypothesized that displacements in retinal input caused by eye movements can be dissociated from displacements in the external world using extra-retinal information, such as a corollary discharge from the oculomotor system. The extra-retinal information can inform the visual system about an upcoming eye movement and accompanying displacements in retinal input. The parietal cortex has been hypothesized to be critically involved in integrating retinal and extra-retinal information. Two tasks have been widely used to assess the quality of this integration: double-step saccades and intra-saccadic displacements. Double-step saccades performed by patients with parietal cortex lesions seemed to show hypometric second saccades. However, recently idea has been refuted by demonstrating that patients with very similar lesions were able to perform the double step saccades, albeit taking multiple saccades to reach the saccade target. So, it seems that extra-retinal information is still available for saccade execution after a lesion to the parietal lobe. Here, we investigated whether extra-retinal signals are also available for perceptual judgements in nine patients with strokes affecting the posterior parietal cortex. We assessed perceptual continuity with the intra-saccadic displacement task. We exploited the increased sensitivity when a small temporal blank is introduced after saccade offset (blank effect). The blank effect is thought to reflect the availability of extra-retinal signals for perceptual judgements. Although patients exhibited a relative difference to control subjects, they still demonstrated the blank effect. The data suggest that a lesion to the posterior parietal cortex (PPC) alters the processing of extra-retinal signals but does not abolish their influence altogether.

Memory for retinotopic locations is more accurate than memory for spatiotopic locations, even for visually guided reaching

To interact successfully with objects, we must maintain stable representations of their locations in the world. However, their images on the retina may be displaced several times per second by large, rapid eye movements. A number of studies have demonstrated that visual processing is heavily influenced by gaze-centered (retinotopic) information, including a recent finding that memory for an object's location is more accurate and precise in gaze-centered (retinotopic) than world-centered (spatiotopic) coordinates (Golomb & Kanwisher, 2012b). This effect is somewhat surprising, given our intuition that behavior is successfully guided by spatiotopic representations. In the present experiment, we asked whether the visual system may rely on a more spatiotopic memory store depending on the mode of responding. Specifically, we tested whether reaching toward and tapping directly on an object's location could improve memory for its spatiotopic location. Participants performed a spatial working memory task under four conditions: retinotopic vs. spatiotopic task, and computer mouse click vs. touchscreen reaching response. When participants responded by clicking with a mouse on the screen, we replicated Golomb & Kanwisher's original results, finding that memory was more accurate in retinotopic than spatiotopic coordinates and that the accuracy of spatiotopic memory deteriorated substantially more than retinotopic memory with additional eye movements during the memory delay. Critically, we found the same pattern of results when participants responded by using their finger to reach and tap the remembered location on the monitor. These results further support the hypothesis that spatial memory is natively retinotopic; we found no evidence that engaging the motor system improves spatiotopic memory across saccades.

Corollary Discharge Contributions to Perceptual Continuity Across Saccades

Our vision depends upon shifting our high-resolution fovea to objects of interest in the visual field. Each saccade displaces the image on the retina, which should produce a chaotic scene with jerks occurring several times per second. It does not. This review examines how an internal signal in the primate brain (a corollary discharge) contributes to visual continuity across saccades. The article begins with a review of evidence for a corollary discharge in the monkey and evidence from inactivation experiments that it contributes to perception. The next section examines a specific neuronal mechanism for visual continuity, based on corollary discharge that is referred to as visual remapping. Both the basic characteristics of this anticipatory remapping and the factors that control it are enumerated. The last section considers hypotheses relating remapping to the perceived visual continuity across saccades, including remapping's contribution to perceived visual stability across saccades.

Passive eye movements induced by electromagnetic force (EMF) in rats

Accurate information on eye position in the orbit is available from visual feedback, efference copy of the oculomotor commands and proprioceptive signals from the extraocular muscles (EOM). Whereas visual feedback and oculomotor commands have been extensively studied, central processing of EOM proprioceptive signals remains to be elucidated. A challenge to the field is to develop an approach to induce passive eye movements without physically contacting the eyes. A novel method was developed to generate passive eye movements in rats. A small rare-earth magnet disk (0.7 mm diameter, 0.5 mm thickness) was attached to the surface of a rat's eyeball. A metal rod (5 mm diameter) wrapped with an electromagnetic (EM) coil was placed near the magnet (8-15 mm). By passing currents to the EM coil, electromagnetic force (EMF) was generated and acted upon the magnet and induced passive eye movements. The EMF induced well-defined passive eye movements, whose directions were dependent on current polarity and amplitudes and peak velocities were dependent on current intensity and duration. Peak velocities of the EMF-induced eye movements were linearly related to amplitudes, exhibiting main sequence relationships similar to that of saccades in awake rats and eye movements induced by electrical microstimulation of the abducens nucleus in anesthetized rats. Histological examination showed that repetitive EMF stimulations did not appear to result in damages in the EOM fibers. These results validated the EMF approach as a novel tool to investigate EOM proprioceptive signals and their roles in visual localization and gaze control.

Remapping locations and features across saccades: a dual-spotlight theory of attentional updating

How do we maintain visual stability across eye movements? Much work has focused on how visual information is rapidly updated to maintain spatiotopic representations. However, predictive spatial remapping is only part of the story. Here I review key findings, recent debates, and open questions regarding remapping and its implications for visual attention and perception. This review focuses on two key questions: when does remapping occur, and what is the impact on feature perception? Findings are reviewed within the framework of a two-stage, or dual- spotlight, remapping process, where spatial attention must be both updated to the new location (fast, predictive stage) and withdrawn from the previous retinotopic location (slow, post-saccadic stage), with a particular focus on the link between spatial and feature information across eye movements.

The influence of retinal image motion on the perceptual grouping of temporally asynchronous stimuli

Briefly presented stimuli can reveal the lower limit of retinal-based perceptual stabilization mechanisms. This is demonstrated in perceptual grouping of temporally asynchronous stimuli, in which alternate row or column elements of a regular grid are presented over two successive display frames with an imperceptible temporal offset. The grouping phenomenon results from a subtle shift between alternate grid elements due to incomplete compensation of small, fixational eye movements occurring between the two presentation frames. This suggests that larger retinal shifts should amplify the introduced shifts between alternate grid elements and improve grouping performance. However, large shifts are necessarily absent in small eye movements. Furthermore, shifts follow a random walk, making the relationship between shift magnitude and performance difficult to explore systematically. Here, we established a systematic relationship between retinal image motion and perceptual grouping by presenting alternate grid elements (untracked) during smooth pursuit of known velocities. Our results show grouping performance to improve in direct proportion to pursuit velocity. Any potential compensation by extraretinal signals (e.g., efference copy) does not seem to occur.

Motor Impairment and Developmental Psychotic Risk: Connecting the Dots and Narrowing the Pathophysiological Gap

The motor system in its manifold articulations is receiving increasing clinical and research attention. This is because motor impairments constitute a central, expressive component of the mental state examination and a key transdiagnostic feature indexing disease severity. Furthermore, within the schizophrenia spectrum, the integration of neurophysiological, developmental, and phenomenological perspectives suggests that motor impairment is not simply a generic, extrinsic proxy of an altered neurodevelopment, but might be more intimately related to psychotic risk. Therefore, an increased understanding, conceptualization, and knowledge of such motor system and its anomalies could empower contemporary risk prediction and diagnostic procedures.

Eye movements during path integration

Self-motion induces spontaneous eye movements which serve the purpose of stabilizing the visual image on the retina. Previous studies have mainly focused on their reflexive nature and how the perceptual system disentangles visual flow components caused by eye movements and self-motion. Here, we investigated the role of eye movements in distance reproduction (path integration). We used bimodal (visual-auditory)-simulated self-motion: visual optic flow was paired with an auditory stimulus whose frequency was scaled with simulated speed. The task of the subjects in each trial was, first, to observe the simulated self-motion over a certain distance (Encoding phase) and, second, to actively reproduce the observed distance using only visual, only auditory, or bimodal feedback (Reproduction phase). We found that eye positions and eye speeds were strongly correlated between the Encoding and the Reproduction phases. This was the case even when reproduction relied solely on auditory information and thus no visual stimulus was presented. We believe that these correlations are indicative of a contribution of eye movements to path integration.

Vision During Saccadic Eye Movements

The perceptual consequences of eye movements are manifold: Each large saccade is accompanied by a drop of sensitivity to luminance-contrast, low-frequency stimuli, impacting both conscious vision and involuntary responses, including pupillary constrictions. They also produce transient distortions of space, time, and number, which cannot be attributed to the mere motion on the retinae. All these are signs that the visual system evokes active processes to predict and counteract the consequences of saccades. We propose that a key mechanism is the reorganization of spatiotemporal visual fields, which transiently increases the temporal and spatial uncertainty of visual representations just before and during saccades. On one hand, this accounts for the spatiotemporal distortions of visual perception; on the other hand, it implements a mechanism for fusing pre- and postsaccadic stimuli. This, together with the active suppression of motion signals, ensures the stability and continuity of our visual experience.

How the Brain's Navigation System Shapes Our Visual Experience

We explore the environment not only by navigating, but also by viewing our surroundings with our eyes. Here we review growing evidence that the mammalian hippocampal formation, extensively studied in the context of navigation and memory, mediates a representation of visual space that is stably anchored to the external world. This visual representation puts the hippocampal formation in a central position to guide viewing behavior and to modulate visual processing beyond the medial temporal lobe (MTL). We suggest that vision and navigation share several key computational challenges that are solved by overlapping and potentially common neural systems, making vision an optimal domain to explore whether and how the MTL supports cognitive operations beyond navigation.

Space and time in the brain

Nothing is more intuitive, yet more complex, than the concepts of space and time. In contrast to spacetime in physics, space and time in neuroscience remain separate coordinates to which we attach our observations. Investigators of navigation and memory relate neuronal activity to position, distance, time point, and duration and compare these parameters to units of measuring instruments. Although spatial-temporal sequences of brain activity often correlate with distance and duration measures, these correlations may not correspond to neuronal representations of space or time. Neither instruments nor brains sense space or time. Neuronal activity can be described as a succession of events without resorting to the concepts of space or time. Instead of searching for brain representations of our preconceived ideas, we suggest investigating how brain mechanisms give rise to inferential, model-building explanations.

Execution of saccadic eye movements affects speed perception

Due to the foveal organization of our visual system we have to constantly move our eyes to gain precise information about our environment. Doing so massively alters the retinal input. This is problematic for the perception of moving objects, because physical motion and retinal motion become decoupled and the brain has to discount the eye movements to recover the speed of moving objects. Two different types of eye movements, pursuit and saccades, are combined for tracking. We investigated how the way we track moving targets can affect the perceived target speed. We found that the execution of corrective saccades during pursuit initiation modifies how fast the target is perceived compared with pure pursuit. When participants executed a forward (catch-up) saccade they perceived the target to be moving faster. When they executed a backward saccade they perceived the target to be moving more slowly. Variations in pursuit velocity without corrective saccades did not affect perceptual judgments. We present a model for these effects, assuming that the eye velocity signal for small corrective saccades gets integrated with the retinal velocity signal during pursuit. In our model, the execution of corrective saccades modulates the integration of these two signals by giving less weight to the retinal information around the time of corrective saccades.

A spatial memory signal shows that the parietal cortex has access to a craniotopic representation of space

Humans effortlessly establish a gist-like memory of their environment whenever they enter a new place, a memory that can guide action even in the absence of vision. Neurons in the lateral intraparietal area (LIP) of the monkey exhibit a form of this environmental memory. These neurons respond when a monkey makes a saccade that brings the spatial location of a stimulus that appeared on a number of prior trials, but not on the present trial, into their receptive fields (RFs). The stimulus need never have appeared in the neuron’s RF. This memory response is usually weaker, with a longer latency than the neuron’s visual response. We suggest that these results demonstrate that LIP has access to a supraretinal memory of space, which is activated when the spatial location of the vanished stimulus can be described by a retinotopic vector from the center of gaze to the remembered spatial location.

Learning Peri-saccadic Remapping of Receptive Field from Experience in Lateral Intraparietal Area

Our eyes move constantly at a frequency of 3-5 times per second. These movements, called saccades, induce the sweeping of visual images on the retina, yet we perceive the world as stable. It has been suggested that the brain achieves this visual stability via predictive remapping of neuronal receptive field (RF). A recent experimental study disclosed details of this remapping process in the lateral intraparietal area (LIP), that is, about the time of the saccade, the neuronal RF expands along the saccadic trajectory temporally, covering the current RF (CRF), the future RF (FRF), and the region the eye will sweep through during the saccade. A cortical wave (CW) model was also proposed, which attributes the RF remapping as a consequence of neural activity propagating in the cortex, triggered jointly by a visual stimulus and the corollary discharge (CD) signal responsible for the saccade. In this study, we investigate how this CW model is learned naturally from visual experiences at the development of the brain. We build a two-layer network, with one layer consisting of LIP neurons and the other superior colliculus (SC) neurons. Initially, neuronal connections are random and non-selective. A saccade will cause a static visual image to sweep through the retina passively, creating the effect of the visual stimulus moving in the opposite direction of the saccade. According to the spiking-time-dependent-plasticity rule, the connection path in the opposite direction of the saccade between LIP neurons and the connection path from SC to LIP are enhanced. Over many such visual experiences, the CW model is developed, which generates the peri-saccadic RF remapping in LIP as observed in the experiment.