Spatiotemporal transformations for gaze control

Choice anticipation as gated accumulation of sensory predictions

Predictions are combined with sensory information when making choices. Accumulator models have conceptualized predictions as trial-by-trial updates to a baseline evidence level. These models have been successful in explaining the influence of choice history across-trials, however, they do not account for how sensory information is transformed into choice evidence. Here, we derive a gated accumulator that models the onset of evidence accumulation as a combination of delayed sensory information and a prediction of sensory timing. To test how delays interact with predictions, we designed a free-choice saccade task where participants directed eye movements to either of two targets that appeared with variable delays and asynchronies. Despite instructions not to anticipate, participants responded before target onset on some trials. We reasoned that anticipatory responses reflected a trade-off between inhibiting and facilitating the onset of evidence accumulation via a gating mechanism as target appearance became more likely. We then found that anticipatory responses were more likely following repeated choices, suggesting that the balance between anticipatory and sensory responses was driven by a prediction of sensory timing. By fitting the gated accumulator model to the data, we found that variance in within-trial fluctuations in baseline evidence best explained the joint increase of anticipatory responses and faster sensory-guided responses with longer delays. Thus, we conclude that a prediction of sensory timing is involved in balancing the costs of anticipation with lowering the amount of accumulated evidence required to trigger saccadic choice.NEW & NOTEWORTHY Evidence accumulation models are used to study how recent history impacts the processes underlying how we make choices. Biophysical evidence suggests that the accumulation of evidence is gated, however, classic accumulator models do not account for this. In this work, we show that predictions of the timing of sensory information are important in controlling how evidence accumulation is gated and that signatures of these predictions can be detected even in randomized task environments.

Neural integration of egocentric and allocentric visual cues in the gaze system

A fundamental question in neuroscience is how the brain integrates egocentric (body-centered) and allocentric (landmark-centered) visual cues, but for many years this question was ignored in sensorimotor studies. This changed in recent behavioral experiments, but the underlying physiology of ego/allocentric integration remained largely unstudied. The specific goal of this review is to explain how prefrontal neurons integrate eye-centered and landmark-centered visual codes for optimal gaze behavior. First, we briefly review the whole brain/behavioral mechanisms for ego/allocentric integration in the human and summarize egocentric coding mechanisms in the primate gaze system. We then focus in more depth on cellular mechanisms for ego/allocentric coding in the frontal and supplementary eye fields. We first explain how prefrontal visual responses integrate eye-centered target and landmark codes to produce a transformation toward landmark-centered coordinates. Next, we describe what happens when a landmark shifts during the delay between seeing and acquiring a remembered target, initially resulting in independently coexisting ego/allocentric memory codes. We then describe how these codes are reintegrated in the motor burst for the gaze shift. Deep network simulations suggest that these properties emerge spontaneously for optimal gaze behavior. Finally, we synthesize these observations and relate them to normal brain function through a simplified conceptual model. Together, these results show that integration of visuospatial features continues well beyond visual cortex and suggest a general cellular mechanism for goal-directed visual behavior.

KETAMINE: Neural- and network-level changes

Ketamine is a widely used clinical drug that has several functional and clinical applications, including its use as an anaesthetic, analgesic, anti-depressive, anti-suicidal agent, among others. Among its diverse behavioral effects, it influences short-term memory and induces psychedelic effects. At the neural level across different brain areas, it modulates neural firing rates, neural tuning, brain oscillations, and modularity, while promoting hypersynchrony and random connectivity between neurons. In our recent studies we demonstrated that topical application of ketamine on the visual cortex alters neural tuning and promotes vigorous connectivity between neurons by decreasing their firing variability. Here, we begin with a brief review of the literature, followed by results from our lab, where we synthesize a dendritic model of neural tuning and network changes following ketamine application. This model has potential implications for focused modulation of cortical networks in clinical settings. Finally, we identify current gaps in research and suggest directions for future studies, particularly emphasizing the need for more animal experiments to establish a platform for effective translation and synergistic therapies combining ketamine with other protocols such as training and adaptation. In summary, investigating ketamine's broader systemic effects, not only provides deeper insight into cognitive functions and consciousness but also paves the way to advance therapies for neuropsychiatric disorders.

Multiunit Frontal Eye Field Activity Codes the Visuomotor Transformation, But Not Gaze Prediction or Retrospective Target Memory, in a Delayed Saccade Task

Single-unit (SU) activity-action potentials isolated from one neuron-has traditionally been employed to relate neuronal activity to behavior. However, recent investigations have shown that multiunit (MU) activity-ensemble neural activity recorded within the vicinity of one microelectrode-may also contain accurate estimations of task-related neural population dynamics. Here, using an established model-fitting approach, we compared the spatial codes of SU response fields with corresponding MU response fields recorded from the frontal eye fields (FEFs) in head-unrestrained monkeys (Macaca mulatta) during a memory-guided saccade task. Overall, both SU and MU populations showed a simple visuomotor transformation: the visual response coded target-in-eye coordinates, transitioning progressively during the delay toward a future gaze-in-eye code in the saccade motor response. However, the SU population showed additional secondary codes, including a predictive gaze code in the visual response and retention of a target code in the motor response. Further, when SUs were separated into regular/fast spiking neurons, these cell types showed different spatial code progressions during the late delay period, only converging toward gaze coding during the final saccade motor response. Finally, reconstructing MU populations (by summing SU data within the same sites) failed to replicate either the SU or MU pattern. These results confirm the theoretical and practical potential of MU activity recordings as a biomarker for fundamental sensorimotor transformations (e.g., target-to-gaze coding in the oculomotor system), while also highlighting the importance of SU activity for coding more subtle (e.g., predictive/memory) aspects of sensorimotor behavior.

Toward a neuroscience of natural behavior

One of the most exciting new developments in systems neuroscience is the progress being made toward neurophysiological experiments that move beyond simplified laboratory settings and address the richness of natural behavior. This is enabled by technological advances such as wireless recording in freely moving animals, automated quantification of behavior, and new methods for analyzing large data sets. Beyond new empirical methods and data, however, there is also a need for new theories and concepts to interpret that data. Such theories need to address the particular challenges of natural behavior, which often differ significantly from the scenarios studied in traditional laboratory settings. Here, we discuss some strategies for developing such novel theories and concepts and some example hypotheses being proposed.

Integration of landmark and saccade target signals in macaque frontal cortex visual responses

Visual landmarks influence spatial cognition and behavior, but their influence on visual codes for action is poorly understood. Here, we test landmark influence on the visual response to saccade targets recorded from 312 frontal and 256 supplementary eye field neurons in rhesus macaques. Visual response fields are characterized by recording neural responses to various target-landmark combinations, and then we test against several candidate spatial models. Overall, frontal/supplementary eye fields response fields preferentially code either saccade targets (40%/40%) or landmarks (30%/4.5%) in gaze fixation-centered coordinates, but most cells show multiplexed target-landmark coding within intermediate reference frames (between fixation-centered and landmark-centered). Further, these coding schemes interact: neurons with near-equal target and landmark coding show the biggest shift from fixation-centered toward landmark-centered target coding. These data show that landmark information is preserved and influences target coding in prefrontal visual responses, likely to stabilize movement goals in the presence of noisy egocentric signals.

Hindbrain modules differentially transform activity of single collicular neurons to coordinate movements

Seemingly simple behaviors such as swatting a mosquito or glancing at a signpost involve the precise coordination of multiple body parts. Neural control of coordinated movements is widely thought to entail transforming a desired overall displacement into displacements for each body part. Here we reveal a different logic implemented in the mouse gaze system. Stimulating superior colliculus (SC) elicits head movements with stereotyped displacements but eye movements with stereotyped endpoints. This is achieved by individual SC neurons whose branched axons innervate modules in medulla and pons that drive head movements with stereotyped displacements and eye movements with stereotyped endpoints, respectively. Thus, single neurons specify a mixture of endpoints and displacements for different body parts, not overall displacement, with displacements for different body parts computed at distinct anatomical stages. Our study establishes an approach for unraveling motor hierarchies and identifies a logic for coordinating movements and the resulting pose.

Rapid Input-Output Transformation between Local Field Potential and Spiking Activity during Sensation but not Action in the Superior Colliculus

Sensorimotor transformation is the sequential process of registering a sensory signal in the environment and then responding with the relevant movement at an appropriate time. For visually guided eye movements, neural signatures in the form of spiking activity of neurons have been extensively studied along the dorsoventral axis of the superior colliculus (SC). In contrast, the local field potential (LFP), which represents the putative input to a region, remains largely unexplored in the SC. We therefore compared amplitude levels and onset times of both spike bursts and LFP modulations recorded simultaneously with a laminar probe along the dorsoventral axis of SC in 3 male monkeys performing the visually guided delayed saccade task. Both signals displayed a gradual transition from sensory activity in the superficial layers to a predominantly motor response in the deeper layers, although the transition from principally sensory to mostly motor response occurred ∼500 μm deeper for the LFP. For the sensory response, LFP modulation preceded spike burst onset by <5 ms in the superficial and intermediate layers and only when data were analyzed on a trial-by-trial basis. The motor burst in the spiking activity led LFP modulation by >25 ms in the deeper layers. The results reveal a fast and efficient input-output transformation between LFP modulation and spike burst in the visually responsive layers activity during sensation but not during action. The spiking pattern observed during the movement phase is likely dominated by intracollicular processing that is not captured in the LFP.SIGNIFICANCE STATEMENT What is the sequence of events between local field potential (LFP) modulation and spiking activity during sensorimotor transformation? A trial-by-trial analysis reveals that the LFP activity leads the spike burst in the superficial and intermediate layers of the superior colliculus during visual processing, while both trial-by-trial and the average analyses show that the spike burst leads the LFP modulation during movement generation. These results suggest an almost instantaneous LFP input, spike burst output transformation in the visually responsive layers of the superior colliculus when registering the stimulus. In contrast, substantial intracollicular processing likely results in a saccade-related spike burst that leads LFP modulation.

Different strategies in pointing tasks and their impact on clinical bedside tests of spatial orientation

Deficits in spatial memory, orientation, and navigation are often early or neglected signs of degenerative and vestibular neurological disorders. A simple and reliable bedside test of these functions would be extremely relevant for diagnostic routine. Pointing at targets in the 3D environment is a basic well-trained common sensorimotor ability that provides a suitable measure. We here describe a smartphone-based pointing device using the built-in inertial sensors for analysis of pointing performance in azimuth and polar spatial coordinates. Interpretation of the vectors measured in this way is not trivial, since the individuals tested may use at least two different strategies: first, they may perform the task in an egocentric eye-based reference system by aligning the fingertip with the target retinotopically or second, by aligning the stretched arm and the index finger with the visual line of sight in allocentric world-based coordinates similar to using a rifle. The two strategies result in considerable differences of target coordinates. A pilot test with a further developed design of the device and an app for a standardized bedside utilization in five healthy volunteers revealed an overall mean deviation of less than 5° between the measured and the true coordinates. Future investigations of neurological patients comparing their performance before and after changes in body position (chair rotation) may allow differentiation of distinct orientational deficits in peripheral (vestibulopathy) or central (hippocampal or cortical) disorders.

Decoding the Time Course of Spatial Information from Spiking and Local Field Potential Activities in the Superior Colliculus

Place code representation is ubiquitous in circuits that encode spatial parameters. For visually guided eye movements, neurons in many brain regions emit spikes when a stimulus is presented in their receptive fields and/or when a movement is directed into their movement fields. Crucially, individual neurons respond for a broad range of directions or eccentricities away from the optimal vector, making it difficult to decode the stimulus location or the saccade vector from each cell's activity. We investigated whether it is possible to decode the spatial parameter with a population-level analysis, even when the optimal vectors are similar across neurons. Spiking activity and local field potentials (LFPs) in the superior colliculus (SC) were recorded with a laminar probe as monkeys performed a delayed saccade task to one of eight targets radially equidistant in direction. A classifier was applied offline to decode the spatial configuration as the trial progresses from sensation to action. For spiking activity, decoding performance across all eight directions was highest during the visual and motor epochs and lower but well above chance during the delay period. Classification performance followed a similar pattern for LFP activity too, except the performance during the delay period was limited mostly to the preferred direction. Increasing the number of neurons in the population consistently increased classifier performance for both modalities. Overall, this study demonstrates the power of population activity for decoding spatial information not possible from individual neurons.

Visual Attention in the Prefrontal Cortex

Voluntary attention selects behaviorally relevant signals for further processing while filtering out distracter signals. Neural correlates of voluntary visual attention have been reported across multiple areas of the primate visual processing streams, with the earliest and strongest effects isolated in the prefrontal cortex. In this article, I review evidence supporting the hypothesis that signals guiding the allocation of voluntary attention emerge in areas of the prefrontal cortex and reach upstream areas to modulate the processing of incoming visual information according to its behavioral relevance. Areas located anterior and dorsal to the arcuate sulcus and the frontal eye fields produce signals that guide the allocation of spatial attention. Areas located anterior and ventral to the arcuate sulcus produce signals for feature-based attention. Prefrontal microcircuits are particularly suited to supporting voluntary attention because of their ability to generate attentional template signals and implement signal gating and their extensive connectivity with the rest of the brain. Expected final online publication date for the Annual Review of Vision Science, Volume 8 is September 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

The role of temporal cortex in the control of attention

Attention is an indispensable component of active vision. Contrary to the widely accepted notion that temporal cortex processing primarily focusses on passive object recognition, a series of very recent studies emphasize the role of temporal cortex structures, specifically the superior temporal sulcus (STS) and inferotemporal (IT) cortex, in guiding attention and implementing cognitive programs relevant for behavioral tasks. The goal of this theoretical paper is to advance the hypothesis that the temporal cortex attention network (TAN) entails necessary components to actively participate in attentional control in a flexible task-dependent manner. First, we will briefly discuss the general architecture of the temporal cortex with a focus on the STS and IT cortex of monkeys and their modulation with attention. Then we will review evidence from behavioral and neurophysiological studies that support their guidance of attention in the presence of cognitive control signals. Next, we propose a mechanistic framework for executive control of attention in the temporal cortex. Finally, we summarize the role of temporal cortex in implementing cognitive programs and discuss how they contribute to the dynamic nature of visual attention to ensure flexible behavior.

Depression and Cognitive Impairment: Current Understanding of Its Neurobiology and Diagnosis

BACKGROUND

Eye movement is critical for obtaining precise visual information and providing sensorimotor processes and advanced cognitive functions to the brain behavioral indicator.

METHODS

In this article, we present a narrative review of the eye-movement paradigms (such as fixation, smooth pursuit eye movements, and memory-guided saccade tasks) in major depression.

RESULTS

Characteristics of eye movement are considered to reflect several aspects of cognitive deficits regarded as an aid to diagnosis. Findings regarding depressive disorders showed differences from the healthy population in paradigms, the characteristics of eye movement may reflect cognitive deficits in depression. Neuroimaging studies have demonstrated the effectiveness of different eye movement paradigms for MDD screening.

CONCLUSION

Depression can be distinguished from other mental illnesses based on eye movements. Eye movement reflects cognitive deficits that can help diagnose depression, and it can make the entire diagnostic process more accurate.

Design and Preliminary Evaluation of a Tongue-Operated Exoskeleton System for Upper Limb Rehabilitation

Stroke is a devastating condition that may cause upper limb paralysis. Robotic rehabilitation with self-initiated and assisted movements is a promising technology that could help restore upper limb function. Previous studies have established that the tongue motion can be used to communicate human intent and control a rehabilitation robot/assistive device. The goal of this study was to evaluate a tongue-operated exoskeleton system (TDS-KA), which we have developed for upper limb rehabilitation. We adopted a tongue-operated assistive technology, called the tongue drive system (TDS), and interfaced it with the exoskeleton KINARM. We also developed arm reaching and tracking tasks, controlled by different tongue operation modes, for training and evaluation of arm motor function. Arm reaching and tracking tasks were tested in 10 healthy participants (seven males and three females, 23-60 years) and two female stroke survivors with upper extremity impairment (32 and 58 years). All healthy and two stroke participants successfully performed the tasks. One stroke subject demonstrated a clinically significant improvement in Fugl-Meyer upper extremity score after practicing the tasks in six 3-h sessions. We conclude that the TDS-KA system can accurately translate tongue commands to exoskeleton arm movements, quantify the function of the arm, and perform rehabilitation training.

Role of the Superior Colliculus in Guiding Movements Not Made by the Eyes

The superior colliculus (SC) has long been associated with the neural control of eye movements. Over seventy years ago, the orderly topography of saccade vectors and corresponding visual field locations was discovered in the cat SC. Since then, numerous high-impact studies have investigated and manipulated the relationship between visuotopic space and saccade vector across this topography to better understand the physiological underpinnings of the sensorimotor signal transformation. However, less attention has been paid to the other motor responses that may be associated with SC activity, ranging in complexity from concerted movements of skeletomotor muscle groups, such as arm-reaching movements, to behaviors that involve whole-body movement sequences, such as fight-or-flight responses in murine models. This review surveys these more complex movements associated with SC (optic tectum in nonmammalian species) activity and, where possible, provides phylogenetic and ethological perspective. Expected final online publication date for the Annual Review of Vision Science, Volume 7 is September 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Active vision at the foveal scale in the primate superior colliculus

The primate superior colliculus (SC) has recently been shown to possess both a large foveal representation as well as a varied visual processing repertoire. This structure is also known to contribute to eye movement generation. Here, we describe our current understanding of how SC visual and movement-related signals interact within the realm of small eye movements associated with the foveal scale of visuomotor behavior. Within the SC's foveal representation, there is a full spectrum of visual, visual-motor, and motor-related discharge for fixational eye movements. Moreover, a substantial number of neurons only emit movement-related discharge when microsaccades are visually guided, but not when similar movements are generated toward a blank. This represents a particularly striking example of integrating vision and action at the foveal scale. Beyond that, SC visual responses themselves are strongly modulated, and in multiple ways, by the occurrence of small eye movements. Intriguingly, this impact can extend to eccentricities well beyond the fovea, causing both sensitivity enhancement and suppression in the periphery. Because of large foveal magnification of neural tissue, such long-range eccentricity effects are neurally warped into smaller differences in anatomical space, providing a structural means for linking peripheral and foveal visual modulations around fixational eye movements. Finally, even the retinal-image visual flows associated with tiny fixational eye movements are signaled fairly faithfully by peripheral SC neurons with relatively large receptive fields. These results demonstrate how studying active vision at the foveal scale represents an opportunity for understanding primate vision during natural behaviors involving ever-present foveating eye movements.NEW & NOTEWORTHY The primate superior colliculus (SC) is ideally suited for active vision at the foveal scale: it enables detailed foveal visual analysis by accurately driving small eye movements, and it also possesses a visual processing machinery that is sensitive to active eye movement behavior. Studying active vision at the foveal scale in the primate SC is informative for broader aspects of active perception, including the overt and covert processing of peripheral extra-foveal visual scene locations.

Spatiotemporal Coding in the Macaque Supplementary Eye Fields: Landmark Influence in the Target-to-Gaze Transformation

Eye-centered (egocentric) and landmark-centered (allocentric) visual signals influence spatial cognition, navigation, and goal-directed action, but the neural mechanisms that integrate these signals for motor control are poorly understood. A likely candidate for egocentric/allocentric integration in the gaze control system is the supplementary eye fields (SEF), a mediofrontal structure with high-level “executive” functions, spatially tuned visual/motor response fields, and reciprocal projections with the frontal eye fields (FEF). To test this hypothesis, we trained two head-unrestrained monkeys (Macaca mulatta) to saccade toward a remembered visual target in the presence of a visual landmark that shifted during the delay, causing gaze end points to shift partially in the same direction. A total of 256 SEF neurons were recorded, including 68 with spatially tuned response fields. Model fits to the latter established that, like the FEF and superior colliculus (SC), spatially tuned SEF responses primarily showed an egocentric (eye-centered) target-to-gaze position transformation. However, the landmark shift influenced this default egocentric transformation: during the delay, motor neurons (with no visual response) showed a transient but unintegrated shift (i.e., not correlated with the target-to-gaze transformation), whereas during the saccade-related burst visuomotor (VM) neurons showed an integrated shift (i.e., correlated with the target-to-gaze transformation). This differed from our simultaneous FEF recordings (Bharmauria et al., 2020), which showed a transient shift in VM neurons, followed by an integrated response in all motor responses. Based on these findings and past literature, we propose that prefrontal cortex incorporates landmark-centered information into a distributed, eye-centered target-to-gaze transformation through a reciprocal prefrontal circuit.