Effects of learning on smooth pursuit during transient disappearance of a visual target

Metrics of two-dimensional smooth pursuit are diverse across participants and stable across days

Smooth pursuit eye movements are used to volitionally track moving objects, keeping their image near the fovea. Pursuit gain, the ratio of eye to stimulus speed, is used to quantify tracking accuracy and is usually close to 1 for healthy observers. Although previous studies have shown directional asymmetries such as horizontal gain exceeding vertical gain, the temporal stability of these biases and the correlation between oculomotor metrics for tracking in different directions and speeds have not been investigated. Here, in testing sessions 4 to 10 days apart, 45 human observers tracked targets moving along two-dimensional trajectories. Horizontal, vertical, and radial pursuit gain had high test-retest reliability (mean intraclass correlation 0.84). The frequency of all saccades and anticipatory saccades during pursuit also had high test-retest reliability (intraclass correlation coefficients = 0.66 and 0.73, respectively). In addition, gain metrics showed strong intermetric correlation, and saccade metrics separately showed strong intercorrelation; however, gain and saccade metrics showed only weak intercorrelation. These correlations are likely to originate from a mixture of sensory, motor, and integrative mechanisms. The test-retest reliability of multiple distinct pursuit metrics represents a "pursuit identity" for individuals, but we argue against this ultimately contributing to an oculomotor biomarker.

Prefrontal cortex activity and functional organisation in dual-task ocular pursuit is affected by concurrent upper limb movement

Tracking a moving object with the eyes seems like a simple task but involves areas of prefrontal cortex (PFC) associated with attention, working memory and prediction. Increasing the demand on these processes with secondary tasks can affect eye movements and/or perceptual judgments. This is particularly evident in chronic or acute neurological conditions such as Alzheimer's disease or mild traumatic brain injury. Here, we combined near infrared spectroscopy and video-oculography to examine the effects of concurrent upper limb movement, which provides additional afference and efference that facilitates tracking of a moving object, in a novel dual-task pursuit protocol. We confirmed the expected effects on judgement accuracy in the primary and secondary tasks, as well as a reduction in eye velocity when the moving object was occluded. Although there was limited evidence of oculo-manual facilitation on behavioural measures, performing concurrent upper limb movement did result in lower activity in left medial PFC, as well as a change in PFC network organisation, which was shown by Graph analysis to be locally and globally more efficient. These findings extend upon previous work by showing how PFC is functionally organised to support eye-hand coordination when task demands more closely replicate daily activities.

Interactions between circuit architecture and plasticity in a closed-loop cerebellar system

Determining the sites and directions of plasticity underlying changes in neural activity and behavior is critical for understanding mechanisms of learning. Identifying such plasticity from neural recording data can be challenging due to feedback pathways that impede reasoning about cause and effect. We studied interactions between feedback, neural activity, and plasticity in the context of a closed-loop motor learning task for which there is disagreement about the loci and directions of plasticity: vestibulo-ocular reflex learning. We constructed a set of circuit models that differed in the strength of their recurrent feedback, from no feedback to very strong feedback. Despite these differences, each model successfully fit a large set of neural and behavioral data. However, the patterns of plasticity predicted by the models fundamentally differed, with the direction of plasticity at a key site changing from depression to potentiation as feedback strength increased. Guided by our analysis, we suggest how such models can be experimentally disambiguated. Our results address a long-standing debate regarding cerebellum-dependent motor learning, suggesting a reconciliation in which learning-related changes in the strength of synaptic inputs to Purkinje cells are compatible with seemingly oppositely directed changes in Purkinje cell spiking activity. More broadly, these results demonstrate how changes in neural activity over learning can appear to contradict the sign of the underlying plasticity when either internal feedback or feedback through the environment is present.

How the eyes respond to sounds

Eye movements have been extensively studied with respect to visual stimulation. However, we live in a multisensory world, and how the eyes are driven by other senses has been explored much less. Here, we review the evidence on how audition can trigger and drive different eye responses and which cortical and subcortical neural correlates are involved. We provide an overview on how different types of sounds, from simple tones and noise bursts to spatially localized sounds and complex linguistic stimuli, influence saccades, microsaccades, smooth pursuit, pupil dilation, and eye blinks. The reviewed evidence reveals how the auditory system interacts with the oculomotor system, both behaviorally and neurally, and how this differs from visually driven eye responses. Some evidence points to multisensory interaction, and potential multisensory integration, but the underlying computational and neural mechanisms are still unclear. While there are marked differences in how the eyes respond to auditory compared to visual stimuli, many aspects of auditory-evoked eye responses remain underexplored, and we summarize the key open questions for future research.

Normative tDCS over V5 and FEF reveals practice-induced modulation of extraretinal smooth pursuit mechanisms, but no specific stimulation effect

The neural networks subserving smooth pursuit eye movements (SPEM) provide an ideal model for investigating the interaction of sensory processing and motor control during ongoing movements. To better understand core plasticity aspects of sensorimotor processing for SPEM, normative sham, anodal or cathodal transcranial direct current stimulation (tDCS) was applied over visual area V5 and frontal eye fields (FEF) in sixty healthy participants. The identical within-subject paradigm was used to assess SPEM modulations by practice. While no specific tDCS effects were revealed, within- and between-session practice effects indicate plasticity of top-down extraretinal mechanisms that mainly affect SPEM in the absence of visual input and during SPEM initiation. To explore the potential of tDCS effects, individual electric field simulations were computed based on calibrated finite element head models and individual functional localization of V5 and FEF location (using functional MRI) and orientation (using combined EEG/MEG) was conducted. Simulations revealed only limited electric field target intensities induced by the applied normative tDCS montages but indicate the potential efficacy of personalized tDCS for the modulation of SPEM. In sum, results indicate the potential susceptibility of extraretinal SPEM control to targeted external neuromodulation (e.g., personalized tDCS) and intrinsic learning protocols.

Is Altered Oculomotor Control during Smooth Pursuit Neck Torsion Test Related to Subjective Visual Complaints in Patients with Neck Pain Disorders?

Subjective visual complaints are commonly reported in patients with neck pain, but their relation to objectively measured oculomotor functions during smooth pursuit neck torsion tests (SPNTs) has not yet been investigated. The aim of the study was to analyse classification accuracy of visual symptom intensity and frequency based on SPNT results. Forty-three patients with neck pain were referred by orthopaedic outpatient clinics where they were required to fill out 16-item proformas of visual complaints. Infrared video-oculography was used to measure smooth pursuit eye movements during neutral and neck torsion positions. Parameters of gain and SPNT difference (SPNTdiff) were taken into the Naïve Bayes model as classifiers, while intensity and frequency of visual symptoms were taken as predicted class. Intensity and, to a lesser degree, frequency of visual symptoms previously associated with neck pain or focal vision disorders (computer vision syndrome) showed better classification accuracy using gain at neck torsion position, indicating cervical driven visual disturbances. Moreover, SPNTdiff presented with slightly lower classification accuracy as compared to gain at neck torsion position. Our study confirmed the relationship between cervical driven oculomotor deficits and some visual complaints (concentrating to read, words moving on page, blurred vision, difficulty judging distance, sore eyes, heavy eyes, red eyes, and eyes strain).

The influence of neck torsion and sequence of cycles on intra-trial reliability of smooth pursuit eye movement test in patients with neck pain disorders

The sensory mismatch commonly observed in patients with neck pain disorders could alter intra-trial reliability in simple implicit smooth pursuit eye movement tasks. This could be more pronounced when neck is in torsioned position (SPNT). The aim of this study was to explore the effects of neck torsion, target movement velocity and amplitude on intra-trial reliability of smooth pursuit eye movements in patients with neck pain disorders and healthy individuals. SPNT test was evaluated in 32 chronic neck pain patients and 32 healthy controls. Ten cycles were performed using video-oculography at three different velocities (20° s^-1, 30° s^-1 and 40° s^-1) and at three different amplitudes (30°, 40° and 50°) of target movement. Intra-trial reliability and differences between average gain and SPNT difference from the second to fifth cycle and from the sixth to ninth cycle were assessed using ICC_3.1 and factorial analysis of variance, respectively. Intra-trial reliability for gain and SPNT difference at all target movement amplitudes and velocities proved to be good to excellent in both observed groups. Patients with neck pain disorders presented with a trend of inferior gain performance between the sixth and ninth cycle at 30° s^-1 of target movement as compared to healthy individuals which was only evident when neck was in torsioned position. Although intra-trial reliability of smooth pursuit neck torsion test is good to excellent, the effects of learning are not as pronounced in patients with neck pain disorders.

Dignity neuroscience: universal rights are rooted in human brain science

Universal human rights are defined by international agreements, law, foreign policy, and the concept of inherent human dignity. However, rights defined on this basis can be readily subverted by overt and covert disagreements and can be treated as distant geopolitical events rather than bearing on individuals’ everyday lives. A robust case for universal human rights is urgently needed and must meet several disparate requirements: (1) a framework that resolves tautological definitions reached solely by mutual, revocable agreement; (2) a rationale that transcends differences in beliefs, creed, and culture; and (3) a personalization that empowers both individuals and governments to further human rights protections. We propose that human rights in existing agreements comprise five elemental types: (1) agency, autonomy, and self‐determination; (2) freedom from want; (3) freedom from fear; (4) uniqueness; and (5) unconditionality, including protections for vulnerable populations. We further propose these rights and protections are rooted in fundamental properties of the human brain. We provide a robust, empirical foundation for universal rights based on emerging work in human brain science that we term dignity neuroscience. Dignity neuroscience provides an empirical foundation to support and foster human dignity, universal rights, and their active furtherance by individuals, nations, and international law.

Frontotemporal dementia patients exhibit deficits in predictive saccades

Prediction and time estimation are all but required for motor function in everyday life. In the context of eye movements, for instance, they allow predictive saccades and eye re-acceleration in anticipation of a target re-appearance. While the neural pathways involved are not fully understood, it is known that the frontal lobe plays an important role. As such, neurological disorders that affect it, such as frontotemporal (FTD) dementia, are likely to induce deficits in such movements. In this work, we study the performances of frontotemporal dementia patients in an oculomotor task designed to elicit predictive saccades at different rates, and compare them to young and older adults. Clear deficits in the production of predictive saccades were found in patients, in particular when the time between saccades was short (~500 ms). Furthermore, one asymptomatic C9ORF72 mutation bearer showed patterns of oculomotor behavior similar to FTD patients. He exhibited FTD symptoms within 3 years post-measure, suggesting that an impairment of oculomotor function could be an early clinical sign. Taken together, these results argue in favor of a role of the frontal lobe in predictive movements timing over short timescales, and suggest that predictive saccades in FTD patients warrant further investigation to fully assess their potential as a diagnostic aid.

Different mechanisms for modulation of the initiation and steady-state of smooth pursuit eye movements

Smooth pursuit eye movements are used by primates to track moving objects. They are initiated by sensory estimates of target speed represented in the middle temporal (MT) area of extrastriate visual cortex and then supported by motor feedback to maintain steady-state eye speed at target speed. Here, we show that reducing the coherence in a patch of dots for a tracking target degrades the eye speed both at the initiation of pursuit and during steady-state tracking, when eye speed reaches an asymptote well below target speed. The deficits are quantitatively different between the motor-supported steady-state of pursuit and the sensory-driven initiation of pursuit, suggesting separate mechanisms. The deficit in visually guided pursuit initiation could not explain the deficit in steady-state tracking. Pulses of target speed during steady-state tracking revealed lower sensitivities to image motion across the retina for lower values of dot coherence. However, sensitivity was not zero, implying that visual motion should still be driving eye velocity toward target velocity. When we changed dot coherence from 100% to lower values during accurate steady-state pursuit, we observed larger eye decelerations for lower coherences, as expected if motor feedback was reduced in gain. A simple pursuit model accounts for our data based on separate modulation of the strength of visual-motor transmission and motor feedback. We suggest that reduced dot coherence allows us to observe evidence for separate modulations of the gain of visual-motor transmission during pursuit initiation and of the motor corollary discharges that comprise eye velocity memory and support steady-state tracking.NEW & NOTEWORTHY We exploit low-coherence patches of dots to control the initiation and steady state of smooth pursuit eye movements and show that these two phases of movement are modulated separately by the reliability of visual motion signals. We conclude that the neural circuit for pursuit includes separate modulation of the strength of visual-motor transmission for movement initiation and of eye velocity positive feedback to support steady-state tracking.

Ocular tracking of occluded ballistic trajectories: Effects of visual context and of target law of motion

In tracking a moving target, the visual context may provide cues for an observer to interpret the causal nature of the target motion and extract features to which the visual system is weakly sensitive, such as target acceleration. This information could be critical when vision of the target is temporarily impeded, requiring visual motion extrapolation processes. Here we investigated how visual context influences ocular tracking of motion either congruent or not with natural gravity. To this end, 28 subjects tracked computer-simulated ballistic trajectories either perturbed in the descending segment with altered gravity effects (0g/2g) or retaining natural-like motion (1g). Shortly after the perturbation (550 ms), targets disappeared for either 450 or 650 ms and became visible again until landing. Target motion occurred with either quasi-realistic pictorial cues or a uniform background, presented in counterbalanced order. We analyzed saccadic and pursuit movements after 0g and 2g target-motion perturbations and for corresponding intervals of unperturbed 1g trajectories, as well as after corresponding occlusions. Moreover, we considered the eye-to-target distance at target reappearance. Tracking parameters differed significantly between scenarios: With a neutral background, eye movements did not depend consistently on target motion, whereas with pictorial background they showed significant dependence, denoting better tracking of accelerated targets. These results suggest that oculomotor control is tuned to realistic properties of the visual scene.

Reinforcement effects in anticipatory smooth eye movements

When predictive information about target motion is available, anticipatory smooth pursuit eye movements (aSPEM) are consistently generated before target appearance, thereby reducing the typical sensorimotor delay between target motion onset and foveation. By manipulating the probability for target motion direction, we were able to bias the direction and mean velocity of aSPEM. This suggests that motion-direction expectancy has a strong effect on the initiation of anticipatory movements. To further understand the nature of anticipatory smooth eye movements, we investigated different effects of reinforcement on aSPEM. In a first experiment, the reinforcement was contingent to a particular anticipatory behavior. A monetary reward was associated to a criterion-matching anticipatory velocity as estimated online during the gap before target motion onset. Our results showed a small but significant effect of behavior-contingent monetary reward on aSPEM. In a second experiment, the proportion of rewarded trials was manipulated across motion directions (right vs. left) independently from participants' behavior. Our results indicate that a bias in expected reward does not systematically affect anticipatory eye movements. Overall, these findings strengthen the notion that anticipatory eye movements can be considered as an operant behavior (similar to visually guided ones), whereas the expectancy for a noncontingent reward cannot efficiently bias them.

Weighted integration of short-term memory and sensory signals in the oculomotor system

Oculomotor behaviors integrate sensory and prior information to overcome sensory-motor delays and noise. After much debate about this process, reliability-based integration has recently been proposed and several models of smooth pursuit now include recurrent Bayesian integration or Kalman filtering. However, there is a lack of behavioral evidence in humans supporting these theoretical predictions. Here, we independently manipulated the reliability of visual and prior information in a smooth pursuit task. Our results show that both smooth pursuit eye velocity and catch-up saccade amplitude were modulated by visual and prior information reliability. We interpret these findings as the continuous reliability-based integration of a short-term memory of target motion with visual information, which support modeling work. Furthermore, we suggest that saccadic and pursuit systems share this short-term memory. We propose that this short-term memory of target motion is quickly built and continuously updated, and constitutes a general building block present in all sensorimotor systems.

Davida Teller Award Lecture 2017: What can be learned from natural behavior?

The essentially active nature of vision has long been acknowledged but has been difficult to investigate because of limitations in the available instrumentation, both for measuring eye and body movements and for presenting realistic stimuli in the context of active behavior. These limitations have been substantially reduced in recent years, opening up a wider range of contexts where experimental control is possible. Given this, it is important to examine just what the benefits are for exploring natural vision, with its attendant disadvantages. Work over the last two decades provides insights into these benefits. Natural behavior turns out to be a rich domain for investigation, as it is remarkably stable and opens up new questions, and the behavioral context helps specify the momentary visual computations and their temporal evolution.

Vision and Action

Investigation of natural behavior has contributed a number of insights to our understanding of visual guidance of actions by highlighting the importance of behavioral goals and focusing attention on how vision and action play out in time. In this context, humans make continuous sequences of sensory-motor decisions to satisfy current behavioral goals, and the role of vision is to provide the relevant information for making good decisions in order to achieve those goals. This conceptualization of visually guided actions as a sequence of sensory-motor decisions has been formalized within the framework of statistical decision theory, which structures the problem and provides the context for much recent progress in vision and action. Components of a good decision include the task, which defines the behavioral goals, the rewards and costs associated with those goals, uncertainty about the state of the world, and prior knowledge.

Eye Tracking of Occluded Self-Moved Targets: Role of Haptic Feedback and Hand-Target Dynamics

Previous studies on smooth pursuit eye movements have shown that humans can continue to track the position of their hand, or a target controlled by the hand, after it is occluded, thereby demonstrating that arm motor commands contribute to the prediction of target motion driving pursuit eye movements. Here, we investigated this predictive mechanism by manipulating both the complexity of the hand-target mapping and the provision of haptic feedback. Two hand-target mappings were used, either a rigid (simple) one in which hand and target motion matched perfectly or a nonrigid (complex) one in which the target behaved as a mass attached to the hand by means of a spring. Target animation was obtained by asking participants to oscillate a lightweight robotic device that provided (or not) haptic feedback consistent with the target dynamics. Results showed that as long as 7 s after target occlusion, smooth pursuit continued to be the main contributor to total eye displacement (∼60%). However, the accuracy of eye-tracking varied substantially across experimental conditions. In general, eye-tracking was less accurate under the nonrigid mapping, as reflected by higher positional and velocity errors. Interestingly, haptic feedback helped to reduce the detrimental effects of target occlusion when participants used the nonrigid mapping, but not when they used the rigid one. Overall, we conclude that the ability to maintain smooth pursuit in the absence of visual information can extend to complex hand-target mappings, but the provision of haptic feedback is critical for the maintenance of accurate eye-tracking performance.

Continuous Auditory Feedback of Eye Movements: An Exploratory Study toward Improving Oculomotor Control

As eye movements are mostly automatic and overtly generated to attain visual goals, individuals have a poor metacognitive knowledge of their own eye movements. We present an exploratory study on the effects of real-time continuous auditory feedback generated by eye movements. We considered both a tracking task and a production task where smooth pursuit eye movements (SPEM) can be endogenously generated. In particular, we used a visual paradigm which enables to generate and control SPEM in the absence of a moving visual target. We investigated whether real-time auditory feedback of eye movement dynamics might improve learning in both tasks, through a training protocol over 8 days. The results indicate that real-time sonification of eye movements can actually modify the oculomotor behavior, and reinforce intrinsic oculomotor perception. Nevertheless, large inter-individual differences were observed preventing us from reaching a strong conclusion on sensorimotor learning improvements.

Behavioral characterization of prediction and internal models in adolescents with autistic spectrum disorders

Autism has been considered as a deficit in prediction of the upcoming event or of the sensory consequences of our own movements. To test this hypothesis, we recorded eye movements from high-functioning autistic adolescents and from age-matched controls during a blanking paradigm. In this paradigm, adolescents were instructed to follow a moving target with their eyes even during its transient disappearance. Given the absence of visual information during the blanking period, eye movements during this period are solely controlled on the basis of the prediction of the ongoing target motion. Typical markers of predictive eye movements such as the number and accuracy of predictive saccades and the predictive reacceleration before target reappearance were identical in the two populations. In addition, the synergy of predictive saccades and smooth pursuit observed during the blanking periods, which is a marker for the quality of internal models about target/eye motions, was comparable between these two populations. These results suggest that, in our large population of high-functioning autistic adolescent, both predictive abilities and internal models are left intact in Autism, at least for low-level sensorimotor transformations.

Eye tracking a self-moved target with complex hand-target dynamics

Previous work has shown that the ability to track with the eye a moving target is substantially improved when the target is self-moved by the subject's hand compared with when being externally moved. Here, we explored a situation in which the mapping between hand movement and target motion was perturbed by simulating an elastic relationship between the hand and target. Our objective was to determine whether the predictive mechanisms driving eye-hand coordination could be updated to accommodate this complex hand-target dynamics. To fully appreciate the behavioral effects of this perturbation, we compared eye tracking performance when self-moving a target with a rigid mapping (simple) and a spring mapping as well as when the subject tracked target trajectories that he/she had previously generated when using the rigid or spring mapping. Concerning the rigid mapping, our results confirmed that smooth pursuit was more accurate when the target was self-moved than externally moved. In contrast, with the spring mapping, eye tracking had initially similar low spatial accuracy (though shorter temporal lag) in the self versus externally moved conditions. However, within ∼5 min of practice, smooth pursuit improved in the self-moved spring condition, up to a level similar to the self-moved rigid condition. Subsequently, when the mapping unexpectedly switched from spring to rigid, the eye initially followed the expected target trajectory and not the real one, thereby suggesting that subjects used an internal representation of the new hand-target dynamics. Overall, these results emphasize the stunning adaptability of smooth pursuit when self-maneuvering objects with complex dynamics.

Disappearance of the inversion effect during memory-guided tracking of scrambled biological motion

The human visual system is highly sensitive to biological motion. Even when a point-light walker is temporarily occluded from view by other objects, our eyes are still able to maintain tracking continuity. To investigate how the visual system establishes a correspondence between the biological-motion stimuli visible before and after the disruption, we used the occlusion paradigm with biological-motion stimuli that were intact or scrambled. The results showed that during visually guided tracking, both the observers' predicted times and predictive smooth pursuit were more accurate for upright biological motion (intact and scrambled) than for inverted biological motion. During memory-guided tracking, however, the processing advantage for upright as compared with inverted biological motion was not found in the scrambled condition, but in the intact condition only. This suggests that spatial location information alone is not sufficient to build and maintain the representational continuity of the biological motion across the occlusion, and that the object identity may act as an important information source in visual tracking. The inversion effect disappeared when the scrambled biological motion was occluded, which indicates that when biological motion is temporarily occluded and there is a complete absence of visual feedback signals, an oculomotor prediction is executed to maintain the tracking continuity, which is established not only by updating the target's spatial location, but also by the retrieval of identity information stored in long-term memory.

Development of internal models and predictive abilities for visual tracking during childhood

The prediction of the consequences of our own actions through internal models is an essential component of motor control. Previous studies showed improvement of anticipatory behaviors with age for grasping, drawing, and postural control. Since these actions require visual and proprioceptive feedback, these improvements might reflect both the development of internal models and the feedback control. In contrast, visual tracking of a temporarily invisible target gives specific markers of prediction and internal models for eye movements. Therefore, we recorded eye movements in 50 children (aged 5-19 yr) and in 10 adults, who were asked to pursue a visual target that is temporarily blanked. Results show that the youngest children (5-7 yr) have a general oculomotor behavior in this task, qualitatively similar to the one observed in adults. However, the overall performance of older subjects in terms of accuracy at target reappearance and variability in their behavior was much better than the youngest children. This late maturation of predictive mechanisms with age was reflected into the development of the accuracy of the internal models governing the synergy between the saccadic and pursuit systems with age. Altogether, we hypothesize that the maturation of the interaction between smooth pursuit and saccades that relies on internal models of the eye and target displacement is related to the continuous maturation of the cerebellum.

Filling gaps in visual motion for target capture

A remarkable challenge our brain must face constantly when interacting with the environment is represented by ambiguous and, at times, even missing sensory information. This is particularly compelling for visual information, being the main sensory system we rely upon to gather cues about the external world. It is not uncommon, for example, that objects catching our attention may disappear temporarily from view, occluded by visual obstacles in the foreground. Nevertheless, we are often able to keep our gaze on them throughout the occlusion or even catch them on the fly in the face of the transient lack of visual motion information. This implies that the brain can fill the gaps of missing sensory information by extrapolating the object motion through the occlusion. In recent years, much experimental evidence has been accumulated that both perceptual and motor processes exploit visual motion extrapolation mechanisms. Moreover, neurophysiological and neuroimaging studies have identified brain regions potentially involved in the predictive representation of the occluded target motion. Within this framework, ocular pursuit and manual interceptive behavior have proven to be useful experimental models for investigating visual extrapolation mechanisms. Studies in these fields have pointed out that visual motion extrapolation processes depend on manifold information related to short-term memory representations of the target motion before the occlusion, as well as to longer term representations derived from previous experience with the environment. We will review recent oculomotor and manual interception literature to provide up-to-date views on the neurophysiological underpinnings of visual motion extrapolation.

Influence of predictability on control of extra-retinal components of smooth pursuit during prolonged 2D tracking

We compared pursuit responses to 2D target motion in three separate conditions: predictable, randomised and randomised with timing cues. The target moved on a continuous quadrilateral path in which right-angle direction changes allowed anticipatory eye acceleration and deceleration in orthogonal axes to be assessed. Results indicated that whether the timing of direction changes was random or predictable, anticipatory acceleration, initiated by extra-retinal mechanisms, occurred in the new direction at approximately the same time as anticipatory deceleration in the terminating direction, but deceleration was of greater magnitude than acceleration. When path duration was randomised within a range of durations, the timing of acceleration and deceleration was almost constant irrespective of actual ramp duration but was dependent on the mean duration of the range. When ramp duration was predictable both deceleration and acceleration increased, the latter allowing peak velocity to be attained earlier than when randomised. When timing cues were given at a fixed time prior to direction change in randomised stimuli, this also resulted in higher anticipatory acceleration/deceleration. When both duration and velocity of sequential ramps were randomised, deceleration was dependent on target velocity, but acceleration remained constant. Altogether these findings show that although acceleration and deceleration in orthogonal axes occur almost simultaneously and are similarly affected by predictability, control of their magnitude is relatively independent. We suggest that deceleration and acceleration result from the switching off and on, respectively, of retinal and extra-retinal oculomotor components prior to direction change, with dynamics dependent on predictability of stimulus magnitude and timing.

Active inference, eye movements and oculomotor delays

This paper considers the problem of sensorimotor delays in the optimal control of (smooth) eye movements under uncertainty. Specifically, we consider delays in the visuo-oculomotor loop and their implications for active inference. Active inference uses a generalisation of Kalman filtering to provide Bayes optimal estimates of hidden states and action in generalised coordinates of motion. Representing hidden states in generalised coordinates provides a simple way of compensating for both sensory and oculomotor delays. The efficacy of this scheme is illustrated using neuronal simulations of pursuit initiation responses, with and without compensation. We then consider an extension of the generative model to simulate smooth pursuit eye movements-in which the visuo-oculomotor system believes both the target and its centre of gaze are attracted to a (hidden) point moving in the visual field. Finally, the generative model is equipped with a hierarchical structure, so that it can recognise and remember unseen (occluded) trajectories and emit anticipatory responses. These simulations speak to a straightforward and neurobiologically plausible solution to the generic problem of integrating information from different sources with different temporal delays and the particular difficulties encountered when a system-like the oculomotor system-tries to control its environment with delayed signals.

Dynamic integration of information about salience and value for smooth pursuit eye movements

Eye movement behavior can be determined by bottom-up factors like visual salience and by top-down factors like expected value. These different types of signals have to be combined for the control of eye movements. In this study we investigated how smooth pursuit eye movements integrate salience and value information. Observers were asked to track a random-dot kinematogram containing two coherent motion directions. To manipulate salience, the coherence or the density of one of the motion signals was varied. To manipulate value, observers won or lost money in a separate experiment if they were tracking one or the other motion direction. Our results show that pursuit direction was initially determined only by salience. 300-400 ms after target motion onset, pursuit steered towards the rewarded direction and the salience effects disappeared. The time course of this effect depended crucially on the difficulty to segment the two signal directions. These results indicate that salience determines early pursuit responses in the same way as saccades with short latencies. Value information is processed slower and dominates pursuit after several 100 ms.

Motion integration for ocular pursuit does not hinder perceptual segregation of moving objects

When confronted with a complex moving stimulus, the brain can integrate local element velocities to obtain a single motion signal, or segregate the elements to maintain awareness of their identities. The integrated motion signal can drive smooth-pursuit eye movements (Heinen and Watamaniuk, 1998), whereas the segregated signal guides attentive tracking of individual elements in multiple-object tracking tasks (MOT; Pylyshyn and Storm, 1988). It is evident that these processes can occur simultaneously, because we can effortlessly pursue ambulating creatures while inspecting disjoint moving features, such as arms and legs, but the underlying mechanism is unknown. Here, we provide evidence that separate neural circuits perform the mathematically opposed operations of integration and segregation, by demonstrating with a dual-task paradigm that the two processes do not share attentional resources. Human observers attentively tracked a subset of target elements composing a small MOT stimulus, while pursuing it ocularly as it translated across a computer display. Integration of the multidot stimulus yielded optimal pursuit. Importantly, performing MOT while pursuing the stimulus did not degrade performance on either task compared with when each was performed alone, indicating that they did not share attention. A control experiment showed that pursuit was not driven by integration of only the nontargets, leaving the MOT targets free for segregation. Nor was a predictive strategy used to pursue the stimulus, because sudden changes in its global velocity were accurately followed. The results suggest that separate neural mechanisms can simultaneously segregate and integrate the same motion signals.

Kalman filtering naturally accounts for visually guided and predictive smooth pursuit dynamics

The brain makes use of noisy sensory inputs to produce eye, head, or arm motion. In most instances, the brain combines this sensory information with predictions about future events. Here, we propose that Kalman filtering can account for the dynamics of both visually guided and predictive motor behaviors within one simple unifying mechanism. Our model relies on two Kalman filters: (1) one processing visual information about retinal input; and (2) one maintaining a dynamic internal memory of target motion. The outputs of both Kalman filters are then combined in a statistically optimal manner, i.e., weighted with respect to their reliability. The model was tested on data from several smooth pursuit experiments and reproduced all major characteristics of visually guided and predictive smooth pursuit. This contrasts with the common belief that anticipatory pursuit, pursuit maintenance during target blanking, and zero-lag pursuit of sinusoidally moving targets all result from different control systems. This is the first instance of a model integrating all aspects of pursuit dynamics within one coherent and simple model and without switching between different parallel mechanisms. Our model suggests that the brain circuitry generating a pursuit command might be simpler than previously believed and only implement the functional equivalents of two Kalman filters whose outputs are optimally combined. It provides a general framework of how the brain can combine continuous sensory information with a dynamic internal memory and transform it into motor commands.

Motion-based prediction explains the role of tracking in motion extrapolation

During normal viewing, the continuous stream of visual input is regularly interrupted, for instance by blinks of the eye. Despite these frequents blanks (that is the transient absence of a raw sensory source), the visual system is most often able to maintain a continuous representation of motion. For instance, it maintains the movement of the eye such as to stabilize the image of an object. This ability suggests the existence of a generic neural mechanism of motion extrapolation to deal with fragmented inputs. In this paper, we have modeled how the visual system may extrapolate the trajectory of an object during a blank using motion-based prediction. This implies that using a prior on the coherency of motion, the system may integrate previous motion information even in the absence of a stimulus. In order to compare with experimental results, we simulated tracking velocity responses. We found that the response of the motion integration process to a blanked trajectory pauses at the onset of the blank, but that it quickly recovers the information on the trajectory after reappearance. This is compatible with behavioral and neural observations on motion extrapolation. To understand these mechanisms, we have recorded the response of the model to a noisy stimulus. Crucially, we found that motion-based prediction acted at the global level as a gain control mechanism and that we could switch from a smooth regime to a binary tracking behavior where the dot is tracked or lost. Our results imply that a local prior implementing motion-based prediction is sufficient to explain a large range of neural and behavioral results at a more global level. We show that the tracking behavior deteriorates for sensory noise levels higher than a certain value, where motion coherency and predictability fail to hold longer. In particular, we found that motion-based prediction leads to the emergence of a tracking behavior only when enough information from the trajectory has been accumulated. Then, during tracking, trajectory estimation is robust to blanks even in the presence of relatively high levels of noise. Moreover, we found that tracking is necessary for motion extrapolation, this calls for further experimental work exploring the role of noise in motion extrapolation.

Memory and prediction in natural gaze control

In addition to stimulus properties and task factors, memory is an important determinant of the allocation of attention and gaze in the natural world. One way that the role of memory is revealed is by predictive eye movements. Both smooth pursuit and saccadic eye movements demonstrate predictive effects based on previous experience. We have previously shown that unskilled subjects make highly accurate predictive saccades to the anticipated location of a ball prior to a bounce in a virtual racquetball setting. In this experiment, we examined this predictive behaviour. We asked whether the period after the bounce provides subjects with visual information about the ball trajectory that is used to programme the pursuit movement initiated when the ball passes through the fixation point. We occluded a 100 ms period of the ball's trajectory immediately after the bounce, and found very little effect on the subsequent pursuit movement. Subjects did not appear to modify their strategy to prolong the fixation. Neither were we able to find an effect on interception performance. Thus, it is possible that the occluded trajectory information is not critical for subsequent pursuit, and subjects may use an estimate of the ball's trajectory to programme pursuit. These results provide further support for the role of memory in eye movements.

Useful field of view in simulated driving: Reaction times and eye movements of drivers

To examine the spatial distribution of a useful field of view (UFOV) in driving, reaction times (RTs) and eye movements were measured in simulated driving. In the experiment, a normal or mirror-reversed letter “E” was presented on driving images with different eccentricities and directions from the current gaze position. The results showed significantly slower RTs in the upper and upper left directions than in the other directions. The RTs were significantly slower in the left directions than in the right directions. These results suggest that the UFOV in driving may be asymmetrical among the meridians in the visual field.

Saccades to future ball location reveal memory-based prediction in a virtual-reality interception task

Despite general agreement that prediction is a central aspect of perception, there is relatively little evidence concerning the basis on which visual predictions are made. Although both saccadic and pursuit eye-movements reveal knowledge of the future position of a moving visual target, in many of these studies targets move along simple trajectories through a fronto-parallel plane. Here, using a naturalistic and racquet-based interception task in a virtual environment, we demonstrate that subjects make accurate predictions of visual target motion, even when targets follow trajectories determined by the complex dynamics of physical interactions and the head and body are unrestrained. Furthermore, we found that, following a change in ball elasticity, subjects were able to accurately adjust their prebounce predictions of the ball's post-bounce trajectory. This suggests that prediction is guided by experience-based models of how information in the visual image will change over time.

Predictive eye movements in natural vision

In the natural world, the brain must handle inherent delays in visual processing. This is a problem particularly during dynamic tasks. A possible solution to visuo-motor delays is prediction of a future state of the environment based on the current state and properties of the environment learned from experience. Prediction is well known to occur in both saccades and pursuit movements and is likely to depend on some kind of internal visual model as the basis for this prediction. However, most evidence comes from controlled laboratory studies using simple paradigms. In this study, we examine eye movements made in the context of demanding natural behavior, while playing squash. We show that prediction is a pervasive component of gaze behavior in this context. We show in addition that these predictive movements are extraordinarily precise and operate continuously in time across multiple trajectories and multiple movements. This suggests that prediction is based on complex dynamic visual models of the way that balls move, accumulated over extensive experience. Since eye, head, arm, and body movements all co-occur, it seems likely that a common internal model of predicted visual state is shared by different effectors to allow flexible coordination patterns. It is generally agreed that internal models are responsible for predicting future sensory state for control of body movements. The present work suggests that model-based prediction is likely to be a pervasive component in natural gaze control as well.

Reinforcing saccadic amplitude variability

Saccadic endpoint variability is often viewed as the outcome of neural noise occurring during sensorimotor processing. However, part of this variability might result from operant learning. We tested this hypothesis by reinforcing dispersions of saccadic amplitude distributions, while maintaining constant their medians. In a first experiment we reinforced the least frequent saccadic amplitudes to increase variability, and then reinforced the central part of the amplitude distributions to reduce variability. The target was placed at a constant distance from the fovea after the saccade to maintain the postsaccadic visual signal constant and an auditory reinforcement was delivered depending on saccadic amplitude. The second experiment tested the effects of the contingency. We reinforced high levels of variability in 4 participants, whereas 4 other participants were assigned to a yoked control group. On average, saccadic amplitude standard deviations were doubled while the medians remained mostly unchanged in the experimental participants in both experiments, and variability returned to baseline level when low variability was reinforced. In the control group no consistent changes in amplitude distributions were observed. These results, showing that variability can be reinforced, challenge the idea of a stochastic neural noise. We instead propose that selection processes constrain saccadic amplitude distributions.

Modification of saccadic gain by reinforcement

Control of saccadic gain is often viewed as a simple compensatory process in which gain is adjusted over many trials by the postsaccadic retinal error, thereby maintaining saccadic accuracy. Here, we propose that gain might also be changed by a reinforcement process not requiring a visual error. To test this hypothesis, we used experimental paradigms in which retinal error was removed by extinguishing the target at the start of each saccade and either an auditory tone or the vision of the target on the fovea was provided as reinforcement after those saccades that met an amplitude criterion. These reinforcement procedures caused a progressive change in saccade amplitude in nearly all subjects, although the rate of adaptation differed greatly among subjects. When we reversed the contingencies and reinforced those saccades landing closer to the original target location, saccade gain changed back toward normal gain in most subjects. When subjects had saccades adapted first by reinforcement and a week later by conventional intrasaccadic step adaptation, both paradigms yielded similar degrees of gain changes and similar transfer to new amplitudes and to new starting positions of the target step as well as comparable rates of recovery. We interpret these changes in saccadic gain in the absence of postsaccadic retinal error as showing that saccade adaptation is not controlled by a single error signal. More generally, our findings suggest that normal saccade adaptation might involve general learning mechanisms rather than only specialized mechanisms for motor calibration.

Anticipatory eye movements evoked after active following versus passive observation of a predictable motion stimulus

We used passive and active following of a predictable smooth pursuit stimulus in order to establish if predictive eye movement responses are equivalent under both passive and active conditions. The smooth pursuit stimulus was presented in pairs that were either 'predictable' in which both presentations were matched in timing and velocity, or 'randomized' in which each presentation in the pair was varied in both timing and velocity. A visual cue signaled the type of response required from the subject; a green cue indicated the subject should follow both the target presentations (Go-Go), a pink cue indicated that the subject should passively observe the 1st target and follow the 2nd target (NoGo-Go), and finally a green cue with a black cross revealed a randomized (Rnd) trial in which the subject should follow both presentations. The results revealed better prediction in the Go-Go trials than in the NoGo-Go trials, as indicated by higher anticipatory velocity and earlier eye movement onset (latency). We conclude that velocity and timing information stored from passive observation of a moving target is diminished when compared to active following of the target. This study has significant consequences for understanding how visuomotor memory is generated, stored and subsequently released from short-term memory.

Predictive disjunctive pursuit of virtual images perceived to move in depth

Human and nonhuman primates predictively use smooth pursuit and saccades to track visual targets that move in a fronto-parallel plane. This behaviour is believed to be facilitated by short-term memory of the target motion and/or an efference copy of the subject's motor effort. Subjects in our experiments tracked dichoptically viewed targets that appeared to move vertically, right or left and towards the subject. The virtual image of the target was tracked using disjunctive smooth pursuit and saccades. To reveal predictive tracking, targets were blanked 100 ms after the onset of motion for intervals of 800 ms. During the blanked interval, subjects initiated pursuit and predictively tracked the unseen virtual image using memory guided eye movements. Our data are consistent with recent electrophysiological studies that describe cells that encode target or eye movements in depth when a target is briefly blanked but pursuit is maintained. However, predictive pursuit of a virtual target with disjunctive eye movements poses a challenge for understanding how a short-term memory store might encode the desired eye movement, its coordinate frame, and how it is transformed into motor commands.

Evidence for a link between the extra-retinal component of random-onset pursuit and the anticipatory pursuit of predictable object motion

During pursuit of moving targets that temporarily disappear, residual smooth eye movements represent the internal (extra-retinal) component of pursuit. However, this response is dependent on expectation of target reappearance. By comparing responses with and without such expectation during early random-onset pursuit, we examined the temporal development of the extra-retinal component and compared it with anticipatory pursuit, another form of internally driven response. In an initial task (mid-ramp extinction), a moving, random-velocity target was initially visible for 100 or 150 ms but then extinguished for 600 ms before reappearing and continuing to move. Responses comprised an initial visually driven rapid rise in eye velocity, followed by a secondary slower increase during extinction. In a second task (short ramp), with identical initial target presentation but no expectation of target reappearance, the initial rapid rise in eye velocity was followed by decay toward zero. The expectation-dependent difference between responses to these tasks increased in velocity during extinction much more slowly than the initial, visually driven component. In a third task (initial extinction), the moving target was extinguished at motion onset but reappeared 600 ms later. Repetition of identical stimuli evoked anticipatory pursuit triggered by initial target offset. Temporal development and scaling of this anticipatory response, which was based on remembered velocity from prior stimuli, was remarkably similar to and covaried with the difference between mid-ramp extinction and short ramp tasks. Results suggest a common mechanism is responsible for anticipatory pursuit and the extra-retinal component of random-onset pursuit, a finding that is consistent with a previously developed model of pursuit.

The influence of briefly presented randomized target motion on the extraretinal component of ocular pursuit

We assessed the ability to extract velocity information from brief exposure of a moving target and sought evidence that this information could be used to modulate the extraretinal component of ocular pursuit. A step-ramp target motion was initially visible for a brief randomized period of 50, 100, 150, or 200 ms, but then extinguished for a randomized period of 400 or 600 ms before reappearing and continuing along its trajectory. Target speed (5-20 degrees /s), direction (left/right), and intertrial interval (2.7-3.7 s) were also randomized. Smooth eye movements were initiated after about 130 ms and comprised an initial visually dependent component, which reached a peak velocity that increased with target velocity and initial exposure duration, followed by a sustained secondary component that actually increased throughout extinction for 50- and 100-ms initial exposures. End-extinction eye velocity, reflecting extraretinal drive, increased with initial exposure from 50 to 100 ms but remained similar for longer exposures; it was significantly scaled to target velocity for 150- and 200-ms exposures. The results suggest that extraretinal drive is based on a sample of target velocity, mostly acquired during the first 150 ms, that is stored and forms a goal for generating appropriately scaled eye movements during absence of visual input. End-extinction eye velocity was significantly higher when target reappearance was expected than when it was not, confirming the importance of expectation in generating sustained smooth movement. However, end-extinction eye displacement remained similar irrespective of expectation, suggesting that the ability to use sampled velocity information to predict future target displacement operates independently of the control of smooth eye movement.

Trial-by-trial updating of the gain in preparation for smooth pursuit eye movement based on past experience in humans

To understand how the CNS uses past experiences to generate movements that accommodate minute-by-minute environmental changes, we studied the trial-by-trial updating of the gain for initiating smooth pursuit eye movements and how this relates to the history of previous trials. Ocular responses in humans elicited by a small perturbing motion presented 300 ms after appearance of a target were used as a measure of the gain of visuomotor transmission. After the perturbation, the target was either moved horizontally (pursuit trial) or remained in a stationary position (fixation trial). The trial sequence randomly included pursuit and fixation. The amplitude of the response to the perturbation was modulated in a trial-by-trial manner based on the immediately preceding trial, with preceding fixation and pursuit trials decreasing and increasing the gain, respectively. The effect of the previous trial was larger with shorter intertrial intervals, but did not diminish for at least 2,000 ms. A time-series analysis showed that the response amplitude was significantly correlated with the past few trials, with dynamics that could be approximated by a first-order linear system. The results suggest that the CNS integrates recent experiences to set the gain in preparation for upcoming tracking movements in a changing environment.

Saccades and pursuit: two outcomes of a single sensorimotor process

Saccades and smooth pursuit eye movements are two different modes of oculomotor control. Saccades are primarily directed toward stationary targets whereas smooth pursuit is elicited to track moving targets. In recent years, behavioural and neurophysiological data demonstrated that both types of eye movements work in synergy for visual tracking. This suggests that saccades and pursuit are two outcomes of a single sensorimotor process that aims at orienting the visual axis.

Velocity scaling of cue-induced smooth pursuit acceleration obeys constraints of natural motion

Information about the future trajectory of a visual target is contained not only in the history of target motion but also in static visual cues, e.g., the street provides information about the car's future trajectory. For most natural moving targets, this information imposes strong constraints on the relation between velocity and acceleration which can be exploited by predictive smooth pursuit mechanisms. We questioned how cue-induced predictive changes in pursuit direction depend on target speed and how cue- and target-induced pursuit interact. Subjects pursued a target entering a +/-90 degrees curve and moving on either a homogeneous background or on a low contrast static band indicating the future trajectory. The cue induced a predictive change of pursuit direction, which occurred before curve onset of the target. The predictive velocity component orthogonal to the initial pursuit direction started later and became faster with increasing target velocity. The predictive eye acceleration increased quadratically with target velocity and was independent of the initial target direction. After curve onset, cue- and target-induced pursuit velocity components were not linearly superimposed. The quadratic increase of eye acceleration with target velocity is consistent with the natural velocity scaling implied by the two-thirds power law, which is a characteristic of biological controlled movements. Comparison with linear pursuit models reveals that the ratio between eye acceleration and actual or expected retinal slip cannot be considered a constant gain factor. To obey a natural velocity scaling, this acceleration gain must linearly increase with target or pursuit velocity. We suggest that gain control mechanisms, which affect target-induced changes of pursuit velocity, act similarly on predictive changes of pursuit induced by static visual cues.

A quantitative synchronization model for smooth pursuit target tracking

We propose a quantitative model for human smooth pursuit tracking of a continuously moving visual target which is based on synchronization of an internal expectancy model of the target position coupled to the retinal target signal. The model predictions are tested in a smooth circular pursuit eye tracking experiment with transient target blanking of variable duration. In subjects with a high tracking accuracy, the model accounts for smooth pursuit and repeatedly reproduces quantitatively characteristic patterns of the eye dynamics during target blanking. In its simplest form, the model has only one free parameter, a coupling constant. An extended model with a second parameter, a time delay or memory term, accounts for predictive smooth pursuit eye movements which advance the target. The model constitutes an example of synchronization of a complex biological system with perceived sensory signals.