Express averaging saccades in monkeys

Probabilistically constrained vector summation of motion direction in the mouse superior colliculus

Visual motion is a crucial cue for the brain to track objects and take appropriate actions, enabling effective interactions with the environment. Here, we study how the superior colliculus (SC) integrates motion information using asymmetric plaids composed of drifting gratings of different directions and speeds. With both in vivo electrophysiology and two-photon calcium imaging, we find that mouse SC neurons integrate motion direction by performing vector summation of the component gratings. The computation is constrained probabilistically by the possible physical motions consistent with each grating. Excitatory and inhibitory SC neurons respond similarly to the plaid stimuli. Finally, the probabilistically constrained vector summation also guides optokinetic eye movements. Such a computation is fundamentally different from that in the visual cortex, where motion integration follows the intersection of the constraints. Our studies thus demonstrate a novel neural computation in motion processing and raise intriguing questions regarding its neuronal implementation and functional significance.

The adaptive global effect: Luminance contrast modulates the global effect zone

When two peripheral objects are presented in close proximity, saccades towards one of these objects land at a weighted average location between the two objects. This phenomenon, known as the 'global effect' or 'saccade averaging', disappears when the distance between the objects increases. When objects are further apart, outside the averaging zone, saccades land on one of the objects with little or no saccade averaging. Although it is known that the strength of the global effect is dependent on the specific features of the two objects, it is unclear if the size of the zone in which averaging can occur (i.e., the averaging zone) is adaptive. The aim of the current study was to investigate whether the size of the averaging zone adapts to variations in object luminance contrast of the objects. In order to systematically assess changes in the averaging zone, in two experiments, observers made saccadic eye movements while the luminance of the target and the distractor varied. We report three major findings: 1) When a distractor was more luminant relative to the target, the averaging zone increased (Exp. 1). Notably, saccade averaging never entirely ceased to exist, even for remote distractors. 2) When target and distractor were equiluminant, the averaging zone did not change with absolute luminance (Exp. 2). 3) Higher (relative and absolute) luminance increased the averaging zone especially for shorter saccadic response times (SRT). We conclude that the averaging zone is adaptive and becomes larger with increasing relative luminance and especially when SRTs are short.

Pathways for Naturalistic Looking Behavior in Primate I: Behavioral Characteristics and Brainstem Circuits

Organisms control their visual worlds by moving their eyes, heads, and bodies. This control of "gaze" or "looking" is key to survival and intelligence, but our investigation of the underlying neural mechanisms in natural conditions is hindered by technical limitations. Recent advances have enabled measurement of both brain and behavior in freely moving animals in complex environments, expanding on historical head-fixed laboratory investigations. We juxtapose looking behavior as traditionally measured in the laboratory against looking behavior in naturalistic conditions, finding that behavior changes when animals are free to move or when stimuli have depth or sound. We specifically focus on the brainstem circuits driving gaze shifts and gaze stabilization. The overarching goal of this review is to reconcile historical understanding of the differential neural circuits for different "classes" of gaze shift with two inconvenient truths. (1) "classes" of gaze behavior are artificial. (2) The neural circuits historically identified to control each "class" of behavior do not operate in isolation during natural behavior. Instead, multiple pathways combine adaptively and non-linearly depending on individual experience. While the neural circuits for reflexive and voluntary gaze behaviors traverse somewhat independent brainstem and spinal cord circuits, both can be modulated by feedback, meaning that most gaze behaviors are learned rather than hardcoded. Despite this flexibility, there are broadly enumerable neural pathways commonly adopted among primate gaze systems. Parallel pathways which carry simultaneous evolutionary and homeostatic drives converge in superior colliculus, a layered midbrain structure which integrates and relays these volitional signals to brainstem gaze-control circuits.

The reduction of saccadic inhibition by distractor repetition

When visual distractors are presented far from the goal of an impending voluntary saccadic eye movement, saccade execution will occur less frequently about 90 ms after distractor appearance, a phenomenon known as saccadic inhibition. However, it is also known that neural responses in visual and visuomotor areas of the brain will be attenuated if a visual stimulus appears several times in the same location in rapid succession. In particular, such visual adaptation can affect neurons in the mammalian superior colliculus (SC). As the SC is known to be intimately involved in the production of saccadic eye movements, and thus perhaps in saccadic inhibition, we used a memory-guided saccade task to test whether saccadic inhibition in humans would diminish if a distractor appeared several times in quick succession. We found that distractor repetition reduced saccadic inhibition considerably when distractors appeared opposite in space to the goal of the impending saccade. In addition, when three distractors appeared in quick succession but in different, spatially disparate locations, with only the final distractor appearing opposite the saccade goal, saccadic inhibition was reduced by an intermediate level, suggesting that its reduction due to distractor inhibition spatially generalizes. This suggests that distractor suppression can help reduce the impact that suddenly appearing visual stimuli have on purposive eye movement behavior.NEW & NOTEWORTHY This work combines approaches studying saccadic inhibition and visual adaptation to demonstrate that saccadic inhibition is largely eliminated with stimulus repetition. This is likely to be the largest demonstrated effect of visual stimulus context on saccadic inhibition. It also provides evidence for the existence of a mechanism that acts to suppress the effect of frequently appearing visual stimuli on purposive eye movement behavior in dynamic visual environments.

Single-Trial Dynamics of Competing Reach Plans in the Human Motor Periphery

Contemporary motor control theories propose competition between multiple motor plans before the winning command is executed. While most competitions are completed before movement onset, movements are often initiated before the competition has been resolved. An example of this is saccadic averaging, wherein the eyes land at an intermediate location between two visual targets. Behavioral and neurophysiological signatures of competing motor commands have also been reported for reaching movements, but debate remains about whether such signatures attest to an unresolved competition, arise from averaging across many trials, or reflect a strategy to optimize behavior given task constraints. Here, we recorded EMG activity from an upper limb muscle (m. pectoralis) while 12 (8 female) participants performed an immediate response reach task, freely choosing between one of two identical and suddenly presented visual targets. On each trial, muscle recruitment showed two distinct phases of directionally tuned activity. In the first wave, time-locked ∼100 ms of target presentation, muscle activity was clearly influenced by the nonchosen target, reflecting a competition between reach commands that was biased in favor of the ultimately chosen target. This resulted in an initial movement intermediate between the two targets. In contrast, the second wave, time-locked to voluntary reach onset, was not biased toward the nonchosen target, showing that the competition between targets was resolved. Instead, this wave of activity compensated for the averaging induced by the first wave. Thus, single-trial analysis reveals an evolution in how the nonchosen target differentially influences the first and second wave of muscle activity.SIGNIFICANCE STATEMENT Contemporary theories of motor control suggest that multiple motor plans compete for selection before the winning command is executed. Evidence for this is found in intermediate reach movements toward two potential target locations, but recent findings have challenged this notion by arguing that intermediate reaching movements reflect an optimal response strategy. By examining upper limb muscle recruitment during a free-choice reach task, we show early recruitment of a suboptimal averaged motor command to the two targets that subsequently transitions to a single motor command that compensates for the initially averaged motor command. Recording limb muscle activity permits single-trial resolution of the dynamic influence of the nonchosen target through time.

Parallel movement planning is achieved via an optimal preparatory state in motor cortex

How do patterns of neural activity in the motor cortex contribute to the planning of a movement? A recent theory developed for single movements proposes that the motor cortex acts as a dynamical system whose initial state is optimized during the preparatory phase of the movement. This theory makes important yet untested predictions about preparatory dynamics in more complex behavioral settings. Here, we analyze preparatory activity in non-human primates planning not one but two movements simultaneously. As predicted by the theory, we find that parallel planning is achieved by adjusting preparatory activity within an optimal subspace to an intermediate state reflecting a trade-off between the two movements. The theory quantitatively accounts for the relationship between this intermediate state and fluctuations in the animals' behavior down at the trial level. These results uncover a simple mechanism for planning multiple movements in parallel and further point to motor planning as a controlled dynamical process.

A neurocomputational theory of action regulation predicts motor behavior in neurotypical individuals and patients with Parkinson’s disease

Surviving in an uncertain environment requires not only the ability to select the best action, but also the flexibility to withhold inappropriate actions when the environmental conditions change. Although selecting and withholding actions have been extensively studied in both human and animals, there is still lack of consensus on the mechanism underlying these action regulation functions, and more importantly, how they inter-relate. A critical gap impeding progress is the lack of a computational theory that will integrate the mechanisms of action regulation into a unified framework. The current study aims to advance our understanding by developing a neurodynamical computational theory that models the mechanism of action regulation that involves suppressing responses, and predicts how disruption of this mechanism can lead to motor deficits in Parkinson’s disease (PD) patients. We tested the model predictions in neurotypical individuals and PD patients in three behavioral tasks that involve free action selection between two opposed directions, action selection in the presence of conflicting information and abandoning an ongoing action when a stop signal is presented. Our results and theory suggest an integrated mechanism of action regulation that affects both action initiation and inhibition. When this mechanism is disrupted, motor behavior is affected, leading to longer reaction times and higher error rates in action inhibition.

Motor planning under uncertainty

Actions often require the selection of a specific goal amongst a range of possibilities, like when a softball player must precisely position her glove to field a fast-approaching ground ball. Previous studies have suggested that during goal uncertainty the brain prepares for all potential goals in parallel and averages the corresponding motor plans to command an intermediate movement that is progressively refined as additional information becomes available. Although intermediate movements are widely observed, they could instead reflect a neural decision about the single best action choice given the uncertainty present. Here we systematically dissociate these possibilities using novel experimental manipulations and find that when confronted with uncertainty, humans generate a motor plan that optimizes task performance rather than averaging potential motor plans. In addition to accurate predictions of population-averaged changes in motor output, a novel computational model based on this performance-optimization theory accounted for a majority of the variance in individual differences between participants. Our findings resolve a long-standing question about how the brain selects an action to execute during goal uncertainty, providing fundamental insight into motor planning in the nervous system.

Visual attention and eye movement control during oculomotor competition

Saccadic eye movements are typically preceded by selective shifts of visual attention. Recent evidence, however, suggests that oculomotor selection can occur in the absence of attentional selection when saccades erroneously land in between nearby competing objects (saccade averaging). This study combined a saccade task with a visual discrimination task to investigate saccade target selection during episodes of competition between a saccade target and a nearby distractor. We manipulated the spatial predictability of target and distractor locations and asked participants to execute saccades upon variably delayed go-signals. This allowed us to systematically investigate the capacity to exert top-down eye movement control (as reflected in saccade endpoints) based on the spatiotemporal dynamics of visual attention during movement preparation (measured as visual sensitivity). Our data demonstrate that the predictability of target and distractor locations, despite not affecting the deployment of visual attention prior to movement preparation, largely improved the accuracy of short-latency saccades. Under spatial uncertainty, a short go-signal delay likewise enhanced saccade accuracy substantially, which was associated with a more selective deployment of attentional resources to the saccade target. Moreover, we observed a systematic relationship between the deployment of visual attention and saccade accuracy, with visual discrimination performance being significantly enhanced at the saccade target relative to the distractor only before the execution of saccades accurately landing at the saccade target. Our results provide novel insights linking top-down eye movement control to the operation of selective visual attention during movement preparation.

Inhibition of saccade initiation improves saccade accuracy: The role of local and remote visual distractors in the control of saccadic eye movements

When a distractor appears close to the target location, saccades are less accurate. However, the presence of a further distractor, remote from those stimuli, increases the saccade response latency and improves accuracy. Explanations for this are either that the second, remote distractor impacts directly on target selection processes or that the remote distractor merely impairs the ability to initiate a saccade and changes the time at which unaffected target selection processes are accessed. In order to tease these two explanations apart, here we examine the relationship between latency and accuracy of saccades to a target and close distractor pair while a remote distractor appears at variable distance. Accuracy improvements are found to follow a similar pattern, regardless of the presence of the remote distractor, which suggests that the effect of the remote distractor is not the result of a direct impact on the target selection process. Our findings support the proposal that a remote distractor impairs the ability to initiate a saccade, meaning the competition between target and close distractor is accessed at a later time, thus resulting in more accurate saccades.

Exogenous Orientation of Attention to the Center of Mass in a Visual Search Task

Anne Treisman's scientific career included broad-ranging contributions that advanced our understanding of the attentional mechanisms that people rely on to make sense of the world. In this paper, we describe results from a visual-search paradigm first developed by Grabowecky and Treisman (Grabowecky, 1992). Their design exploited known feature-search asymmetries (Treisman & Gormican, 1988) to investigate the role of a center of mass (CoM) mechanism in determining the initial locus of visual-spatial attention in visual search. The original experiment supported the hypothesis that CoM influences initial orienting of visual-spatial attention, as targets near the CoM of a multi-element array were detected more quickly than targets distant from the CoM. These findings were replicated in a follow-up experiment using a different feature-search asymmetry, with eye-tracking added to verify central fixation. We also investigated whether CoM had any influence on pop-out search, and found no evidence that it does. Surprisingly, the effect of position of the search array on the CoM suggested that CoM may be computed independently for elements contained within each visual hemifield. Whereas our work on CoM with Treisman was initiated within an earlier theoretical context, the present results are also compatible with contemporary theoretical advances; both the early results and the new results can be integrated within current ways of thinking about attention and pre-attentive mechanisms.

The effect of the Müller-Lyer configuration on saccadic eye movements is not fully due to illusory perception

Previous research has shown that both perception and oculomotor control are affected by visual illusions. While these findings appear to suggest a common code of visual processing for perception and oculomotor control, there remains the possibility that the perceptual and the oculomotor effects emerge through partially different processes. In three experiments, we replicated the previous finding that perception and saccades were both biased by the typical Müller-Lyer configurations. However, using a non-Müller-Lyer setup in which the perceptual illusion effect was much restrained, we did not observe a comparable reduction in the saccadic effect. Instead, the saccadic effect by Müller-Lyer configuration could be partially due to the center-of-gravity (CoG) effect (i.e., the tendency for saccades to land at the center of gravity of the stimuli). These results indicate that the influence of the Müller-Lyer configuration on saccadic eye movements is a mixed effect of perceptual representation and CoG, rather than exclusively due to the illusory perception. We further found that the saccadic and perceptual effects were not correlated at the trial-by-trial level, which suggest that there could be largely independent sources of noise for perception and saccadic control.NEW & NOTEWORTHY The Müller-Lyer illusion affects both perception and oculomotor control, but it is unknown whether these effects arise from the same or different underlying mechanisms. We developed a modified version of the Müller-Lyer configuration, which largely reduced the perceptual illusion effect compared with the typical configuration but reduced the saccadic effect to a much less extent. Such difference indicates that influence of the Müller-Lyer configuration on saccadic eye movements is not fully mediated by illusory perception.

Competition between salience and informational value for saccade adaptation

What we see is influenced by where we look. When confronted with multiple relevant targets, inaccurate saccade target selection can impair perceptual performance. Here we ask whether endpoint selection can be optimized by the mechanism maintaining saccade accuracy: saccade adaptation. Therefore, we introduce a double-target adaptation task, where a presaccadic peripheral stimulus (plaid) splits vertically into its two components (Gabor patches) during horizontal saccades. While both targets were task-relevant, one of them provided more information for the perceptual task, because it could only be identified after the saccade with near-foveal vision. The other target was highly salient and could also be identified in the presaccadic plaid using peripheral vision. This double-target paradigm induced saccade adaptation: Without a perceptual task, participants adapted to the salient target. When both targets were judged sequentially, participants mostly adapted to the target they had to judge first. When targets were judged simultaneously, endpoints were biased toward the informative target but showed no gradual learning and fell short of optimality. We observed gradual adaptation when targets shifted randomly such that a strategic adjustment of endpoints was not possible. Overall, these findings show that when multiple targets compete, our oculomotor system can learn to adjust endpoints in order to maximize information for perception. Yet individual variability and other factors affecting target priority play a crucial role.

The temporal and spatial constraints of saccade planning to double-step target displacements

The double-step paradigm investigates the characteristics of planning and execution when the motor system must rapidly adjust for a new goal location. Studies have provided detailed temporal information based on the duration available for the motor system to prepare a new movement trajectory (here referred to as re-preparation time). However, previous work has largely examined single displacement sizes, limiting the spatiotemporal understanding of movement planning and execution. The lack of a description of this behavioral timecourse across increasing displacement sizes is true for saccades, rapid eye movements that redirect the fovea. Furthermore, during the double-step paradigm, the primary saccade often fails to accurately foveate the final target location and a secondary saccade brings the target onto the fovea. However, it is also unknown how this compensation is concurrently modified with the exposure duration and displacement of the movement goal. Here, we examined the amount of time required to change the initial saccade direction to a new target location for relatively small (20°, 30°, and 40°) and large (60° and 90°) target spatial separations. Interestingly, we found a clear relationship between the saccade direction and the amount of time allowed to redirect the movement; across separations, intermediate saccades occurred when approximately 60-140 ms was available to readjust the movement plan. Additionally, there was a consistent relationship between the timing of the secondary saccade and the re-preparation time across jump sizes, suggesting that concurrent movement correction planning was dependent on the amount of exposure to the final movement goal.

Face perception influences the programming of eye movements

Previous studies have shown that face stimuli elicit extremely fast and involuntary saccadic responses toward them, relative to other categories of visual stimuli. In the present study, we further investigated to what extent face stimuli influence the programming and execution of saccades examining their amplitude. We performed two experiments using a saccadic choice task: two images (one with a face, one with a vehicle) were simultaneously displayed in the left and right visual fields of participants who had to initiate a saccade toward the image (Experiment 1) or toward a cross in the image (Experiment 2) containing a target stimulus (a face or a vehicle). Results revealed shorter saccades toward vehicle than face targets, even if participants were explicitly asked to perform their saccades toward a specific location (Experiment 2). Furthermore, error saccades had smaller amplitude than correct saccades. Further analyses showed that error saccades were interrupted in mid-flight to initiate a concurrently-programmed corrective saccade. Overall, these data suggest that the content of visual stimuli can influence the programming of saccade amplitude, and that efficient online correction of saccades can be performed during the saccadic choice task.

Visual attention is not deployed at the endpoint of averaging saccades

The premotor theory of attention postulates that spatial attention arises from the activation of saccade areas and that the deployment of attention is the consequence of motor programming. Yet attentional and oculomotor processes have been shown to be dissociable at the neuronal level in covert attention tasks. To investigate a potential dissociation at the behavioral level, we instructed human participants to move their eyes (saccade) towards 1 of 2 nearby, competing saccade targets. The spatial distribution of visual attention was determined using oriented visual stimuli presented either at the target locations, between them, or at several other equidistant locations. Results demonstrate that accurate saccades towards one of the targets were associated with presaccadic enhancement of visual sensitivity at the respective saccade endpoint compared to the nonsaccaded target location. In contrast, averaging saccades, landing between the 2 targets, were not associated with attentional facilitation at the saccade endpoint. Rather, attention before averaging saccades was equally deployed at the 2 target locations. Taken together, our results reveal that visual attention is not obligatorily coupled to the endpoint of a subsequent saccade. Rather, our results suggest that the oculomotor program depends on the state of attentional selection before saccade onset and that saccade averaging arises from unresolved attentional selection.

The premotor theory of attention postulates that spatial visual attention is a consequence of the brain activity that controls eye movement. Indeed, attention and eye movement share overlapping brain networks, and attention is deployed at the target of an eye movement (saccade) even before the eyes start to move. But is attention always deployed at the endpoint of saccades? Here, we measured visual attention before accurate saccades and before saccades that landed in between 2 targets (averaging saccades). While accurate saccades were associated with a selective enhancement of visual sensitivity at their endpoint, no such enhancement was found at the endpoint of averaging saccades. Rather, visual sensitivity was evenly distributed across the 2 saccade targets, suggesting that saccade averaging arises from unresolved attentional selection. Overall, our results reveal that attention is not always coupled to the endpoint of saccades, arguing against a simplistic view of the premotor theory of attention at the behavioral level. Instead, we propose that saccadic responses depend on the state of attentional selection at saccade onset.

Corrective response times in a coordinated eye-head-arm countermanding task

Inhibition of motor responses has been described as a race between two competing decision processes of motor initiation and inhibition, which manifest as the reaction time (RT) and the stop signal reaction time (SSRT); in the case where motor initiation wins out over inhibition, an erroneous movement occurs that usually needs to be corrected, leading to corrective response times (CRTs). Here we used a combined eye-head-arm movement countermanding task to investigate the mechanisms governing multiple effector coordination and the timing of corrective responses. We found a high degree of correlation between effector response times for RT, SSRT, and CRT, suggesting that decision processes are strongly dependent across effectors. To gain further insight into the mechanisms underlying CRTs, we tested multiple models to describe the distribution of RTs, SSRTs, and CRTs. The best-ranked model (according to 3 information criteria) extends the LATER race model governing RTs and SSRTs, whereby a second motor initiation process triggers the corrective response (CRT) only after the inhibition process completes in an expedited fashion. Our model suggests that the neural processing underpinning a failed decision has a residual effect on subsequent actions. NEW & NOTEWORTHY Failure to inhibit erroneous movements typically results in corrective movements. For coordinated eye-head-hand movements we show that corrective movements are only initiated after the erroneous movement cancellation signal has reached a decision threshold in an accelerated fashion.

Extinction as a deficit of the decision-making circuitry in the posterior parietal cortex

Extinction is a common neurologic deficit that often occurs as one of a constellation of symptoms seen with lesions of the posterior parietal cortex (PPC). Although extinction has typically been considered a deficit in the allocation of attention, new findings, particularly from nonhuman primate studies, point to one potential and important source of extinction as damage to decision-making circuits for actions within the PPC. This new understanding provides clues to potential therapies for extinction. Also the finding that the PPC is important for action decisions and action planning has led to new neuroprosthetic applications using PPC recordings as control signals to assist paralyzed patients.

Dissecting patterns of preparatory activity in the frontal eye fields during pursuit target selection

We investigated the composition of preparatory activity of frontal eye field (FEF) neurons in monkeys performing a pursuit target selection task. In response to the orthogonal motion of a large and a small reward target, monkeys initiated pursuit biased toward the direction of large reward target motion. FEF neurons exhibited robust preparatory activity preceding movement initiation in this task. Preparatory activity consisted of two components, ramping activity that was constant across target selection conditions, and a flat offset in firing rates that signaled the target selection condition. Ramping activity accounted for 50% of the variance in the preparatory activity and was linked most strongly, on a trial-by-trial basis, to pursuit eye movement latency rather than to its direction or gain. The offset in firing rates that discriminated target selection conditions accounted for 25% of the variance in the preparatory activity and was commensurate with a winner-take-all representation, signaling the direction of large reward target motion rather than a representation that matched the parameters of the upcoming movement. These offer new insights into the role that the frontal eye fields play in target selection and pursuit control. They show that preparatory activity in the FEF signals more strongly when to move rather than where or how to move and suggest that structures outside the FEF augment its contributions to the target selection process.NEW & NOTEWORTHY We used the smooth eye movement pursuit system to link between patterns of preparatory activity in the frontal eye fields and movement during a target selection task. The dominant pattern was a ramping signal that did not discriminate between selection conditions and was linked, on trial-by-trial basis, to movement latency. A weaker pattern was composed of a constant signal that discriminated between selection conditions but was only weakly linked to the movement parameters.

Automatic and intentional influences on saccade landing

Saccadic eye movements enable us to rapidly direct our high-resolution fovea onto relevant parts of the visual world. However, while we can intentionally select a location as a saccade target, the wider visual scene also influences our executed movements. In the presence of multiple objects, eye movements may be "captured" to the location of a distractor object, or be biased toward the intermediate position between objects (the "global effect"). Here we examined how the relative strengths of the global effect and visual object capture changed with saccade latency, the separation between visual items and stimulus contrast. Importantly, while many previous studies have omitted giving observers explicit instructions, we instructed participants to either saccade to a specified target object or to the midpoint between two stimuli. This allowed us to examine how their explicit movement goal influenced the likelihood that their saccades terminated at either the target, distractor, or intermediate locations. Using a probabilistic mixture model, we found evidence that both visual object capture and the global effect co-occurred at short latencies and declined as latency increased. As object separation increased, capture came to dominate the landing positions of fast saccades, with reduced global effect. Using the mixture model fits, we dissociated the proportion of unavoidably captured saccades to each location from those intentionally directed to the task goal. From this we could extract the time course of competition between automatic capture and intentional targeting. We show that task instructions substantially altered the distribution of saccade landing points, even at the shortest latencies.NEW & NOTEWORTHY When making an eye movement to a target location, the presence of a nearby distractor can cause the saccade to unintentionally terminate at the distractor itself or the average position in between stimuli. With probabilistic mixture models, we quantified how both unavoidable capture and goal-directed targeting were influenced by changing the task and the target-distractor separation. Using this novel technique, we could extract the time course over which automatic and intentional processes compete for control of saccades.

Rapid Automatic Motor Encoding of Competing Reach Options

Mounting neural evidence suggests that, in situations in which there are multiple potential targets for action, the brain prepares, in parallel, competing movements associated with these targets, prior to implementing one of them. Central to this interpretation is the idea that competing viewed targets, prior to selection, are rapidly and automatically transformed into corresponding motor representations. Here, by applying target-specific, gradual visuomotor rotations and dissociating, unbeknownst to participants, the visual direction of potential targets from the direction of the movements required to reach the same targets, we provide direct evidence for this provocative idea. Our results offer strong empirical support for theories suggesting that competing action options are automatically represented in terms of the movements required to attain them. The rapid motor encoding of potential targets may support the fast optimization of motor costs under conditions of target uncertainty and allow the motor system to inform decisions about target selection.

Efficient Avoidance of the Penalty Zone in Human Eye Movements

People use eye movements extremely effectively to find objects of interest in a cluttered visual scene. Distracting, task-irrelevant attention capturing regions in the visual field should be avoided as they jeopardize the efficiency of search. In the current study, we used eye tracking to determine whether people are able to avoid making saccades to a predetermined visual area associated with a financial penalty, while making fast and accurate saccades towards stimuli placed near the penalty area. We found that in comparison to the same task without a penalty area, the introduction of a penalty area immediately affected eye movement behaviour: the proportion of saccades to the penalty area was immediately reduced. Also, saccadic latencies increased, but quite modestly, and mainly for saccades towards stimuli near the penalty area. We conclude that eye movement behaviour is under efficient cognitive control and thus quite flexible: it can immediately be adapted to changing environmental conditions to improve reward outcome.

Revisiting the global effect and inhibition of return

Saccades toward previously cued locations have longer latencies than saccades toward other locations, a phenomenon known as inhibition of return (IOR). Watanabe (Exp Brain Res 138:330–342. doi:10.1007/s002210100709, 2001) combined IOR with the global effect (where saccade landing points fall in between neighboring objects) to investigate whether IOR can also have a spatial component. When one of two neighboring targets was cued, there was a clear bias away from the cued location. In a condition where both targets were cued, it appeared that the global effect magnitude was similar to the condition without any cues. However, as the latencies in the double cue condition were shorter compared to the no cue condition, it is still an open question whether these results are representative for IOR. Considering the double cue condition can provide valuable insight into the interaction of the mechanisms underlying the two phenomena, here, we revisit this condition in an adapted paradigm. Our paradigm does result in longer latencies for the cued locations, and we find that the magnitude of the global effect is reduced significantly. Unexpectedly, this holds even when only including saccades with the same latencies for both conditions. Thus, the increased latencies associated with IOR cannot directly explain the reduction in global effect. The global effect reduction can likely best be seen as either a result of short-term depression of exogenous visual signals or a result of IOR established at the center of gravity of cues.

Dynamic Integration of Value Information into a Common Probability Currency as a Theory for Flexible Decision Making

Decisions involve two fundamental problems, selecting goals and generating actions to pursue those goals. While simple decisions involve choosing a goal and pursuing it, humans evolved to survive in hostile dynamic environments where goal availability and value can change with time and previous actions, entangling goal decisions with action selection. Recent studies suggest the brain generates concurrent action-plans for competing goals, using online information to bias the competition until a single goal is pursued. This creates a challenging problem of integrating information across diverse types, including both the dynamic value of the goal and the costs of action. We model the computations underlying dynamic decision-making with disparate value types, using the probability of getting the highest pay-off with the least effort as a common currency that supports goal competition. This framework predicts many aspects of decision behavior that have eluded a common explanation.

Hedging your bets: intermediate movements as optimal behavior in the context of an incomplete decision

Existing theories of movement planning suggest that it takes time to select and prepare the actions required to achieve a given goal. These theories often appeal to circumstances where planning apparently goes awry. For instance, if reaction times are forced to be very low, movement trajectories are often directed between two potential targets. These intermediate movements are generally interpreted as errors of movement planning, arising either from planning being incomplete or from parallel movement plans interfering with one another. Here we present an alternative view: that intermediate movements reflect uncertainty about movement goals. We show how intermediate movements are predicted by an optimal feedback control model that incorporates an ongoing decision about movement goals. According to this view, intermediate movements reflect an exploitation of compatibility between goals. Consequently, reducing the compatibility between goals should reduce the incidence of intermediate movements. In human subjects, we varied the compatibility between potential movement goals in two distinct ways: by varying the spatial separation between targets and by introducing a virtual barrier constraining trajectories to the target and penalizing intermediate movements. In both cases we found that decreasing goal compatibility led to a decreasing incidence of intermediate movements. Our results and theory suggest a more integrated view of decision-making and movement planning in which the primary bottleneck to generating a movement is deciding upon task goals. Determining how to move to achieve a given goal is rapid and automatic.

Two critical processes need to occur before a movement can be made: identification of the goal of the movement and selection and preparation of the motor commands that will be sent to muscles to generate the movement—in other words, what movement to make, and how to make it. It has long been thought that preparing motor commands is a time-consuming process, and theories advocating this view have pointed to instances where apparently the wrong motor commands are issued if insufficient time is available to prepare them. The usual pattern of these wayward movements is that they are intermediate between two potential targets. In this article we show how such intermediate movements can alternatively be viewed as reflecting an intelligent and deliberate decision about how to move, given uncertainty about task goals. Our theory is supported by experiments that show that intermediate movements only occur in conditions where they are advantageous. The implication of our theory is that the primary bottleneck to generating a movement is deciding on exactly what to do; deciding how to do it is rapid and automatic.

Biomimetic race model of the loop between the superior colliculus and the basal ganglia: Subcortical selection of saccade targets

The superior colliculus, a laminar structure involved in the retinotopic mapping of the visual field, plays a cardinal role in several cortical and subcortical pathways of the saccadic system. Although the selection of saccade targets has long been thought to be mainly the product of cortical processes, a growing body of evidence hints at the implication of the superior colliculus in selection processes independent from cortical inputs, capable of producing saccades at latencies incompatible with the cortical pathways. This selection ability could be produced firstly by the lateral connections between the neurons of its maps, and secondly by its interactions with the midbrain basal ganglia, already renowned for their role in decision making. We propose a biomimetic population-coded race model of selection based on a dynamic tecto-basal loop that reproduces the observed ability of the superior colliculus to stochastically select between similar stimuli. Our model's selection accuracy depends on the discriminability of the target and the distractors. Our model also offers an explanation for the phenomenon of Remote Distractor Effect based on the lateral connectivity within the basal ganglia circuitry rather than on lateral inhibitions within the collicular maps. Finally, we propose a role for the intermediate layers of the superior colliculus, as stochastic integrators dynamically gated by the selective disinhibition of the basal ganglia channels that is consistent with the recorded activity profiles of these neurons.

Eye movements are primed toward the center of multiple stimuli even when the interstimulus distances are too large to generate saccade averaging

Prior oculomotor research has established that saccades tend to land near the center of multiple saccade targets when they are near each other. This saccade averaging phenomenon (or global effect) has been ascribed to short-distance lateral excitation between neurons in the superior colliculus. Further, at greater inter-stimulus distances, eye movements tend toward the individual elements. This transition to control by local elements (individuation) with inter-stimulus distance has been attributed to long-range lateral inhibition between neurons in winner-take-all models of oculomotor behavior. We hypothesized that the traditional method of requiring a saccade to an array of multiple, simultaneous targets may entail response ambiguity that intensifies with distance. We resolved the ambiguity by focussing on reaction time of our human participants to a single saccade target after one or more simultaneous priming stimuli. At a 50-ms prime-target interval, saccadic reaction time was shortest for targets closer to the center of the prime stimuli independent of the distance between the primes. This effect was gone at 400 ms. These findings challenge the typical inferences about the neural control of oculomotor behavior that have been derived from the boundary between saccade averaging and individuation and provide a new method to explore eye movements with lessened impact from decision processes.

Response normalization in the superficial layers of the superior colliculus as a possible mechanism for saccadic averaging

How does the brain decide where to look? Neuronal networks within the superior colliculus (SC) encode locations of intended eye movements. When faced with multiple targets, the relative activities of neuronal populations compete for the selection of a saccade. However, the computational principles underlying saccadic choices remain poorly understood. We used voltage imaging of slices of rat SC to record circuit dynamics of population responses to single- and dual-site electrical stimulation to begin to reveal some of the principles of how populations of neurons interact. Stimulation of two distant sites simultaneously within the SC produced two distinct peaks of activity, whereas stimulation of two nearby sites simultaneously exhibited a single, merged peak centered between the two sites. The distances required to produce merged peaks of activity corresponded to target separations that evoked averaging saccades in humans performing a corresponding dual target task. The merged activity was well accounted for by a linear weighed summation and a divisive normalization of the responses evoked by the single-site stimulations. Interestingly, the merging of activity occurred within the superficial SC, suggesting a novel pathway for saccadic eye movement choice.

Visual working memory modulates low-level saccade target selection: evidence from rapidly generated saccades in the global effect paradigm

In three experiments, we examined the influence of visual working memory (VWM) on the metrics of saccade landing position in a global effect paradigm. Participants executed a saccade to the more eccentric object in an object pair appearing on the horizontal midline, to the left or right of central fixation. While completing the saccade task, participants maintained a color in VWM for an unrelated memory task. Either the color of the saccade target matched the memory color (target match), the color of the distractor matched the memory color (distractor match), or the colors of neither object matched the memory color (no match). In the no-match condition, saccades tended to land at the midpoint between the two objects: the global, or averaging, effect. However, when one of the two objects matched VWM, the distribution of landing position shifted toward the matching object, both for target match and for distractor match. VWM modulation of landing position was observed even for the fastest quartile of saccades, with a mean latency as low as 112 ms. Effects of VWM on such rapidly generated saccades, with latencies in the express-saccade range, indicate that VWM interacts with the initial sweep of visual sensory processing, modulating perceptual input to oculomotor systems and thereby biasing oculomotor selection. As a result, differences in memory match produce effects on landing position similar to the effects generated by differences in physical salience.

Queuing of concurrent movement plans by basal ganglia

How the brain converts parallel representations of movement goals into sequential movements is not known. We tested the role of basal ganglia (BG) in the temporal control of movement sequences by a convergent approach involving inactivation of the BG by muscimol injections into the caudate nucleus of monkeys and assessing behavior of Parkinson's disease patients, performing a modified double-step saccade task. We tested a critical prediction of a class of competitive queuing models that explains serial behavior as the outcome of a selection of concurrently activated goals. In congruence with these models, we found that inactivation or impairment of the BG unmasked the parallel nature of goal representations such that a significantly greater extent of averaged saccades, curved saccades, and saccade sequence errors were observed. These results suggest that the BG perform a form of competitive queuing, holding the second movement plan in abeyance while the first movement is being executed, allowing the proper temporal control of movement sequences.

How global is the global effect? The spatial characteristics of saccade averaging

When a target and a distractor are presented in close proximity, an eye movement will generally land in between these two elements. This is known as the 'global effect' and has been claimed to be a reflection of the averaged saccade programs towards both locations. The aim of the present study was to systematically investigate whether there is only a limited area in the saccade map in which saccade averaging occurs. To this end, we examined various distances between target and distractor in two experiments and investigated whether the majority of eye movements landed in between the target and the distractor. Results indicated that the endpoint distribution was unimodal for distances up to 35° (in polar coordinates), with saccades generally landing in between the target and the distractor. When the distance was higher than 45°, the saccade endpoint distribution was predominantly bimodal, with saccades landing either on the target or on the distractor. The decrease in saccade averaging was linear until almost no averaging saccades were observed for the longest distances. As saccades landing in between target and distractor reflect a weak, or absent, top-down signal, the present study indicated that top-down information is unable to strongly influence the oculomotor system when target and distractor are presented in close proximity. In this situation, the resulting eye movement is determined by the weighted average of saccade vectors present in a restricted region in the motor map.

Is saccade averaging determined by visual processing or movement planning?

Saccadic averaging that causes subjects' gaze to land between the location of two targets when faced with simultaneously or sequentially presented stimuli has been often used as a probe to investigate the nature of computations that transform sensory representations into an oculomotor plan. Since saccadic movements involve at least two processing stages-a visual stage that selects a target and a movement stage that prepares the response-saccade averaging can either occur due to interference in visual processing or movement planning. By having human subjects perform two versions of a saccadic double-step task, in which the stimuli remained the same, but different instructions were provided (REDIRECT gaze to the later-appearing target vs. FOLLOW the sequence of targets in their order of appearance), we tested two alternative hypotheses. If saccade averaging were due to visual processing alone, the pattern of saccade averaging is expected to remain the same across task conditions. However, whereas subjects produced averaged saccades between two targets in the FOLLOW condition, they produced hypometric saccades in the direction of the initial target in the REDIRECT condition, suggesting that the interaction between competing movement plans produces saccade averaging.

Dissociable spatial and temporal effects of inhibition of return

Inhibition of return (IOR) refers to the relative suppression of processing at locations that have recently been attended. It is frequently explored using a spatial cueing paradigm and is characterized by slower responses to cued than to uncued locations. The current study investigates the impact of IOR on overt visual orienting involving saccadic eye movements. Using a spatial cueing paradigm, our experiments have demonstrated that at a cue-target onset asynchrony (CTOA) of 400 ms saccades to the vicinity of cued locations are not only delayed (temporal cost) but also biased away (spatial effect). Both of these effects are basically no longer present at a CTOA of 1200 ms. At a shorter 200 ms CTOA, the spatial effect becomes stronger while the temporal cost is replaced by a temporal benefit. These findings suggest that IOR has a spatial effect that is dissociable from its temporal effect. Simulations using a neural field model of the superior colliculus (SC) revealed that a theory relying on short-term depression (STD) of the input pathway can explain most, but not all, temporal and spatial effects of IOR.

Dynamic integration of information about salience and value for saccadic eye movements

Humans shift their gaze to a new location several times per second. It is still unclear what determines where they look next. Fixation behavior is influenced by the low-level salience of the visual stimulus, such as luminance, contrast, and color, but also by high-level task demands and prior knowledge. Under natural conditions, different sources of information might conflict with each other and have to be combined. In our paradigm, we trade off visual salience against expected value. We show that both salience and value information influence the saccadic end point within an object, but with different time courses. The relative weights of salience and value are not constant but vary from eye movement to eye movement, depending critically on the availability of the value information at the time when the saccade is programmed. Short-latency saccades are determined mainly by salience, but value information is taken into account for long-latency saccades. We present a model that describes these data by dynamically weighting and integrating detailed topographic maps of visual salience and value. These results support the notion of independent neural pathways for the processing of visual information and value.

Averaging is not everything: the saccade global effect weakens with increasing stimulus size

When two elements are presented closely aligned, the average saccade endpoint will generally be located in between these two elements. This 'global effect' has been explained in terms of the center of gravity account which states that the saccade endpoint is based on the relative saliency of the different elements in the visual display. In the current study, we tested one of the implications of the center of gravity account: when two elements are presented closely aligned with the same size and the same distance from central fixation, the saccade should land on the intermediate location, irrespective of the stimulus size. To this end, two equally-sized elements were presented simultaneously and participants were required to execute an eye movement to the visual information presented on the display. Results showed that the strongest global effect was observed in the condition with smaller stimuli, whereas the saccade averaging was weaker when larger stimuli were presented. In a second experiment, in which only one element was presented, we observed that the width of the distribution of saccade endpoints is influenced by stimulus size in that the distribution is broader with smaller stimuli. We conclude that perfect saccade averaging is not always the default response by the oculomotor system. There appears to be a tendency to initiate an eye movement towards one of the visual elements, which becomes stronger with increasing stimulus size. This effect might be explained by an increased uncertainty in target localization for smaller stimuli, resulting in a higher probability of the merging of two stimulus representations into one representation.

Lateral interactions in the superior colliculus produce saccade deviation in a neural field model

Contrary to human intuition, saccades (rapid eye movements) rarely go directly to their intended destination, but instead typically deviate from the optimal track. Previous studies have demonstrated that saccades may deviate toward or away from irrelevant distractors. Deviation toward distractors is generally explained with theories of "population coding", while deviation away from distractors is believed to be caused by top-down inhibition at the distractor location. With a Mexican-hat shaped lateral interaction kernel, we successfully simulated both deviation toward and away from distractors using a neural field model of the superior colliculus (SC). Our findings suggest that top-down inhibition of the SC is not necessary for the generation of saccade deviations.

Neural dynamics of saccadic and smooth pursuit eye movement coordination during visual tracking of unpredictably moving targets

How does the brain coordinate saccadic and smooth pursuit eye movements to track objects that move in unpredictable directions and speeds? Saccadic eye movements rapidly foveate peripheral visual or auditory targets, and smooth pursuit eye movements keep the fovea pointed toward an attended moving target. Analyses of tracking data in monkeys and humans reveal systematic deviations from predictions of the simplest model of saccade-pursuit interactions, which would use no interactions other than common target selection and recruitment of shared motoneurons. Instead, saccadic and smooth pursuit movements cooperate to cancel errors of gaze position and velocity, and thus to maximize target visibility through time. How are these two systems coordinated to promote visual localization and identification of moving targets? How are saccades calibrated to correctly foveate a target despite its continued motion during the saccade? The neural model proposed here answers these questions. Modeled interactions encompass motion processing areas MT, MST, FPA, DLPN and NRTP; saccade planning and execution areas FEF, LIP, and SC; the saccadic generator in the brain stem; and the cerebellum. Simulations illustrate the model's ability to functionally explain and quantitatively simulate anatomical, neurophysiological and behavioral data about coordinated saccade-pursuit tracking.

Order of operations for decoding superior colliculus activity for saccade generation

To help understand the order of events that occurs when generating saccades, we simulated and tested two commonly stated decoding models that are believed to occur in the oculomotor system: vector averaging (VA) and center-of-mass. To generate accurate saccades, each model incorporates two required criteria: 1) a decoding mechanism that deciphers a population response of the superior colliculus (SC) and 2) an exponential transformation that converts the saccade vector into visual coordinates. The order of these two criteria is used differently within each model, yet the significance of the sequence has not been quantified. To distinguish between each decoding sequence and hence, to determine the order of events necessary to generate accurate saccades, we simulated the two models. Distinguishable predictions were obtained when two simultaneous motor commands are processed by each model. Experimental tests of the models were performed by observing the distribution of endpoints of saccades evoked by weighted, simultaneous microstimulation of two SC sites. The data were consistent with the predictions of the VA model, in which exponential transformation precedes the decoding computation.

Superior colliculus inactivation alters the weighted integration of visual stimuli

The primate superior colliculus (SC) is important for the winner-take-all selection of targets for orienting movements. Such selection takes time, however, and the earliest motor responses typically are guided by a weighted vector average of the visual stimuli, before the winner-take-all selection of a single target. We tested whether SC activity plays a role in this initial stage of orienting by inactivating the SC in two macaques (Macaca mulatta) with local muscimol injections. After SC inactivation, initial orienting responses still followed a vector average, but the contribution of the visual stimulus inside the affected field was decreased, and the contribution of the stimulus outside the affected field was increased. These results demonstrate that the SC plays an important role in the weighted integration of visual signals for orienting, in addition to its role in the winner-take-all selection of the target.

Evidence that indirect inhibition of saccade initiation improves saccade acuracy

Saccadic eye-movements to a visual target are less accurate if there are distracters close to its location (local distracters). The addition of more distracters, remote from the target location (remote distracters), invokes an involuntary increase in the response latency of the saccade and attenuates the effect of local distracters on accuracy. This may be due to the target and distracters directly competing (direct route) or to the remote distracters acting to impair the ability to disengage from fixation (indirect route). To distinguish between these, we examined the development of saccade competition by recording saccade latency and accuracy responses made to a target and local distracter compared with those made with an addition of a remote distracter. The direct route would predict that the remote distracter impacts on the developing competition between target and local distracter, while the indirect route would predict no change as the accuracy benefit here derives from accessing the same competitive process but at a later stage. We found that the presence of the remote distracter did not change the pattern of accuracy improvement. This suggests that the remote distracter was acting along an indirect route that inhibits disengagement from fixation, slows saccade initiation, and enables more accurate saccades to be made.

Prepare for conflict: EEG correlates of the anticipation of target competition during overt and covert shifts of visual attention

When preparing to make a saccadic eye movement in a cued direction, perception of stimuli at the target location is enhanced, just as it is when attention is covertly deployed there. Accordingly, the timing and anatomical sources of preparatory brain activity accompanying shifts of covert attention and saccade preparation tend to exhibit a large degree of overlap. However, there is evidence that preparatory processes are modulated by the foreknowledge of visual distractor competition during covert attention, and it is unknown whether eye movement preparation undergoes equivalent modulation. Here we examine preparatory processes in the electroencephalogram of human participants during four blocked versions of a spatial cueing task, requiring either covert detection or saccade execution, and either containing a distractor or not. As in previous work, a typical pattern of spatially selective occipital, parietal and frontal activity was seen in all task versions. However, whereas distractor presence called on an enhancement of spatially selective visual cortical modulation during covert attention, it instead called on increased activity over frontomedial oculomotor areas in the case of overt saccade preparation. We conclude that, although advance orienting signals may be similar in character during overt and covert conditions, the pattern by which these signals are modulated to ameliorate the behavioral costs of distractor competition is highly distinct, pointing to a degree of separability between the overt and covert systems.

The development of the spatial extent of oculomotor inhibition

Inhibition is intimately involved in the ability to select a target for a goal-directed movement. The effect of distracters on the deviation of oculomotor trajectories and landing positions provides evidence of such inhibition. Individual saccade trajectories and landing positions may deviate initially either towards, or away from, a competing distracter--the direction and extent of this deviation depends upon saccade latency and the target to distracter separation. However, the underlying commonality of the sources of oculomotor inhibition has not been investigated. Here we report the relationship between distracter-related deviation of saccade trajectory, landing position and saccade latency. Observers saccaded to a target which could be accompanied by a distracter shown at various distances from very close (10 angular degrees) to far away (120 angular degrees). A fixation-gap paradigm was used to manipulate latency independently of the influence of competing distracters. When distracters were close to the target, saccade trajectory and landing position deviated toward the distracter position, while at greater separations landing position was always accurate but trajectories deviated away from the distracters. Different spatial patterns of deviations across latency were found. This pattern of results is consistent with the metrics of the saccade reflecting coarse pooling of the ongoing activity at the distracter location: saccade trajectory reflects activity at saccade initiation while landing position reveals activity at saccade end.

Involuntary inhibition of movement initiation alters oculomotor competition resolution

Identifying a stimulus as the target for a goal-directed movement involves inhibiting competing responses. Separable inhibitory interconnections bias local competition to ensure only one stimulus is selected and to alter movement initiation. Behavioural evidence of these inhibitory processes comes from the effects of distracters on oculomotor landing positions and saccade latencies. Here, we investigate the relationship between these two sources of inhibition. Targets were presented with or without close and remote distracters. In separate experiments the possible position and identity of the target and distracters were manipulated. In all cases saccade landing position was found to be less affected by the presence of the close distracter when remote distracters were also present. The involuntary increase in the latency of saccade initiation caused by the presence of the remote distracters alters the state of competitive processes involved in selecting the saccade target thus changing its landing position.

Control of predictive error correction during a saccadic double-step task

We explored the nature of control during error correction using a modified saccadic double-step task in which subjects cancelled the initial saccade to the first target and redirected gaze to a second target. Failure to inhibit was associated with a quick corrective saccade, suggesting that errors and corrections may be planned concurrently. However, because saccade programming constitutes a visual and a motor stage of preparation, the extent to which parallel processing occurs in anticipation of the error is not known. To estimate the time course of error correction, a triple-step condition was introduced that displaced the second target during the error. In these trials, corrective saccades directed at the location of the target prior to the third step suggest motor preparation of the corrective saccade in parallel with the error. To estimate the time course of motor preparation of the corrective saccade, further, we used an accumulator model (LATER) to fit the reaction times to the triple-step stimuli; the best-fit data revealed that the onset of correction could occur even before the start of the error. The estimated start of motor correction was also observed to be delayed as target step delay decreased, suggesting a form of interference between concurrent motor programs. Taken together we interpret these results to indicate that predictive error correction may occur concurrently while the oculomotor system is trying to inhibit an unwanted movement and suggest how inhibitory control and error correction may interact to enable goal-directed behaviors.