A common stochastic accumulator with effector-dependent noise can explain eye-hand coordination

Restart errors reaction time of a two-step inhibition process account for the violation of the race model's independence in multi-effector selective stop signal task

Goal-oriented actions often require the coordinated movement of two or more effectors. Sometimes multi-effector movements need to be adjusted according to a continuously changing environment, requiring stopping an effector without interrupting the movement of the others. This form of control has been investigated by the selective Stop Signal Task (SST), requiring the inhibition of an effector of a multicomponent action. This form of selective inhibition has been hypothesized to act through a two-step process, where a temporary global inhibition deactivating all the ongoing motor responses is followed by a restarting process that reactivates only the moving effector. When this form of inhibition takes place, the reaction time (RT) of the moving effector pays the cost of the previous global inhibition. However, it is poorly investigated if and how this cost delays the RT of the effector that was required to be stopped but was erroneously moved (Stop Error trials). Here we measure the Stop Error RT in a group of participants instructed to simultaneously rotate the wrist and lift the foot when a Go Signal occurred, and interrupt both movements (non-selective Stop version) or only one of them (selective Stop version) when a Stop Signal was presented. We presented this task in two experimental conditions to evaluate how different contexts can influence a possible proactive inhibition on the RT of the moving effector in the selective Stop versions. In one context, we provided the foreknowledge of the effector to be inhibited by presenting the same selective or non-selective Stop versions in the same block of trials. In a different context, while providing no foreknowledge of the effector(s) to be stopped, the selective and non-selective Stop versions were intermingled, and the information on the effector to be stopped was delivered at the time of the Stop Signal presentation. We detected a cost in both Correct and Error selective Stop RTs that was influenced by the different task conditions. Results are discussed within the framework of the race model related to the SST, and its relationship with a restart model developed for selective versions of this paradigm.

Controlling an effector with eye movements: The effect of entangled sensory and motor responsibilities

Restoring arm and hand function has been indicated by individuals with tetraplegia as one of the most important factors for regaining independence. The overall goal of our research is to develop assistive technologies that allow individuals with tetraplegia to control functional reaching movements. This study served as an initial step toward our overall goal by assessing the feasibility of using eye movements to control the motion of an effector in an experimental environment. We aimed to understand how additional motor requirements placed on the eyes affected eye-hand coordination during functional reaching. We were particularly interested in how eye fixation error was affected when the sensory and motor functions of the eyes were entangled due to the additional motor responsibility. We recorded participants’ eye and hand movements while they reached for targets on a monitor. We presented a cursor at the participant’s point of gaze position which can be thought of as being similar to the control of an assistive robot arm. To measure eye fixation error, we used an offline filter to extract eye fixations from the raw eye movement data. We compared the fixations to the locations of the targets presented on the monitor. The results show that not only are humans able to use eye movements to direct the cursor to a desired location (1.04 ± 0.15 cm), but they can do so with error similar to that of the hand (0.84 ± 0.05 cm). In other words, despite the additional motor responsibility placed on the eyes during direct eye-movement control of an effector, the ability to coordinate functional reaching movements was unaffected. The outcomes of this study support the efficacy of using the eyes as a direct command input for controlling movement.

Preparatory activity links the frontal eye field response with small amplitude motor unit recruitment of neck muscles during gaze planning

A hallmark of intelligent behavior is that we can separate intention from action. To understand the mechanism that gates the flow of information between motor planning and execution, we compared the activity of frontal eye field neurons with motor unit activity from neck muscles in the presence of an intervening delay period in which spatial information regarding the target was available to plan a response. Although spatially specific delay period activity was present in the activity of frontal eye field neurons, it was absent in motor unit activity. Nonetheless, motor unit activity was correlated with the time it took to initiate saccades. Interestingly, we observed a heterogeneity of responses among motor units, such that only units with smaller amplitudes showed a clear modulation during the delay period. These small amplitude motor units also had higher spontaneous activity compared with the units which showed modulation only during the movement epoch. Taken together, our results suggest the activity of smaller motor units convey temporal information and explains how the delay period primes muscle activity leading to faster reaction times.NEW & NOTEWORTHY This study shows that the temporal aspects of a motor plan in the oculomotor circuitry can be accessed by peripheral neck muscles hundreds of milliseconds before the instruction to initiate a saccadic eye movement. The coupling between central and peripheral processes during the delay time is mediated by the recruitment pattern of motor units with smaller amplitude. These findings suggest that information processed in cortical areas could be read from periphery before execution.

Computational Mechanisms Mediating Inhibitory Control of Coordinated Eye-Hand Movements

Significant progress has been made in understanding the computational and neural mechanisms that mediate eye and hand movements made in isolation. However, less is known about the mechanisms that control these movements when they are coordinated. Here, we outline our computational approaches using accumulation-to-threshold and race-to-threshold models to elucidate the mechanisms that initiate and inhibit these movements. We suggest that, depending on the behavioral context, the initiation and inhibition of coordinated eye-hand movements can operate in two modes-coupled and decoupled. The coupled mode operates when the task context requires a tight coupling between the effectors; a common command initiates both effectors, and a unitary inhibitory process is responsible for stopping them. Conversely, the decoupled mode operates when the task context demands weaker coupling between the effectors; separate commands initiate the eye and hand, and separate inhibitory processes are responsible for stopping them. We hypothesize that the higher-order control processes assess the behavioral context and choose the most appropriate mode. This computational mechanism can explain the heterogeneous results observed across many studies that have investigated the control of coordinated eye-hand movements and may also serve as a general framework to understand the control of complex multi-effector movements.

Spatiotemporal Coupling between Eye and Hand Trajectories during Curved Hand Movements

Eye and hand movements are often made in isolation but for reaching movements they are usually coupled. Despite this, evidence for spatial coupling between the eye and hand effector is mixed and have usually been restricted to straight-line movements, while real-world hand movements have complex trajectories. Here, using a novel obstacle avoidance task where an obstacle appeared in an infrequent number of trials, we establish a stronger link between the saccade and hand trajectory during more naturalistic curved hand trajectories. We illustrate that the hand trajectory was coupled to the end-point of the saccade which was executed just prior to the hand movement onset. Interestingly, while the saccade end-point was related to whether the hand trajectory followed a straight or a curved path, the y-component of saccade end-point was related to whether the hand took a path passing from over or below the obstacle. Further, we observed a relationship between saccade locations and hand sub-movements where the number and timing of saccades and number of hand velocity peaks were related. These results illustrate a robust spatiotemporal and kinematic coupling between saccades and complex hand movement trajectories suggesting a shared kinematic representation underlying eye-hand movements.

Task context determines whether common or separate inhibitory signals underlie the control of eye-hand movements

Whereas inhibitory control of single effector movements has been widely studied, the control of coordinated eye-hand movements has received less attention. Nevertheless, previous studies have contradictorily suggested that either a common or separate signal/s is/are responsible for inhibition of coordinated eye-hand movements. In continuation of our previous study, we varied behavioral contexts and used a stochastic accumulation-to-threshold model, which predicts a scaling of the mean reaction time distribution with its variance, to study the inhibitory control of eye-hand movements. Participants performed eye-hand movements in different task conditions, and in each condition they had to redirect movements in a fraction of trials. Task contexts where the behavior could be best explained by a common initiation signal had similar error responses for eye and hand, despite having different mean reaction times, indicating a common inhibitory signal. In contrast, behavior that could be best explained by separate initiation signals had dissimilar error responses for eye and hand indicating separate inhibitory signals. These behavioral responses were further validated using electromyography and computational models having either a common or separate inhibitory control signal/s. Interestingly, in a particular context, whereas in majority trials a common initiation and inhibitory signal could explain the behavior, in a subset of trials separate initiation and inhibitory signals predicted the behavior better. This highlights the flexibility that exists in the brain and in effect reconciles the heterogeneous results reported by previous studies.

NEW & NOTEWORTHY Prior studies have contradictorily suggested either a single or separate inhibitory signal/s underlying inhibition of coordinated eye-hand movements. With the use of different tasks, we observed that when eye-hand movements were initiated by a common signal, they were controlled by a common inhibitory signal. However, when the two effectors were initiated by separate signals, they were controlled by separate inhibitory signals. This highlights the flexible control of eye-hand movements and reconciles the heterogeneous results previously reported in the literature.

Contrasting speed-accuracy tradeoffs for eye and hand movements reveal the optimal nature of saccade kinematics

In contrast to hand movements, the existence of a neural representation of saccade kinematics is unclear. Saccade kinematics is typically thought to be specified by motor error/desired displacement and generated by brain stem circuits that are not penetrable to voluntary control. We studied the influence of instructed hand movement velocity on the kinematics of saccades executed without explicit instructions. When the hand movement was slow the saccade velocity decreased, independent of saccade amplitude. We leveraged this modulation of saccade velocity to study the optimality of saccades (in terms of velocity and endpoint accuracy) in relation to the well-known speed-accuracy tradeoff that governs voluntary movements (Fitts' law). In contrast to hand movements that obeyed Fitts' law, normometric saccades exhibited the greatest endpoint accuracy and lower reaction times, relative to saccades accompanying slow and fast hand movements. In the slow condition, where saccade endpoint accuracy suffered, we observed that targets were more likely to be foveated by two saccades resulting in step-saccades. Interestingly, the endpoint accuracy was higher in two-saccade trials, compared with one-saccade trials in both the slow and fast conditions. This indicates that step-saccades are a part of the kinematic plan for optimal control of endpoint accuracy. Taken together, these findings suggest normometric saccades are already optimized to maximize endpoint accuracy and the modulation of saccade velocity by hand velocity is likely to reflect the sharing of kinematic plans between the two effectors.NEW & NOTEWORTHY The optimality of saccade kinematics has been suggested by modeling studies but experimental evidence is lacking. However, we observed that, when subjects voluntarily modulated their hand velocity, the velocity of saccades accompanying these hand movements was also modulated, suggesting a shared kinematic plan for eye and hand movements. We leveraged this modulation to show that saccades had less endpoint accuracy when their velocity decreased, illustrating that normometric saccades have optimal speed and accuracy.

Cortical control and performance monitoring of interrupting and redirecting movements

Voluntary behaviour requires control mechanisms that ensure our ability to act independently of habitual and innate response tendencies. Electrophysiological experiments, using the stop-signal task in humans, monkeys and rats, have uncovered a core network of brain structures that is essential for response inhibition. This network is shared across mammals and seems to be conserved throughout their evolution. Recently, new research building on these earlier findings has started to investigate the interaction between response inhibition and other control mechanisms in the brain. Here we describe recent progress in three different areas: selectivity of movement inhibition across different motor systems, re-orientation of motor actions and action evaluation.This article is part of the themed issue 'Movement suppression: brain mechanisms for stopping and stillness'.

Eye and head movements are complementary in visual selection

In the natural environment, visual selection is accomplished by a system of nested effectors, moving the head and body within space and the eyes within the visual field. However, it is not yet known if the principles of selection for these different effectors are the same or different. We used a novel gaze-contingent display in which an asymmetric window of visibility (a horizontal or vertical slot) was yoked to either head or eye position. Participants showed highly systematic changes in behaviour, revealing clear differences in the principles underlying selection by eye and head. Eye movements were more likely to move in the direction of visible information-horizontally when viewing with a horizontal slot, and vertically with a vertical slot. Head movements showed the opposite and complementary pattern, moving to reveal new information (e.g. vertically with a horizontal slot and vice versa). These results are consistent with a nested system in which the head favours exploration of unknown regions, while the eye exploits what can be seen with finer-scale saccades.

Evidence of common and separate eye and hand accumulators underlying flexible eye-hand coordination

Eye and hand movements are initiated by anatomically separate regions in the brain, and yet these movements can be flexibly coupled and decoupled, depending on the need. The computational architecture that enables this flexible coupling of independent effectors is not understood. Here, we studied the computational architecture that enables flexible eye-hand coordination using a drift diffusion framework, which predicts that the variability of the reaction time (RT) distribution scales with its mean. We show that a common stochastic accumulator to threshold, followed by a noisy effector-dependent delay, explains eye-hand RT distributions and their correlation in a visual search task that required decision-making, while an interactive eye and hand accumulator model did not. In contrast, in an eye-hand dual task, an interactive model better predicted the observed correlations and RT distributions than a common accumulator model. Notably, these two models could only be distinguished on the basis of the variability and not the means of the predicted RT distributions. Additionally, signatures of separate initiation signals were also observed in a small fraction of trials in the visual search task, implying that these distinct computational architectures were not a manifestation of the task design per se. Taken together, our results suggest two unique computational architectures for eye-hand coordination, with task context biasing the brain toward instantiating one of the two architectures.

NEW & NOTEWORTHY

Previous studies on eye-hand coordination have considered mainly the means of eye and hand reaction time (RT) distributions. Here, we leverage the approximately linear relationship between the mean and standard deviation of RT distributions, as predicted by the drift-diffusion model, to propose the existence of two distinct computational architectures underlying coordinated eye-hand movements. These architectures, for the first time, provide a computational basis for the flexible coupling between eye and hand movements.

A common control signal and a ballistic stage can explain the control of coordinated eye-hand movements

Voluntary control has been extensively studied in the context of eye and hand movements made in isolation, yet little is known about the nature of control during eye-hand coordination. We probed this with a redirect task. Here subjects had to make reaching/pointing movements accompanied by coordinated eye movements but had to change their plans when the target occasionally changed its position during some trials. Using a race model framework, we found that separate effector-specific mechanisms may be recruited to control eye and hand movements when executed in isolation but when the same effectors are coordinated a unitary mechanism to control coordinated eye-hand movements is employed. Specifically, we found that performance curves were distinct for the eye and hand when these movements were executed in isolation but were comparable when they were executed together. Second, the time to switch motor plans, called the target step reaction time, was different in the eye-alone and hand-alone conditions but was similar in the coordinated condition under assumption of a ballistic stage of ∼40 ms, on average. Interestingly, the existence of this ballistic stage could predict the extent of eye-hand dissociations seen in individual subjects. Finally, when subjects were explicitly instructed to control specifically a single effector (eye or hand), redirecting one effector had a strong effect on the performance of the other effector. Taken together, these results suggest that a common control signal and a ballistic stage are recruited when coordinated eye-hand movement plans require alteration.

Eye-hand coordination during a double-step task: evidence for a common stochastic accumulator

Many studies of reaching and pointing have shown significant spatial and temporal correlations between eye and hand movements. Nevertheless, it remains unclear whether these correlations are incidental, arising from common inputs (independent model); whether these correlations represent an interaction between otherwise independent eye and hand systems (interactive model); or whether these correlations arise from a single dedicated eye-hand system (common command model). Subjects were instructed to redirect gaze and pointing movements in a double-step task in an attempt to decouple eye-hand movements and causally distinguish between the three architectures. We used a drift-diffusion framework in the context of a race model, which has been previously used to explain redirect behavior for eye and hand movements separately, to predict the pattern of eye-hand decoupling. We found that the common command architecture could best explain the observed frequency of different eye and hand response patterns to the target step. A common stochastic accumulator for eye-hand coordination also predicts comparable variances, despite significant difference in the means of the eye and hand reaction time (RT) distributions, which we tested. Consistent with this prediction, we observed that the variances of the eye and hand RTs were similar, despite much larger hand RTs (∼90 ms). Moreover, changes in mean eye RTs, which also increased eye RT variance, produced a similar increase in mean and variance of the associated hand RT. Taken together, these data suggest that a dedicated circuit underlies coordinated eye-hand planning.