Saccadic eye movements in the dark while performing an automatized sequential high-speed sensorimotor task

Warning signals only support the first action in a sequence

Acting upon target stimuli from the environment becomes faster when the targets are preceded by a warning (alerting) cue. Accordingly, alerting is often used to support action in safety-critical contexts (e.g., honking to alert others of a traffic situation). Crucially, however, the benefits of alerting for action have been established using laboratory tasks assessing only simple choice reactions. Real-world actions are considerably more complex and mainly consist of sensorimotor sequences of several sub-actions. Therefore, it is still unknown if the benefits of alerting for action transfer from simple choice reactions to such sensorimotor sequences. Here, we investigated how alerting affected performance in a sequential action task derived from the Trail-Making-Test, a well-established neuropsychological test of cognitive action control (Experiment 1). In addition to this task, participants performed a classic alerting paradigm including a simple choice reaction task (Experiment 2). Results showed that alerting sped up responding in both tasks, but in the sequential action task, this benefit was restricted to the first action of a sequence. This was the case, even when multiple actions were performed within a short time (Experiment 3), ruling out that the restriction of alerting to the first action was due to its short-lived nature. Taken together, these findings reveal the existence of an interface between phasic alertness and action control that supports the next action.

Looking ahead in working memory to guide sequential behaviour

Working memory can maintain multiple sensory representations to serve unfolding sequential behaviour, such as while making tea or planning a route. How the human mind juggles internal representations as they become relevant to guide sequential behaviour remains poorly understood. Specifically, while there is good evidence that we can flexibly switch priorities among representations in working memory^1-4, it is unclear how and when dormant memory representations are brought into focus during sequential behaviour. Capitalising on a recently established and temporally precise gaze marker of internal selection^5,6, we reveal that the focus in the mind moves to the next-relevant memory representation while behaviour associated with the presently relevant memory representation is still ongoing. Thus, like visual sampling of external objects in the world^7-9, internal visual sampling also 'looks ahead' to the next object in memory during sequential behaviour.

The function of "looking-at-nothing" for sequential sensorimotor tasks: Eye movements to remembered action-target locations

When performing manual actions, eye movements precede hand movements to target locations: Before we grasp an object, we look at it. Eye-hand guidance is even preserved when visual targets are unavailable, e.g., grasping behind an occlusion. This "looking-atnothing" behavior might be functional, e.g., as "deictic pointer" for manual control or as memory-retrieval cue, or a by-product of automatization. Here, it is studied if looking at empty locations before acting on them is beneficial for sensorimotor performance. In five experiments, participants completed a click sequence on eight visual targets for 0-100 trials while they had either to fixate on the screen center or could move their eyes freely. During 50-100 consecutive trials, participants clicked the same sequence on a blank screen with free or fixed gaze. During both phases, participants looked at target locations when gaze shifts were allowed. With visual targets, target fixations led to faster, more precise clicking, fewer errors, and sparser cursor-paths than central fixation. Without visual information, a tiny free-gaze benefit could sometimes be observed and was rather a memory than a motor-calculation benefit. Interestingly, central fixation during learning forced early explicit encoding causing a strong benefit for acting on remembered targets later, independent of whether eyes moved then.

Parallel learning processes of a visuomotor adaptation task in a changing environment

During the control of reaching movements, a key contribution of the visual system is the localization of relevant environmental targets. In motor adaptation processes, the visual evaluation of effector motor behavior enables learning from errors, which demands continuous visual attentional focus. However, most current adaptation paradigms include static targets; therefore, when a learning situation develops in a highly variable environment and there is a double demand for visual resources (environment and motor performance), the evolution of learning processes is unknown. In order to understand how learning processes evolve in a variable environment, a video game task was designed in which subjects were asked to manage a 60° counterclockwise-rotated cursor to capture descending targets with initially unpredictable trajectories. During the task, the cursor and eye movements were recorded to dissect visuomotor coordination. We observed that the pursuit of the targets conditioned a predominant and continuous visual inspection of the environment instead of the rotated cursor. As learning progressed, subjects exhibited a linear reduction in directional error and selected a motor strategy based on the degree of reward, which improved the performance. These results suggest that when the environment demands high visual attention, error-based and reinforced motor learning processes are implemented simultaneously, thus enabling efficient predictive behavior.

Saccadic inhibition interrupts ongoing oculomotor activity to enable the rapid deployment of alternate movement plans

Diverse psychophysical and neurophysiological results show that oculomotor networks are continuously active, such that plans for making the next eye movement are always ongoing. So, when new visual information arrives unexpectedly, how are those plans affected? At what point can the new information start guiding an eye movement, and how? Here, based on modeling and simulation results, we make two observations that are relevant to these questions. First, we note that many experiments, including those investigating the phenomenon known as "saccadic inhibition", are consistent with the idea that sudden-onset stimuli briefly interrupt the gradual rise in neural activity associated with the preparation of an impending saccade. And second, we show that this stimulus-driven interruption is functionally adaptive, but only if perception is fast. In that case, putting on hold an ongoing saccade plan toward location A allows the oculomotor system to initiate a concurrent, alternative plan toward location B (where a stimulus just appeared), deliberate (briefly) on the priority of each target, and determine which plan should continue. Based on physiological data, we estimate that the advantage of this strategy, relative to one in which any plan once initiated must be completed, is of several tens of milliseconds per saccade.

"Looking-at-nothing" during sequential sensorimotor actions: Long-term memory-based eye scanning of remembered target locations

Before acting humans saccade to a target object to extract relevant visual information. Even when acting on remembered objects, locations previously occupied by relevant objects are fixated during imagery and memory tasks - a phenomenon called "looking-at-nothing". While looking-at-nothing was robustly found in tasks encouraging declarative memory built-up, results are mixed in the case of procedural sensorimotor tasks. Eye-guidance to manual targets in complete darkness was observed in a task practiced for days beforehand, while investigations using only a single session did not find fixations to remembered action targets. Here, it is asked whether looking-at-nothing can be found in a single sensorimotor session and thus independent from sleep consolidation, and how it progresses when visual information is repeatedly unavailable. Eye movements were investigated in a computerized version of the trail making test. Participants clicked on numbered circles in ascending sequence. Fifty trials were performed with the same spatial arrangement of 9 visual targets to enable long-term memory consolidation. During 50 consecutive trials, participants had to click the remembered target sequence on an empty screen. Participants scanned the visual targets and also the empty target locations sequentially with their eyes, however, the latter less precise than the former. Over the course of the memory trials, manual and oculomotor sequential target scanning became more similar to the visual trials. Results argue for robust looking-at-nothing during procedural sensorimotor tasks provided that long-term memory information is sufficient.

Dynamic modulation of illusory and physical target size on separate and coordinated eye and hand movements

In everyday behavior, two of the most common visually guided actions-eye and hand movements-can be performed independently, but are often synergistically coupled. In this study, we examine whether the same visual representation is used for different stages of saccades and pointing, namely movement preparation and execution, and whether this usage is consistent between independent and naturalistic coordinated eye and hand movements. To address these questions, we used the Ponzo illusion to dissociate the perceived and physical sizes of visual targets and measured the effects on movement preparation and execution for independent and coordinated saccades and pointing. During independent movements, we demonstrated that both physically and perceptually larger targets produced faster preparation for both effectors. Furthermore, participants who showed a greater influence of the illusion on saccade preparation also showed a greater influence on pointing preparation, suggesting that a shared mechanism involved in preparation across effectors is influenced by illusions. However, only physical but not perceptual target sizes influenced saccade and pointing execution. When pointing was coordinated with saccades, we observed different dynamics: pointing no longer showed modulation from illusory size, while saccades showed illusion modulation for both preparation and execution. Interestingly, in independent and coordinated movements, the illusion modulated saccade preparation more than pointing preparation, with this effect more pronounced during coordination. These results suggest a shared mechanism, dominated by the eyes, may underlie visually guided action preparation across effectors. Furthermore, the influence of illusions on action may operate within such a mechanism, leading to dynamic interactions between action modalities based on task demands.

Effects of High-Definition Anodal Transcranial Direct Current Stimulation Applied Simultaneously to Both Primary Motor Cortices on Bimanual Sensorimotor Performance

Many daily activities, such as tying one’s shoe laces, opening a jar of jam or performing a free throw in basketball, require the skillful coordinated use of both hands. Even though the non-invasive method of transcranial direct current stimulation (tDCS) has been repeatedly shown to improve unimanual motor performance, little is known about its effects on bimanual motor performance. More knowledge about how tDCS may improve bimanual behavior would be relevant to motor recovery, e.g., in persons with bilateral impairment of hand function. We therefore examined the impact of high-definition anodal tDCS (HD-atDCS) on the performance of a bimanual sequential sensorimotor task. Thirty-two volunteers (age M = 24.25; SD = 2.75; 14 females) participated in this double-blind study and performed sport stacking in six experimental sessions. In sport stacking, 12 specially designed cups must be stacked (stacked up) and dismantled (stacked down) in predefined patterns as fast as possible. During a pretest, posttest and follow-up test, two sport stacking formations (3-6-3 stack and 1-10-1 stack) were performed. Between the pretest and posttest, all participants were trained in sport stacking with concurrent brain stimulation for three consecutive days. The experimental group (STIM-M1) received HD-atDCS over both primary motor cortices (M1), while the control group received a sham stimulation (SHAM). Three-way analysis of variance (ANOVA) revealed a significant main effect of TIME and a significant interaction of TIME × GROUP. No significant effects were found for GROUP, nor for the three-way interaction of TIME × GROUP × FORMATION. Further two-way ANOVAs showed a significant main effect of TIME and a non-significant main effect for GROUP in both sport stacking formations. A significant interaction between TIME × GROUP was found only for the 3-6-3 formation, indicating superior performance gains for the experimental group (STIM-M1). To account and control for baseline influences on the outcome measurements, ANCOVAs treating pretest scores as covariates revealed a significant effect of the stimulation. From this, we conclude that bilateral HD-atDCS over both M1 improves motor performance in a bimanual sequential sensorimotor task. These results may indicate a beneficial use of tDCS for learning and recovery of bimanual motor skills.

Task-Irrelevant Expectation Violations in Sequential Manual Actions: Evidence for a "Check-after-Surprise" Mode of Visual Attention and Eye-Hand Decoupling

When performing sequential manual actions (e.g., cooking), visual information is prioritized according to the task determining where and when to attend, look, and act. In well-practiced sequential actions, long-term memory (LTM)-based expectations specify which action targets might be found where and when. We have previously demonstrated (Foerster and Schneider, 2015b) that violations of such expectations that are task-relevant (e.g., target location change) cause a regression from a memory-based mode of attentional selection to visual search. How might task-irrelevant expectation violations in such well-practiced sequential manual actions modify attentional selection? This question was investigated by a computerized version of the number-connection test. Participants clicked on nine spatially distributed numbered target circles in ascending order while eye movements were recorded as proxy for covert attention. Target’s visual features and locations stayed constant for 65 prechange-trials, allowing practicing the manual action sequence. Consecutively, a task-irrelevant expectation violation occurred and stayed for 20 change-trials. Specifically, action target number 4 appeared in a different font. In 15 reversion-trials, number 4 returned to the original font. During the first task-irrelevant change trial, manual clicking was slower and eye scanpaths were larger and contained more fixations. The additional fixations were mainly checking fixations on the changed target while acting on later targets. Whereas the eyes repeatedly revisited the task-irrelevant change, cursor-paths remained completely unaffected. Effects lasted for 2–3 change trials and did not reappear during reversion. In conclusion, an unexpected task-irrelevant change on a task-defining feature of a well-practiced manual sequence leads to eye-hand decoupling and a “check-after-surprise” mode of attentional selection.

Using space to represent categories: insights from gaze position

We investigated the boundaries among imagery, memory, and perception by measuring gaze during retrieved versus imagined visual information. Eye fixations during recall were bound to the location at which a specific stimulus was encoded. However, eye position information generalized to novel objects of the same category that had not been seen before. For example, encoding an image of a dog in a specific location enhanced the likelihood of looking at the same location during subsequent mental imagery of other mammals. The results suggest that eye movements can also be launched by abstract representations of categories and not exclusively by a single episode or a specific visual exemplar.

Long-term memory-based control of attention in multi-step tasks requires working memory: evidence from domain-specific interference

Evidence for long-term memory (LTM)-based control of attention has been found during the execution of highly practiced multi-step tasks. However, does LTM directly control for attention or are working memory (WM) processes involved? In the present study, this question was investigated with a dual-task paradigm. Participants executed either a highly practiced visuospatial sensorimotor task (speed stacking) or a verbal task (high-speed poem reciting), while maintaining visuospatial or verbal information in WM. Results revealed unidirectional and domain-specific interference. Neither speed stacking nor high-speed poem reciting was influenced by WM retention. Stacking disrupted the retention of visuospatial locations, but did not modify memory performance of verbal material (letters). Reciting reduced the retention of verbal material substantially whereas it affected the memory performance of visuospatial locations to a smaller degree. We suggest that the selection of task-relevant information from LTM for the execution of overlearned multi-step tasks recruits domain-specific WM.

Functional connectivity patterns of medial and lateral macaque frontal eye fields reveal distinct visuomotor networks

It has been previously shown that small- and large-amplitude saccades have different functions during vision in natural environments. Large saccades are associated with reaching movements toward objects, whereas small saccades facilitate the identification of more detailed object features necessary for successful grasping and manual manipulation. To determine whether these represent dichotomous processing streams, we used resting-state functional MRI to examine the functional connectivity patterns of the medial and lateral frontal eye field (FEF) regions that encode large- and small-amplitude saccades, respectively. We found that the spontaneous blood oxygen level-dependent signals of the medial FEF were functionally correlated with areas known to be involved in reaching movements and executive control processes, whereas lateral FEF was functionally correlated with cortical areas involved in object processing and in grasping, fixation, and manipulation of objects. The results provide strong evidence for two distinct visuomotor network systems in the primate brain that likely reflect the alternating phases of vision for action in natural environments.