Electrophysiological recordings in humans reveal reduced location-specific attentional-shift activity prior to recentering saccades

Refixation behavior in naturalistic viewing: Methods, mechanisms, and neural correlates

When freely viewing a scene, the eyes often return to previously visited locations. By tracking eye movements and coregistering eye movements and EEG, such refixations are shown to have multiple roles: repairing insufficient encoding from precursor fixations, supporting ongoing viewing by resampling relevant locations prioritized by precursor fixations, and aiding the construction of memory representations. All these functions of refixation behavior are understood to be underpinned by three oculomotor and cognitive systems and their associated brain structures. First, immediate saccade planning prior to refixations involves attentional selection of candidate locations to revisit. This process is likely supported by the dorsal attentional network. Second, visual working memory, involved in maintaining task-related information, is likely supported by the visual cortex. Third, higher-order relevance of scene locations, which depends on general knowledge and understanding of scene meaning, is likely supported by the hippocampal memory system. Working together, these structures bring about viewing behavior that balances exploring previously unvisited areas of a scene with exploiting visited areas through refixations.

Is covert attention necessary for programming accurate saccades? Evidence from saccade-locked event-related potentials

For decades, researchers have assumed that shifts of covert attention mandatorily occur prior to eye movements to improve perceptual processing of objects before they are fixated. However, recent research suggests that the N2pc component-a neural measure of covert attentional allocation-does not always precede eye movements. The current study investigated whether the N2pc component mandatorily precedes eye movements and assessed its role in the accuracy of gaze control. In three experiments, participants searched for a letter of a specific color (e.g., red) and directed gaze to it as a response. Electroencephalograms and eye movements were coregistered to determine whether neural markers of covert attention preceded the initial shift of gaze. The results showed that the presaccadic N2pc only occurred under limited conditions: when there were many potential target locations and distractors. Crucially, there was no evidence that the presence or magnitude of the presaccadic N2pc was associated with improved eye movement accuracy in any of the experiments. Interestingly, ERP decoding analyses were able to classify the target location well before the eyes started to move, which likely reflects a presaccadic cognitive process that is distinct from the attentional process measured by the N2pc. Ultimately, we conclude that the covert attentional mechanism indexed by the N2pc is not necessary for precise gaze control.

Early visual modulation and selection predict saccadic timing during visual search: An ERP study

Saccadic eye movements, a critical aspect of real-world visual behavior, are preceded by an initial accumulation of visual information followed by the selection of a single location to move one's eyes. However, it is currently unclear how each of these stages uniquely affects saccadic timing. In this study, participants searched for a contour integration target while EEG was used to measure posterior cortical activity between search display onset and first saccade initiation. The goal was to determine whether saccade timing could be attributed to differences in early ERP amplitudes, with the P1 reflecting the magnitude of early perceptual information accumulation and the N1 reflecting the strength of selection leading to the saccadic decision. EOG was used to measure saccade timing, and trials were divided into fast, middle, and slow bins. The N1 response was smallest in the slow saccade tertile, relative to both the fast and middle tertiles, suggesting weak selection. In contrast, the P1 response was largest for this same slow saccadic tertile relative to the middle saccadic tertile, suggesting vigorous information accumulation. Therefore, delays in saccadic behavior may occur when the visual system is overwhelmed with visual input, thus increasing the time to reach a saccadic decision. These findings reconcile models of eye movement behavior which often prioritize either the impact of information accrual or selection, rather than regarding both as an integrated whole.

Physiological oculo-auricular-facial-mandibular synkinesis elicited in humans by gaze deviations

Integrated motor behaviors involving ocular motion-associated movements of the head, neck, pinna, and parts of the face are commonly seen in animals orienting to a visual target. A number of coordinated movements have also been observed in humans making rapid gaze shifts to horizontal extremes, which may be vestiges of these. Since such integrated mechanisms point to a non-pathological co-activation of several anatomically separate cranial circuits in humans, it is important to see how the different pairs of integrative motor behaviors with a common trigger (i.e., ocular motion) manifest in relation to one another. Here, we systematically examined the pattern of eye movement-induced recruitment of multiple cranial muscles in humans. Simultaneous video-oculography and bilateral surface electromyograms of transverse auricular, temporalis, frontalis, and masseter muscles were recorded in 15 healthy subjects (8 females; 29.3±5.2 years) while they made head-fixed, horizontal saccadic, pursuit and optokinetic eye movements. Potential chin laterotrusion linked to contractions of masticator muscles was captured with a yaw-fixed accelerometer. Our findings objectively show an orchestrated aural-facial-masticatory muscle response to a range of horizontal eye movements (prevalence of 21-93%). These responses were most prominent during eccentric saccades. We further reveal distinctions between the various observed activation patterns in terms of their profile (transient or sustained), laterality (with respect to direction of gaze) and timing (with respect to saccade onset). Possible underlying neural substrates, their atavistic behavioral significance, and potential clinical applications for monitoring sensory attention and designing attention-directed hearing aids in the future are discussed.

Eye movements are not mandatorily preceded by the N2pc component

Researchers typically distinguish between two mechanisms of attentional selection in vision: overt and covert attention. A commonplace assumption is that overt eye movements are automatically preceded by shifts of covert attention during visual search. Although the N2pc component is a putative index of covert attentional orienting, little is currently known about its relationship with overt eye movements. This is because most previous studies of the N2pc component prohibit overt eye movements. The current study assessed this relationship by concurrently measuring covert attention (via the N2pc) and overt eye movements (via eye tracking). Participants searched displays for a lateralized target stimulus and were allowed to generate overt eye movements during the search. We then assessed whether overt eye movements were preceded by the N2pc component. The results indicated that saccades were preceded by an N2pc component, but only when participants were required to carefully inspect the target stimulus before initiating the eye movement. When participants were allowed to make naturalistic eye movements in service of visual search, there was no evidence of an N2pc component before eye movements. These findings suggest that the N2pc component does not always precede overt eye movements during visual search. Implications for understanding the relationship between covert and overt attention are discussed.

Neural correlates of task-related refixation behavior

Eye movement research has shown that attention shifts from the currently fixated location to the next before a saccade is executed. We investigated whether the cost of the attention shift depends on higher-order processing at the time of fixation, in particular on visual working memory load differences between fixations and refixations on task-relevant items. The attention shift is reflected in EEG activity in the saccade-related potential (SRP). In a free viewing task involving visual search and memorization of multiple targets amongst distractors, we compared the SRP in first fixations versus refixations on targets and distractors. The task-relevance of targets implies that more information will be loaded in memory (e.g. both identity and location) than for distractors (e.g. location only). First fixations will involve greater memory load than refixations, since first fixations involve loading of new items, while refixations involve rehearsal of previously visited items. The SRP in the interval preceding the saccade away from a target or distractor revealed that saccade preparation is affected by task-relevance and refixation behavior. For task-relevant items only, we found longer fixation duration and higher SRP amplitudes for first fixations than for refixations over the occipital region and the opposite effect over the frontal region. Our findings provide first neurophysiological evidence that working memory loading of task-relevant information at fixation affects saccade planning.

Asymmetries of the visual system and their influence on visual performance and oculomotor dynamics

Our representation of the visual field is not homogenous. There are differences in resolution not only between the fovea and regions eccentric to it, but also between the nasal and temporal hemiretinae, that can be traced to asymmetric distributions of photoreceptors and ganglion cells. We review evidence for differences in visual and attentional processing and oculomotor behaviour that can be traced to asymmetries of the visual system, mainly emphasising nasal-temporal asymmetries. Asymmetries in the visual system manifest in various measures, in basic psychophysical tests of visual performance, attentional processing, choice behaviour, saccadic peak velocity, and latencies. Nasal-temporal asymmetries on saccadic latency seem primarily to occur for express saccades. Neural asymmetries between the upper and lower hemifields are strong and cause corresponding differences in performance between the hemifields. There are interesting individual differences in asymmetric processing which seem to be related to the strength of eye dominance. These neurophysiological asymmetries and the corresponding asymmetries in visual performance and oculomotor behaviour can strongly influence experimental results in vision and must be considered during experimental design and the interpretation of results.

Refixation control in free viewing: a specialized mechanism divulged by eye-movement-related brain activity

In free viewing, the eyes return to previously visited locations rather frequently, even though the attentional and memory-related processes controlling eye-movement show a strong antirefixation bias. To overcome this bias, a special refixation triggering mechanism may have to be recruited. We probed the neural evidence for such a mechanism by combining eye tracking with EEG recording. A distinctive signal associated with refixation planning was observed in the EEG during the presaccadic interval: the presaccadic potential was reduced in amplitude before a refixation compared with normal fixations. The result offers direct evidence for a special refixation mechanism that operates in the saccade planning stage of eye movement control. Once the eyes have landed on the revisited location, acquisition of visual information proceeds indistinguishably from ordinary fixations. NEW & NOTEWORTHY A substantial proportion of eye fixations in human natural viewing behavior are revisits of recently visited locations, i.e., refixations. Our recently developed methods enabled us to study refixations in a free viewing visual search task, using combined eye movement and EEG recording. We identified in the EEG a distinctive refixation-related signal, signifying a control mechanism specific to refixations as opposed to ordinary eye fixations.

Presaccadic EEG activity predicts visual saliency in free-viewing contour integration

While viewing a scene, the eyes are attracted to salient stimuli. We set out to identify the brain signals controlling this process. In a contour integration task, in which participants searched for a collinear contour in a field of randomly oriented Gabor elements, a previously established model was applied to calculate a visual saliency value for each fixation location. We studied brain activity related to the modeled saliency values, using coregistered eye tracking and EEG. To disentangle EEG signals reflecting salience in free viewing from overlapping EEG responses to sequential eye movements, we adopted generalized additive mixed modeling (GAMM) to single epochs of saccade-related EEG. We found that, when saliency at the next fixation location was high, amplitude of the presaccadic EEG activity was low. Since presaccadic activity reflects covert attention to the saccade target, our results indicate that larger attentional effort is needed for selecting less salient saccade targets than more salient ones. This effect was prominent in contour-present conditions (half of the trials), but ambiguous in the contour-absent condition. Presaccadic EEG activity may thus be indicative of bottom-up factors in saccade guidance. The results underscore the utility of GAMM for EEG-eye movement coregistration research.

EEG Negativity in Fixations Used for Gaze-Based Control: Toward Converting Intentions into Actions with an Eye-Brain-Computer Interface

We usually look at an object when we are going to manipulate it. Thus, eye tracking can be used to communicate intended actions. An effective human-machine interface, however, should be able to differentiate intentional and spontaneous eye movements. We report an electroencephalogram (EEG) marker that differentiates gaze fixations used for control from spontaneous fixations involved in visual exploration. Eight healthy participants played a game with their eye movements only. Their gaze-synchronized EEG data (fixation-related potentials, FRPs) were collected during game's control-on and control-off conditions. A slow negative wave with a maximum in the parietooccipital region was present in each participant's averaged FRPs in the control-on conditions and was absent or had much lower amplitude in the control-off condition. This wave was similar but not identical to stimulus-preceding negativity, a slow negative wave that can be observed during feedback expectation. Classification of intentional vs. spontaneous fixations was based on amplitude features from 13 EEG channels using 300 ms length segments free from electrooculogram contamination (200-500 ms relative to the fixation onset). For the first fixations in the fixation triplets required to make moves in the game, classified against control-off data, a committee of greedy classifiers provided 0.90 ± 0.07 specificity and 0.38 ± 0.14 sensitivity. Similar (slightly lower) results were obtained for the shrinkage Linear Discriminate Analysis (LDA) classifier. The second and third fixations in the triplets were classified at lower rate. We expect that, with improved feature sets and classifiers, a hybrid dwell-based Eye-Brain-Computer Interface (EBCI) can be built using the FRP difference between the intended and spontaneous fixations. If this direction of BCI development will be successful, such a multimodal interface may improve the fluency of interaction and can possibly become the basis for a new input device for paralyzed and healthy users, the EBCI "Wish Mouse."

Visual straight-ahead preference in saccadic eye movements

Ocular saccades bringing the gaze toward the straight-ahead direction (centripetal) exhibit higher dynamics than those steering the gaze away (centrifugal). This is generally explained by oculomotor determinants: centripetal saccades are more efficient because they pull the eyes back toward their primary orbital position. However, visual determinants might also be invoked: elements located straight-ahead trigger saccades more efficiently because they receive a privileged visual processing. Here, we addressed this issue by using both pro- and anti-saccade tasks in order to dissociate the centripetal/centrifugal directions of the saccades, from the straight-ahead/eccentric locations of the visual elements triggering those saccades. Twenty participants underwent alternating blocks of pro- and anti-saccades during which eye movements were recorded binocularly at 1 kHz. The results confirm that centripetal saccades are always executed faster than centrifugal ones, irrespective of whether the visual elements have straight-ahead or eccentric locations. However, by contrast, saccades triggered by elements located straight-ahead are consistently initiated more rapidly than those evoked by eccentric elements, irrespective of their centripetal or centrifugal direction. Importantly, this double dissociation reveals that the higher dynamics of centripetal pro-saccades stem from both oculomotor and visual determinants, which act respectively on the execution and initiation of ocular saccades.

Visual encoding and fixation target selection in free viewing: presaccadic brain potentials

In scrutinizing a scene, the eyes alternate between fixations and saccades. During a fixation, two component processes can be distinguished: visual encoding and selection of the next fixation target. We aimed to distinguish the neural correlates of these processes in the electrical brain activity prior to a saccade onset. Participants viewed color photographs of natural scenes, in preparation for a change detection task. Then, for each participant and each scene we computed an image heat map, with temperature representing the duration and density of fixations. The temperature difference between the start and end points of saccades was taken as a measure of the expected task-relevance of the information concentrated in specific regions of a scene. Visual encoding was evaluated according to whether subsequent change was correctly detected. Saccades with larger temperature difference were more likely to be followed by correct detection than ones with smaller temperature differences. The amplitude of presaccadic activity over anterior brain areas was larger for correct detection than for detection failure. This difference was observed for short "scrutinizing" but not for long "explorative" saccades, suggesting that presaccadic activity reflects top-down saccade guidance. Thus, successful encoding requires local scanning of scene regions which are expected to be task-relevant. Next, we evaluated fixation target selection. Saccades "moving up" in temperature were preceded by presaccadic activity of higher amplitude than those "moving down". This finding suggests that presaccadic activity reflects attention deployed to the following fixation location. Our findings illustrate how presaccadic activity can elucidate concurrent brain processes related to the immediate goal of planning the next saccade and the larger-scale goal of constructing a robust representation of the visual scene.