Human microsaccade-related visual brain responses

Understanding the role of eye movement pattern and consistency during face recognition through EEG decoding

Eye movement patterns and consistency during face recognition are both associated with recognition performance. We examined whether they reflect different mechanisms through EEG decoding. Eighty-four participants performed an old-new face recognition task with eye movement pattern and consistency quantified using eye movement analysis with hidden Markov models (EMHMM). Temporal dynamics of neural representation quality for face recognition were assessed through decoding old vs new faces using a support vector machine classifier. Results showed that a more eye-focused pattern was associated with higher decoding accuracy in the high-alpha band, reflecting better neural representation quality. In contrast, higher eye movement consistency was associated with shorter latency of peak decoding accuracy in the high-alpha band, which suggested more efficient neural representation development, in addition to higher ERP decoding accuracy. Thus, eye movement patterns are associated with neural representation effectiveness, whereas eye movement consistency reflects neural representation development efficiency, unraveling different aspects of cognitive processes.

Direct and averted gaze modulate the event-related brain responses to social exclusion signals

Social exclusion is a negative experience in social interaction which can be modulated by the perceived eye gaze. In our Cyberball study, we asked how gaze direction - direct versus averted - affects the processing of social exclusion by using electroencephalogram (EEG) methods. Participants encountered neutral gazes in the inclusion phase (Block1) and then either direct or averted gazes during the exclusion phase (Block2). We measured self-reports on Need-Threat Questionnaires (belonging, control, meaningful existence, self-esteem) and negative mood, along with event-related potentials (ERPs), specifically the P3 component (P300) - an indicator of expectation violation. Findings demonstrated that the P3 amplitude showed differential changes between gaze conditions during the transition from inclusion to exclusion, with direct gaze eliciting stronger neural responses. This difference signals that direct gaze enhances the expected participation, thus sensitizing participants for exclusionary signals. Simultaneously, self-reports indicated that the transition evoked weaker negative emotions in the direct gaze group and stronger negative emotions in the averted gaze group, signalling that direct gaze also serves as an affiliative signal. These results illuminate how gaze directions shape cognitive and emotional reactions to social exclusion. However, further research is necessary to unravel these effects amidst other non-verbal cues and fully comprehend their impact on neural and behavioural responses to social exclusion.

The Representation of Stimulus Features during Stable Fixation and Active Vision

Predictive updating of an object's spatial coordinates from presaccade to postsaccade contributes to stable visual perception. Whether object features are predictively remapped remains contested. We set out to characterize the spatiotemporal dynamics of feature processing during stable fixation and active vision. To do so, we applied multivariate decoding methods to EEG data collected while human participants (male and female) viewed brief visual stimuli. Stimuli appeared at different locations across the visual field at either high or low spatial frequency (SF). During fixation, classifiers were trained to decode SF presented at one parafoveal location and cross-tested on SF from either the same, adjacent, or more peripheral locations. When training and testing on the same location, SF was classified shortly after stimulus onset (∼79 ms). Decoding of SF at locations farther from the trained location emerged later (∼144-295 ms), with decoding latency modulated by eccentricity. This analysis provides a detailed time course for the spread of feature information across the visual field. Next, we investigated how active vision impacts the emergence of SF information. In the presence of a saccade, the decoding time of peripheral SF at parafoveal locations was earlier, indicating predictive anticipation of SF due to the saccade. Crucially, however, this predictive effect was not limited to the specific remapped location. Rather, peripheral SF was correctly classified, at an accelerated time course, at all parafoveal positions. This indicates spatially coarse, predictive anticipation of stimulus features during active vision, likely enabling a smooth transition on saccade landing.

(Micro)saccade-related potentials during face recognition: A study combining EEG, eye-tracking, and deconvolution modeling

Under natural viewing conditions, complex stimuli such as human faces are typically looked at several times in succession, implying that their recognition may unfold across multiple eye fixations. Although electrophysiological (EEG) experiments on face recognition typically prohibit eye movements, participants still execute frequent (micro)saccades on the face, each of which generates its own visuocortical response. This finding raises the question of whether the fixation-related potentials (FRPs) evoked by these tiny gaze shifts also contain psychologically valuable information about face processing. Here, we investigated this question by corecording EEG and eye movements in an experiment with emotional faces (happy, angry, neutral). Deconvolution modeling was used to separate the stimulus ERPs to face onset from the FRPs generated by subsequent microsaccades-induced refixations on the face. As expected, stimulus ERPs exhibited typical emotion effects, with a larger early posterior negativity (EPN) for happy/angry compared with neutral faces. Eye tracking confirmed that participants made small saccades in 98% of the trials, which were often aimed at the left eye of the stimulus face. However, while each saccade produced a strong response over visual areas, this response was unaffected by the face's emotional expression, both for the first and for subsequent (micro)saccades. This finding suggests that the face's affective content is rapidly evaluated after stimulus onset, leading to only a short-lived sensory enhancement by arousing stimuli that does not repeat itself during immediate refixations. Methodologically, our work demonstrates how eye tracking and deconvolution modeling can be used to extract several brain responses from each EEG trial, providing insights into neural processing at different latencies after stimulus onset.

Refixation behavior in naturalistic viewing: Methods, mechanisms, and neural correlates

When freely viewing a scene, the eyes often return to previously visited locations. By tracking eye movements and coregistering eye movements and EEG, such refixations are shown to have multiple roles: repairing insufficient encoding from precursor fixations, supporting ongoing viewing by resampling relevant locations prioritized by precursor fixations, and aiding the construction of memory representations. All these functions of refixation behavior are understood to be underpinned by three oculomotor and cognitive systems and their associated brain structures. First, immediate saccade planning prior to refixations involves attentional selection of candidate locations to revisit. This process is likely supported by the dorsal attentional network. Second, visual working memory, involved in maintaining task-related information, is likely supported by the visual cortex. Third, higher-order relevance of scene locations, which depends on general knowledge and understanding of scene meaning, is likely supported by the hippocampal memory system. Working together, these structures bring about viewing behavior that balances exploring previously unvisited areas of a scene with exploiting visited areas through refixations.

Decoding Remapped Spatial Information in the Peri-Saccadic Period

It has been suggested that, prior to a saccade, visual neurons predictively respond to stimuli that will fall in their receptive fields after completion of the saccade. This saccadic remapping process is thought to compensate for the shift of the visual world across the retina caused by eye movements. To map the timing of this predictive process in the brain, we recorded neural activity using electroencephalography during a saccade task. Human participants (male and female) made saccades between two fixation points while covertly attending to oriented gratings briefly presented at various locations on the screen. Data recorded during trials in which participants maintained fixation were used to train classifiers on stimuli in different positions. Subsequently, data collected during saccade trials were used to test for the presence of remapped stimulus information at the post-saccadic retinotopic location in the peri-saccadic period, providing unique insight into when remapped information becomes available. We found that the stimulus could be decoded at the remapped location ∼180 ms post-stimulus onset, but only when the stimulus was presented 100-200 ms before saccade onset. Within this range, we found that the timing of remapping was dictated by stimulus onset rather than saccade onset. We conclude that presenting the stimulus immediately before the saccade allows for optimal integration of the corollary discharge signal with the incoming peripheral visual information, resulting in a remapping of activation to the relevant post-saccadic retinotopic neurons.

Why behaviour matters: Studying inter-brain coordination during child-caregiver interaction

Modern technology allows for simultaneous neuroimaging from interacting caregiver-child dyads. Whereas most analyses that examine the coordination between brain regions within an individual brain do so by measuring changes relative to observed events, studies that examine coordination between two interacting brains generally do this by measuring average intra-brain coordination across entire blocks or experimental conditions. In other words, they do not examine changes in inter-brain coordination relative to individual behavioural events. Here, we discuss the limitations of this approach. First, we present data suggesting that fine-grained temporal interdependencies in behaviour can leave residual artifact in neuroimaging data. We show how artifact can manifest as both power and (through that) phase synchrony effects in EEG and affect wavelet transform coherence in fNIRS analyses. Second, we discuss different possible mechanistic explanations of how inter-brain coordination is established and maintained. We argue that non-event-locked approaches struggle to differentiate between them. Instead, we contend that approaches which examine how interpersonal dynamics change around behavioural events have better potential for addressing possible artifactual confounds and for teasing apart the overlapping mechanisms that drive changes in inter-brain coordination.

Event-related theta and gamma band oscillatory dynamics during visuo-spatial sequence memory in younger and older adults

Visuo-spatial working memory (VSWM) for sequences is thought to be crucial for daily behaviors. Decades of research indicate that oscillations in the gamma and theta bands play important functional roles in the support of visuo-spatial working memory, but the vast majority of that research emphasizes measures of neural activity during memory retention. The primary aims of the present study were (1) to determine whether oscillatory dynamics in the Theta and Gamma ranges would reflect item-level sequence encoding during a computerized spatial span task, (2) to determine whether item-level sequence recall is also related to these neural oscillations, and (3) to determine the nature of potential changes to these processes in healthy cognitive aging. Results indicate that VSWM sequence encoding is related to later (∼700 ms) gamma band oscillatory dynamics and may be preserved in healthy older adults; high gamma power over midline frontal and posterior sites increased monotonically as items were added to the spatial sequence in both age groups. Item-level oscillatory dynamics during the recall of VSWM sequences were related only to theta-gamma phase amplitude coupling (PAC), which increased monotonically with serial position in both age groups. Results suggest that, despite a general decrease in frontal theta power during VSWM sequence recall in older adults, gamma band dynamics during encoding and theta-gamma PAC during retrieval play unique roles in VSWM and that the processes they reflect may be spared in healthy aging.

Location- and Object-Based Representational Mechanisms Account for Bilateral Field Advantage in Multiple-Object Tracking

Keeping track of multiple visually identical and independently moving objects is a remarkable feature of the human visual system. Theoretical accounts for this ability focus on resource-based models that describe parametric decreases of performance with increasing demands during the task (i.e., more relevant items, closer distances, higher speed). Additionally, the presence of two central tracking resources, one within each hemisphere, has been proposed, allowing for an independent maintenance of moving targets within each visual hemifield. Behavioral evidence in favor of such a model shows that human subjects are able to track almost twice as many targets across both hemifields compared with within one hemifield. A number of recent publications argue for two separate and parallel tracking mechanisms during standard object tracking tasks that allow for the maintenance of the relevant information in a location-based and object-based manner. Unique electrophysiological correlates for each of those processes have been identified. The current study shows that these electrophysiological components are differentially present during tracking within either the left or right hemifield. The present results suggest that targets are mostly maintained as an object-based representation during left hemifield tracking, while location-based resources are preferentially engaged during right hemifield tracking. Interestingly, the manner of representation does not seem to have an impact on behavioral performance within the subjects, while the electrophysiological component indicating object-based tracking does correlate with performance between subjects. We propose that hemifield independence during multiple-object tracking may be an indication of the underlying hemispheric bias for parallel location-based and object-based tracking mechanisms.

Alpha-Band Lateralization and Microsaccades Elicited by Exogenous Cues Do Not Track Attentional Orienting

We explore the world by constantly shifting our focus of attention toward salient stimuli and then disengaging from them in search of new ones. The alpha rhythm (8-13 Hz) has been suggested as a pivotal neural substrate of these attentional shifts, due to its local synchronization and desynchronization that suppress irrelevant cortical areas and facilitate relevant areas, a phenomenon called alpha lateralization. Whether alpha lateralization tracks the focus of attention from orienting toward a salient stimulus to disengaging from it is still an open question. We addressed it by leveraging the phenomenon of inhibition of return (IOR), consisting of an initial facilitation in response times (RTs) for stimuli appearing at an exogenously cued location, followed by a suppression of that location. Our behavioral data from human participants showed a typical IOR effect with both early facilitation and subsequent inhibition. In contrast, alpha lateralized in the cued direction after the behavioral facilitation effect and never re-lateralized compatibly with the behavioral inhibition. Furthermore, we analyzed the interaction between alpha lateralization and microsaccades: while alpha was lateralized toward the cued location, microsaccades were mostly oriented away from it. Crucially, the two phenomena showed a significant positive correlation. These results indicate that alpha lateralization reflects primarily the processing of salient stimuli, challenging the view that alpha lateralization is directly involved in exogenous attentional orienting per se. We discuss the relevance of the present findings for an oculomotor account of alpha lateralization as a modulator of cortical excitability in preparation of a saccade.

Effects of pupil size as manipulated through ipRGC activation on visual processing

The size of the eyes' pupils determines how much light enters the eye and also how well this light is focused. Through this route, pupil size shapes the earliest stages of visual processing. Yet causal effects of pupil size on vision are poorly understood and rarely studied. Here we introduce a new way to manipulate pupil size, which relies on activation of intrinsically photosensitive retinal ganglion cells (ipRGCs) to induce sustained pupil constriction. We report the effects of both experimentally induced and spontaneous changes in pupil size on visual processing as measured through EEG. We compare these to the effects of stimulus intensity and covert visual attention, because previous studies have shown that these factors all have comparable effects on some common measures of early visual processing, such as detection performance and steady-state visual evoked potentials; yet it is still unclear whether these are superficial similarities, or rather whether they reflect similar underlying processes. Using a mix of neural-network decoding, ERP analyses, and time-frequency analyses, we find that induced pupil size, spontaneous pupil size, stimulus intensity, and covert visual attention all affect EEG responses, mainly over occipital and parietal electrodes, but-crucially-that they do so in qualitatively different ways. Induced and spontaneous pupil-size changes mainly modulate activity patterns (but not overall power or intertrial coherence) in the high-frequency beta range; this may reflect an effect of pupil size on oculomotor activity and/ or visual processing. In addition, spontaneous (but not induced) pupil size tends to correlate positively with intertrial coherence in the alpha band; this may reflect a non-causal relationship, mediated by arousal. Taken together, our findings suggest that pupil size has qualitatively different effects on visual processing from stimulus intensity and covert visual attention. This shows that pupil size as manipulated through ipRGC activation strongly affects visual processing, and provides concrete starting points for further study of this important yet understudied earliest stage of visual processing.

Fixation-related electrical potentials during a free visual search task reveal the timing of visual awareness

It has been repeatedly claimed that emotional faces readily capture attention, and that they may be processed without awareness. Yet some observations cast doubt on these assertions. Part of the problem may lie in the experimental paradigms employed. Here, we used a free viewing visual search task during electroencephalographic recordings, where participants searched for either fearful or neutral facial expressions among distractor expressions. Fixation-related potentials were computed for fearful and neutral targets and the response compared for stimuli consciously reported or not. We showed that awareness was associated with an electrophysiological negativity starting at around 110 ms, while emotional expressions were distinguished on the N170 and early posterior negativity only when stimuli were consciously reported. These results suggest that during unconstrained visual search, the earliest electrical correlate of awareness may emerge as early as 110 ms, and fixating at an emotional face without reporting it may not produce any unconscious processing.

Microsaccades transiently lateralise EEG alpha activity

The lateralisation of 8-12 Hz alpha activity is a canonical signature of human spatial cognition that is typically studied under strict fixation requirements. Yet, even during attempted fixation, the brain produces small involuntary eye movements known as microsaccades. Here we report how spontaneous microsaccades - made in the absence of incentives to look elsewhere - can themselves drive transient lateralisation of EEG alpha power according to microsaccade direction. This transient lateralisation of posterior alpha power occurs similarly following start and return microsaccades and is, at least for start microsaccades, driven by increased alpha power ipsilateral to microsaccade direction. This reveals new links between spontaneous microsaccades and human electrophysiological brain activity. It highlights how microsaccades are an important factor to consider in studies relating alpha activity - including spontaneous fluctuations in alpha activity - to spatial cognition, such as studies on visual attention, anticipation, and working memory.

Gaze onsets during naturalistic infant-caregiver interaction associate with 'sender' but not 'receiver' neural responses, and do not lead to changes in inter-brain synchrony

Temporal coordination during infant-caregiver social interaction is thought to be crucial for supporting early language acquisition and cognitive development. Despite a growing prevalence of theories suggesting that increased inter-brain synchrony associates with many key aspects of social interactions such as mutual gaze, little is known about how this arises during development. Here, we investigated the role of mutual gaze onsets as a potential driver of inter-brain synchrony. We extracted dual EEG activity around naturally occurring gaze onsets during infant-caregiver social interactions in N = 55 dyads (mean age 12 months). We differentiated between two types of gaze onset, depending on each partners' role. 'Sender' gaze onsets were defined at a time when either the adult or the infant made a gaze shift towards their partner at a time when their partner was either already looking at them (mutual) or not looking at them (non-mutual). 'Receiver' gaze onsets were defined at a time when their partner made a gaze shift towards them at a time when either the adult or the infant was already looking at their partner (mutual) or not (non-mutual). Contrary to our hypothesis we found that, during a naturalistic interaction, both mutual and non-mutual gaze onsets were associated with changes in the sender, but not the receiver's brain activity and were not associated with increases in inter-brain synchrony above baseline. Further, we found that mutual, compared to non-mutual gaze onsets were not associated with increased inter brain synchrony. Overall, our results suggest that the effects of mutual gaze are strongest at the intra-brain level, in the 'sender' but not the 'receiver' of the mutual gaze.

Neural bases of predictions during natural reading of known statements: An EEG and eye movements co-registration study

Predictions of incoming words performed during reading have an impact on how the reader moves their eyes and on the electrical brain potentials. Eye tracking (ET) experiments show that less predictable words are fixated for longer periods of times. Electroencephalography (EEG) experiments show that these words elicit a more negative potential around 400ms (N400) after the word onset when reading one word at a time (foveated reading). Nevertheless, there was no N400 potential during the foveated reading of previously known sentences (memory-encoded), which suggests that the prediction of words from memory-encoded sentences is based on different mechanisms than predictions performed on common sentences. Here, we performed an ET-EEG co-registration experiment where participants read common and memory-encoded sentences. Our results show that the N400 potential disappear when the reader recognises the sentence. Furthermore, time-frequency analyses show a larger alpha lateralisation and a beta power increase for memory-encoded sentences. This suggests a more distributed attention and an active maintenance of the cognitive set, in concordance to the predictive coding framework.

Motor-effector dependent modulation of sensory-motor processes identified by the multivariate pattern analysis of EEG activity

Sensory information received through sensory organs is constantly modulated by numerous non-sensory factors. Recent studies have demonstrated that the state of action can modulate sensory representations in cortical areas. Similarly, sensory information can be modulated by the type of action used to report perception; however, systematic investigation of this issue is scarce. In this study, we examined whether sensorimotor processes represented in electroencephalography (EEG) activities vary depending on the type of effector behavior. Nineteen participants performed motion direction discrimination tasks in which visual inputs were the same, and only the effector behaviors for reporting perceived motion directions were different (smooth pursuit, saccadic eye movement, or button press). We used multivariate pattern analysis to compare the EEG activities for identical sensory inputs under different effector behaviors. The EEG activity patterns for the identical sensory stimulus before any motor action varied across the effector behavior conditions, and the choice of motor effectors modulated the neural direction discrimination differently. We suggest that the motor-effector dependent modulation of EEG direction discrimination might be caused by effector-specific motor planning or preparation signals because it did not have functional relevance to behavioral direction discriminability.

Episodic memory formation in unrestricted viewing

The brain systems of episodic memory and oculomotor control are tightly linked, suggesting a crucial role of eye movements in memory. But little is known about the neural mechanisms of memory formation across eye movements in unrestricted viewing behavior. Here, we leverage simultaneous eye tracking and EEG recording to examine episodic memory formation in free viewing. Participants memorized multi-element events while their EEG and eye movements were concurrently recorded. Each event comprised elements from three categories (face, object, place), with two exemplars from each category, in different locations on the screen. A subsequent associative memory test assessed participants' memory for the between-category associations that specified each event. We used a deconvolution approach to overcome the problem of overlapping EEG responses to sequential saccades in free viewing. Brain activity was time-locked to the fixation onsets, and we examined EEG power in the theta and alpha frequency bands, the putative oscillatory correlates of episodic encoding mechanisms. Three modulations of fixation-related EEG predicted high subsequent memory performance: (1) theta increase at fixations after between-category gaze transitions, (2) theta and alpha increase at fixations after within-element gaze transitions, (3) alpha decrease at fixations after between-exemplar gaze transitions. Thus, event encoding with unrestricted viewing behavior was characterized by three neural mechanisms, manifested in fixation-locked theta and alpha EEG activity that rapidly turned on and off during the unfolding eye movement sequences. These three distinct neural mechanisms may be the essential building blocks that subserve the buildup of coherent episodic memories during unrestricted viewing behavior.

Multivariate EEG activity reflects the Bayesian integration and the integrated Galilean relative velocity of sensory motion during sensorimotor behavior

Humans integrate multiple sources of information for action-taking, using the reliability of each source to allocate weight to the data. This reliability-weighted information integration is a crucial property of Bayesian inference. In this study, participants were asked to perform a smooth pursuit eye movement task in which we independently manipulated the reliability of pursuit target motion and the direction-of-motion cue. Through an analysis of pursuit initiation and multivariate electroencephalography activity, we found neural and behavioral evidence of Bayesian information integration: more attraction toward the cue direction was generated when the target motion was weak and unreliable. Furthermore, using mathematical modeling, we found that the neural signature of Bayesian information integration had extra-retinal origins, although most of the multivariate electroencephalography activity patterns during pursuit were best correlated with the retinal velocity errors accumulated over time. Our results demonstrated neural implementation of Bayesian inference in human oculomotor behavior.

Face familiarity revealed by fixational eye movements and fixation-related potentials in free viewing

Event-related potentials (ERPs) and the oculomotor inhibition (OMI) in response to visual transients are known to be sensitive to stimulus properties, attention, and expectation. We have recently found that the OMI is also sensitive to face familiarity. In natural vision, stimulation of the visual cortex is generated primarily by saccades, and it has been recently suggested that fixation-related potentials (FRPs) share similar components with the ERPs. Here, we investigated whether FRPs and microsaccade inhibition (OMI) in free viewing are sensitive to face familiarity. Observers freely watched a slideshow of seven unfamiliar and one familiar facial images presented randomly for 4-s periods, with multiple images per identity. We measured the occipital fixation-related N1 relative to the P1 magnitude as well as the associated fixation-triggered OMI. We found that the average N1-P1 was significantly smaller and the OMI was shorter for the familiar face, compared with any of the seven unfamiliar faces. Moreover, the P1 was suppressed across saccades for the familiar but not for the unfamiliar faces. Our results highlight the sensitivity of the occipital FRPs to stimulus properties such as face familiarity and advance our understanding of the integration process across successive saccades in natural vision.

Active vision in sight recovery individuals with a history of long-lasting congenital blindness

What we see is intimately linked to how we actively and systematically explore the world through eye movements. However, it is unknown to what degree visual experience during early development is necessary for such systematic visual exploration to emerge. The present study investigated visual exploration behavior in ten human participants whose sight had been restored only in childhood or adulthood, after a period of congenital blindness due to dense bilateral congenital cataracts. Participants freely explored real-world images while their eye movements were recorded. Despite severe residual visual impairments and gaze instability (nystagmus), visual exploration patterns were preserved in individuals with reversed congenital cataract. Modelling analyses indicated that similar to healthy controls, visual exploration in individuals with reversed congenital cataract was based on the low-level (luminance contrast) and high-level (object components) visual content of the images. Moreover, participants used visual short-term memory representations for narrowing down the exploration space. More systematic visual exploration in individuals with reversed congenital cataract was associated with better object recognition, suggesting that active vision might be a driving force for visual system development and recovery. The present results argue against a sensitive period for the development of neural mechanisms associated with visual exploration.SIGNIFICANCE STATEMENTHumans explore the visual world with systematic patterns of eye movements, but it is unknown whether early visual experience is necessary for the acquisition of visual exploration. Here, we show that sight recovery individuals who had been born blind demonstrate highly systematic eye movements while exploring real-world images, despite visual impairments and pervasive gaze instability. In fact, their eye movement patterns were predicted by those of normally sighted controls and models calculating eye movements based on low- and high-level visual features, and they moreover took memory information into account. Since object recognition performance was associated with systematic visual exploration it was concluded that eye movements might be a driving factor for the development of the visual system.

Functional but not obligatory link between microsaccades and neural modulation by covert spatial attention

Covert spatial attention is associated with spatial modulation of neural activity as well as with directional biases in fixational eye movements known as microsaccades. We studied how these two 'fingerprints' of attention are interrelated in humans. We investigated spatial modulation of 8-12 Hz EEG alpha activity and microsaccades when attention is directed internally within the spatial layout of visual working memory. Consistent with a common origin, spatial modulations of alpha activity and microsaccades co-vary: alpha lateralisation is stronger in trials with microsaccades toward versus away from the memorised location of the to-be-attended item and occurs earlier in trials with earlier microsaccades toward this item. Critically, however, trials without attention-driven microsaccades nevertheless show clear spatial modulation of alpha activity - comparable to trials with attention-driven microsaccades. Thus, directional biases in microsaccades correlate with neural signatures of spatial attention, but they are not necessary for neural modulation by spatial attention to be manifest.

Optic Flow Speed and Retinal Stimulation Influence Microsaccades

Microsaccades are linked with extraretinal mechanisms that significantly alter spatial perception before the onset of eye movements. We sought to investigate whether microsaccadic activity is modulated by the speed of radial optic flow stimuli. Experiments were performed in the dark on 19 subjects who stood in front of a screen covering 135 × 107° of the visual field. Subjects were instructed to fixate on a central fixation point while optic flow stimuli were presented in full field, in the foveal, and in the peripheral visual field at different dot speeds (8, 11, 14, 17, and 20°/s). Fixation in the dark was used as a control stimulus. For almost all tested speeds, the stimulation of the peripheral retina evoked the highest microsaccade rate. We also found combined effects of optic flow speed and the stimulated retinal region (foveal, peripheral, and full field) for microsaccade latency. These results show that optic flow speed modulates microsaccadic activity when presented in specific retinal portions, suggesting that eye movement generation is strictly dependent on the stimulated retinal regions.

EEG Signals Index a Global Signature of Arousal Embedded in Neuronal Population Recordings

Electroencephalography (EEG) has long been used to index brain states, from early studies describing activity in the presence and absence of visual stimulation to modern work employing complex perceptual tasks. These studies have shed light on brain-wide signals but often lack explanatory power at the single neuron level. Similarly, single neuron recordings can suffer from an inability to measure brain-wide signals accessible using EEG. Here, we combined these techniques while monkeys performed a change detection task and discovered a novel link between spontaneous EEG activity and a neural signal embedded in the spiking responses of neuronal populations. This "slow drift" was associated with fluctuations in the subjects' arousal levels over time: decreases in prestimulus α power were accompanied by increases in pupil size and decreases in microsaccade rate. These results show that brain-wide EEG signals can be used to index modes of activity present in single neuron recordings, that in turn reflect global changes in brain state that influence perception and behavior.

Temporal Patterns of Spontaneous Fixational Eye Movements: The Influence of Basal Ganglia

BACKGROUND

Spontaneity is a unique feature of the nervous system. One of the fundamentally critical and recognized forms of spontaneous motor activity is witnessed in the visuomotor system. Microsaccades, the miniature spontaneous eye movements, are critical for the visual perception. We hypothesized that microsaccades follow specific temporal patterns that are modulated by the basal ganglia output.

METHODS

We used high-resolution video-oculography to capture microsaccades in 48 subjects (31 healthy and 17 with Parkinson's disease) when subjects were asked to hold their gaze on a straight-ahead target projected on white background. We analyzed spontaneous discharge patterns of microsaccades.

RESULTS

The first analysis considering coefficient of variation in intersaccadic interval distribution demonstrated that microsaccades in Parkinson's disease are more dispersed than the control group. The second analysis scrutinized microsaccades' temporal variability and revealed 3 distinct occurrence patterns: regular rhythmic, clustered, and randomly occurring following a Poisson-like process. The regular pattern was relatively more common in Parkinson's disease. Subthalamic DBS modulated this temporal pattern. The amount of change in the temporal variability depended on the DBS-induced volume of tissue activation and its overlap with the subthalamic nucleus. The third analysis determined the autocorrelations of microsaccades within 2-second time windows. We found that Parkinson's disease altered local temporal organization in microsaccade generation, and DBS had a modulatory effect.

CONCLUSION

The microsaccades occur in 3 temporal patterns. The basal ganglia are one of the modulators of the microsaccade spontaneity.

Functional Coupling of the Locus Coeruleus Is Linked to Successful Cognitive Control

The locus coeruleus (LC) is a brainstem structure that sends widespread efferent projections throughout the mammalian brain. The LC constitutes the major source of noradrenaline (NE), a modulatory neurotransmitter that is crucial for fundamental brain functions such as arousal, attention, and cognitive control. This role of the LC-NE is traditionally not believed to reflect functional influences on the frontoparietal network or the striatum, but recent advances in chemogenetic manipulations of the rodent brain have challenged this notion. However, demonstrations of LC-NE functional connectivity with these areas in the human brain are surprisingly sparse. Here, we close this gap. Using an established emotional stroop task, we directly compared trials requiring response conflict control with trials that did not require this, but were matched for visual stimulus properties, response modality, and controlled for pupil dilation differences across both trial types. We found that LC-NE functional coupling with the parietal cortex and regions of the striatum is substantially enhanced during trials requiring response conflict control. Crucially, the strength of this functional coupling was directly related to individual reaction time differences incurred by conflict resolution. Our data concur with recent rodent findings and highlight the importance of converging evidence between human and nonhuman neurophysiology to further understand the neural systems supporting adaptive and maladaptive behavior in health and disease.

“Sound” Decisions: The Combined Role of Ambient Noise and Cognitive Regulation on the Neurophysiology of Food Cravings

Our ability to evaluate long-term goals over immediate rewards is manifested in the brain’s decision circuit. Simplistically, it can be divided into a fast, impulsive, reward “system 1” and a slow, deliberate, control “system 2.” In a noisy eating environment, our cognitive resources may get depleted, potentially leading to cognitive overload, emotional arousal, and consequently more rash decisions, such as unhealthy food choices. Here, we investigated the combined impact of cognitive regulation and ambient noise on food cravings through neurophysiological activity. Thirty-seven participants were recruited for an adapted version of the Regulation of Craving (ROC) task. All participants underwent two sessions of the ROC task; once with soft ambient restaurant noise (∼50 dB) and once with loud ambient restaurant noise (∼70 dB), while data from electroencephalography (EEG), electrodermal activity (EDA), and self-reported craving were collected for all palatable food images presented in the task. The results indicated that thinking about future (“later”) consequences vs. immediate (“now”) sensations associated with the food decreased cravings, which were mediated by frontal EEG alpha power. Likewise, “later” trials also increased frontal alpha asymmetry (FAA) —an index for emotional motivation. Furthermore, loud (vs. soft) noise increased alpha, beta, and theta activity, but for theta activity, this was solely occurring during “later” trials. Similarly, EDA signal peak probability was also higher during loud noise. Collectively, our findings suggest that the presence of loud ambient noise in conjunction with prospective thinking can lead to the highest emotional arousal and cognitive load as measured by EDA and EEG, respectively, both of which are important in regulating cravings and decisions. Thus, exploring the combined effects of interoceptive regulation and exteroceptive cues on food-related decision-making could be methodologically advantageous in consumer neuroscience and entail theoretical, commercial, and managerial implications.

A structured ICA-based process for removing auditory evoked potentials

Transcranial magnetic stimulation (TMS)-evoked potentials (TEPs), recorded using electroencephalography (EEG), reflect a combination of TMS-induced cortical activity and multi-sensory responses to TMS. The auditory evoked potential (AEP) is a high-amplitude sensory potential-evoked by the "click" sound produced by every TMS pulse-that can dominate the TEP and obscure observation of other neural components. The AEP is peripherally evoked and therefore should not be stimulation site specific. We address the problem of disentangling the peripherally evoked AEP of the TEP from components evoked by cortical stimulation and ask whether removal of AEP enables more accurate isolation of TEP. We hypothesized that isolation of the AEP using Independent Components Analysis (ICA) would reveal features that are stimulation site specific and unique individual features. In order to improve the effectiveness of ICA for removal of AEP from the TEP, and thus more clearly separate the transcranial-evoked and non-specific TMS-modulated potentials, we merged sham and active TMS datasets representing multiple stimulation conditions, removed the resulting AEP component, and evaluated performance across different sham protocols and clinical populations using reduction in Global and Local Mean Field Power (GMFP/LMFP) and cosine similarity analysis. We show that removing AEPs significantly reduced GMFP and LMFP in the post-stimulation TEP (14 to 400 ms), driven by time windows consistent with the N100 and P200 temporal characteristics of AEPs. Cosine similarity analysis supports that removing AEPs reduces TEP similarity between subjects and reduces TEP similarity between stimulation conditions. Similarity is reduced most in a mid-latency window consistent with the N100 time-course, but nevertheless remains high in this time window. Residual TEP in this window has a time-course and topography unique from AEPs, which follow-up exploratory analyses suggest could be a modulation in the alpha band that is not stimulation site specific but is unique to individual subject. We show, using two datasets and two implementations of sham, evidence in cortical topography, TEP time-course, GMFP/LMFP and cosine similarity analyses that this procedure is effective and conservative in removing the AEP from TEP, and may thus better isolate TMS-evoked activity. We show TEP remaining in early, mid and late latencies. The early response is site and subject specific. Later response may be consistent with TMS-modulated alpha activity that is not site specific but is unique to the individual. TEP remaining after removal of AEP is unique and can provide insight into TMS-evoked potentials and other modulated oscillatory dynamics.

Eye Movement and Pupil Measures: A Review

Our subjective visual experiences involve complex interaction between our eyes, our brain, and the surrounding world. It gives us the sense of sight, color, stereopsis, distance, pattern recognition, motor coordination, and more. The increasing ubiquity of gaze-aware technology brings with it the ability to track gaze and pupil measures with varying degrees of fidelity. With this in mind, a review that considers the various gaze measures becomes increasingly relevant, especially considering our ability to make sense of these signals given different spatio-temporal sampling capacities. In this paper, we selectively review prior work on eye movements and pupil measures. We first describe the main oculomotor events studied in the literature, and their characteristics exploited by different measures. Next, we review various eye movement and pupil measures from prior literature. Finally, we discuss our observations based on applications of these measures, the benefits and practical challenges involving these measures, and our recommendations on future eye-tracking research directions.

An oscillatory pipelining mechanism supporting previewing during visual exploration and reading

Humans have a remarkable ability to efficiently explore visual scenes and text using eye movements. Humans typically make eye movements (saccades) every ~250 ms. Since saccade initiation and execution take 100 ms, this leaves only ~150 ms to recognize the fixated object (or word) while simultaneously previewing candidates for the next saccade goal. We propose a pipelining mechanism where serial processing occurs within a specific brain region, whereas parallel processing occurs across different brain regions. The mechanism is timed by alpha oscillations that coordinate the saccades, visual recognition, and previewing in the cortical hierarchy. Consequently, the neuronal mechanism supporting natural vision and saccades must be studied in unison to uncover the brain mechanisms supporting visual exploration and reading.

Attentional sampling of visual and auditory objects is captured by theta-modulated neural activity

Recent evidence suggests that visual attention alternately samples two behaviourally relevant objects at approximately 4 Hz, rhythmically shifting between the objects. Whether similar attentional rhythms exist in other sensory modalities, however, is not yet clear. We therefore adapted and extended an established paradigm to investigate visual and potential auditory attentional rhythms, as well as possible interactions, on both a behavioural (detection performance, N = 33) and a neural level (EEG, N = 18). The results during unimodal attention demonstrate that both visual- and auditory-target detection fluctuate at frequencies of approximately 4-8 Hz, confirming that attentional rhythms are not specific to visual processing. The EEG recordings provided evidence of oscillatory activity that underlies these behavioural effects. At right and left occipital EEG electrodes, we detected counter-phasic theta-band activity (4-8 Hz), mirroring behavioural evidence of alternating sampling between the objects presented right and left of central fixation, respectively. Similarly, alpha-band activity as a signature of relatively suppressed sensory encoding showed a theta-rhythmic, counter-phasic change in power. Moreover, these theta-rhythmic changes in alpha power were predictive of behavioural performance in both sensory modalities. Overall, the present findings provide a new perspective on the multimodal rhythmicity of attention.

Spontaneous modulations of high-frequency cortical activity

OBJECTIVE

We clarified the clinical and mechanistic significance of physiological modulations of high-frequency broadband cortical activity associated with spontaneous saccadic eye movements during a resting state.

METHODS

We studied 30 patients who underwent epilepsy surgery following extraoperative electrocorticography and electrooculography recordings. We determined whether high-gamma activity at 70-110 Hz preceding saccade onset would predict upcoming ocular behaviors. We assessed how accurately the model incorporating saccade-related high-gamma modulations would localize the primary visual cortex defined by electrical stimulation.

RESULTS

The dynamic atlas demonstrated transient high-gamma suppression in the striatal cortex before saccade onset and high-gamma augmentation subsequently involving the widespread posterior brain regions. More intense striatal high-gamma suppression predicted the upcoming saccade directed to the ipsilateral side and lasting longer in duration. The bagged-tree-ensemble model demonstrated that intense saccade-related high-gamma modulations localized the visual cortex with an accuracy of 95%.

CONCLUSIONS

We successfully animated the neural dynamics supporting saccadic suppression, a principal mechanism minimizing the perception of blurred vision during rapid eye movements. The primary visual cortex per se may prepare actively in advance for massive image motion expected during upcoming prolonged saccades.

SIGNIFICANCE

Measuring saccade-related electrocorticographic signals may help localize the visual cortex and avoid misperceiving physiological high-frequency activity as epileptogenic.

EPOS: EEG Processing Open-Source Scripts

Background

Since the replication crisis, standardization has become even more important in psychological science and neuroscience. As a result, many methods are being reconsidered, and researchers’ degrees of freedom in these methods are being discussed as a potential source of inconsistencies across studies.

New Method

With the aim of addressing these subjectivity issues, we have been working on a tutorial-like EEG (pre-)processing pipeline to achieve an automated method based on the semi-automated analysis proposed by Delorme and Makeig.

Results

Two scripts are presented and explained step-by-step to perform basic, informed ERP and frequency-domain analyses, including data export to statistical programs and visual representations of the data. The open-source software EEGlab in MATLAB is used as the data handling platform, but scripts based on code provided by Mike Cohen (2014) are also included.

Comparison with existing methods

This accompanying tutorial-like article explains and shows how the processing of our automated pipeline affects the data and addresses, especially beginners in EEG-analysis, as other (pre)-processing chains are mostly targeting rather informed users in specialized areas or only parts of a complete procedure. In this context, we compared our pipeline with a selection of existing approaches.

Conclusion

The need for standardization and replication is evident, yet it is equally important to control the plausibility of the suggested solution by data exploration. Here, we provide the community with a tool to enhance the understanding and capability of EEG-analysis. We aim to contribute to comprehensive and reliable analyses for neuro-scientific research.

Word frequency effect in written production: Evidence from ERPs and neural oscillations

It has been widely documented that word frequency (WF) modulates language processing in various input and output modalities. WF effect has also been reported in the domain of written production; however, how WF affects written production is a controversial issue. The present study attempts to investigate the time course of and neural oscillation underlying the WF effect in handwritten production. Participants were asked to handwrite pictures names of high versus low WF, while concurrently recording EEG. EEG trials were extracted time-locked to picture onsets and then submitted to event-related potential analysis and time-frequency analysis. WF affected ERPs in the time windows of around 98-160 and 282-360 ms after picture onsets. More importantly, WF modulated the evoked and induced theta-band (4-8 Hz) neural oscillations in the time window of around 36-240 and 244-472 ms, respectively. Considering the time course of language production and the role of theta-band oscillation in long-term memory retrieval, we suggest that the two stages of the WF effect, respectively, reflect conceptual preparation and retrieval of orthographic word-forms in written production.

A passive BCI for monitoring the intentionality of the gaze-based moving object selection

OBJECTIVE

The use of an electroencephalogram (EEG) anticipation-related component, the expectancy wave (E-wave), in brain-machine interaction was proposed more than 50 years ago. This possibility was not explored for decades, but recently it was shown that voluntary attempts to select items using eye fixations, but not spontaneous eye fixations, are accompanied by the E-wave. Thus, the use of the E-wave detection was proposed for the enhancement of gaze interaction technology, which has a strong need for a mean to decide if a gaze behaviour is voluntary or not. Here, we attempted at estimating whether this approach can be used in the context of moving object selection through smooth pursuit eye movements.

APPROACH

18 participants selected, one by one, items which moved on a computer screen, by gazing at them. In separate runs, the participants performed tasks not related to voluntary selection but also provoking smooth pursuit. A low-cost consumer-grade eye tracker was used for item selection.

MAIN RESULTS

A component resembling the E-wave was found in the averaged EEG segments time-locked to voluntary selection events of every participant. Linear discriminant analysis with shrinkage regularization (sLDA) classified the intentional and spontaneous smooth pursuit eye movements, using single-trial 300 ms long EEG segments, significantly above chance in eight participants. When the classifier output was averaged over ten subsequent data segments, median group ROC AUC of 0.75 was achieved.

SIGNIFICANCE

The results suggest the possible usefulness of the E-wave detection in the gaze-based selection of moving items, e.g., in video games. This technique might be more effective when trial data can be averaged, thus it could be considered for use in passive interfaces, for example, in estimating the degree of the user's involvement during gaze-based interaction.

Regression-based analysis of combined EEG and eye-tracking data: Theory and applications

Fixation-related potentials (FRPs), neural responses aligned to the end of saccades, are a promising tool for studying the dynamics of attention and cognition under natural viewing conditions. In the past, four methodological problems have complicated the analysis of such combined eye-tracking/electroencephalogram experiments: (1) the synchronization of data streams, (2) the removal of ocular artifacts, (3) the condition-specific temporal overlap between the brain responses evoked by consecutive fixations, and (4) the fact that numerous low-level stimulus and saccade properties also influence the postsaccadic neural responses. Although effective solutions exist for the first two problems, the latter two are only beginning to be addressed. In the current paper, we present and review a unified regression-based framework for FRP analysis that allows us to deconvolve overlapping potentials while also controlling for both linear and nonlinear confounds on the FRP waveform. An open software implementation is provided for all procedures. We then demonstrate the advantages of this proposed (non)linear deconvolution modeling approach for data from three commonly studied paradigms: face perception, scene viewing, and reading. First, for a traditional event-related potential (ERP) face recognition experiment, we show how this technique can separate stimulus ERPs from overlapping muscle and brain potentials produced by small (micro)saccades on the face. Second, in natural scene viewing, we model and isolate multiple nonlinear effects of saccade parameters on the FRP. Finally, for a natural sentence reading experiment using the boundary paradigm, we show how it is possible to study the neural correlates of parafoveal preview after removing spurious overlap effects caused by the associated difference in average fixation time. Our results suggest a principal way of measuring reliable eye movement-related brain activity during natural vision.

Role of the locus coeruleus arousal system in cognitive control

Cognitive control lies at the core of human adaptive behaviour. Humans vary substantially in their ability to execute cognitive control with respect to optimally facing environmental challenges, although the neural origins of this heterogeneity are currently not well understood. Recent theoretical frameworks implicate the locus coeruleus noradrenergic arousal system (LC-NE) in that process. Invasive neurophysiological work in rodents has shown that the LC-NE is an important homeostatic control centre of the body. LC-NE innervates the entire neocortex and has particularly strong connections with the cingulate gyrus. In the present study, using a response conflict task, functional magnetic resonance imaging and concurrent pupil dilation measures (a proxy for LC-NE firing), we provide empirical evidence for a decisive role of the LC-NE in cognitive control in humans. We show that the level of individual behavioural adjustment in cognitive control relates to the level of functional coupling between LC-NE and the dorsomedial prefrontal cortex, as well as dorsolateral prefrontal cortex. Moreover, we show that the pupil is substantially more dilated during conflict trials requiring behavioural adjustment than during no conflict trials. In addition, we explore a potential relationship between pupil dilation and neural activity during choice conflict adjustments. Our data provide novel insight into arousal-related influences on cognitive control and suggest pupil dilation as a potential external marker for endogenous neural processes involved in optimising behavioural control. Our results may also be clinically relevant for a variety of pathologies where cognitive control is compromised, such as anxiety, depression, addiction and post-traumatic stress disorder.

Microsaccades in Applied Environments: Real-World Applications of Fixational Eye Movement Measurements

Across a wide variety of research environments, the recording of microsaccades and other fixational eye movements has provided insight and solutions into practical problems. Here we review the literature on fixational eye movements-especially microsaccades-in applied and ecologically-valid scenarios. Recent technical advances allow noninvasive fixational eye movement recordings in real-world contexts, while observers perform a variety of tasks. Thus, fixational eye movement measures have been obtained in a host of real-world scenarios, such as in connection with driver fatigue, vestibular sensory deprivation in astronauts, and elite athletic training, among others. Here we present the state of the art in the practical applications of fixational eye movement research, examine its potential future uses, and discuss the benefits of including microsaccade measures in existing eye movement detection technologies. Current evidence supports the inclusion of fixational eye movement measures in real-world contexts, as part of the development of new or improved oculomotor assessment tools. The real-world applications of fixational eye movement measurements will only grow larger and wider as affordable high-speed and high-spatial resolution eye trackers become increasingly prevalent.

Face Selective Neural Activity: Comparisons Between Fixed and Free Viewing

Event Related Potentials (ERPs) are widely used to study category-selective EEG responses to visual stimuli, such as the face-selective N170 component. Typically, this is done by flashing stimuli at the point of static gaze fixation. While allowing for good experimental control, these paradigms ignore the dynamic role of eye-movements in natural vision. Fixation-related potentials (FRPs), obtained using simultaneous EEG and eye-tracking, overcome this limitation. Various studies have used FRPs to study processes such as lexical processing, target detection and attention allocation. The goal of this study was to carefully compare face-sensitive activity time-locked to an abrupt stimulus onset at fixation, with that time-locked to a self-generated fixation on a stimulus. Twelve participants participated in three experimental conditions: Free-viewing (FRPs), Cued-viewing (FRPs) and Control (ERPs). We used a multiple regression approach to disentangle overlapping activity components. Our results show that the N170 face-effect is evident for the first fixation on a stimulus, whether it follows a self-generated saccade or stimulus appearance at fixation point. The N170 face-effect has similar topography across viewing conditions, but there were major differences within each stimulus category. We ascribe these differences to an overlap of the fixation-related lambda response and the N170. We tested the plausibility of this account using dipole simulations. Finally, the N170 exhibits category-specific adaptation in free viewing. This study establishes the comparability of the free-viewing N170 face-effect with the classic event-related effect, while highlighting the importance of accounting for eye-movement related effects.

What makes a microsaccade? A review of 70 years of research prompts a new detection method

A new method for detecting microsaccades in eye-movement data is presented, following a review of reported microsaccade properties between the 1940s and today. The review focuses on the parameter ranges within which certain physical markers of microsaccades are thought to occur, as well as any features of microsaccades that have been stably reported over time. One feature of microsaccades, their binocularity, drives the new microsaccade detection method. The binocular correlation method for microsaccade detection is validated on two datasets of binocular eye-movements recorded using video-based systems: one collected as part of this study, and one from Nyström et al, 2017. Comparisons between detection methods are made using precision-recall statistics. This confirms that the binocular correlation method performs well when compared to manual coders and performs favourably compared to the commonly used Engbert & Kliegl (2003) method with subsequent modifications (Engbert & Mergenthaler, 2006). The binocular correlation microsaccade detection method is easy to implement and MATLAB code is made available to download.

VisME: Visual Microsaccades Explorer

This work presents a visual analytics approach to explore microsaccade distributions in high-frequency eye tracking data. Research studies often apply filter algorithms and parameter values for microsaccade detection. Even when the same algorithms are employed, different parameter values might be adopted across different studies. In this paper, we present a visual analytics system (VisME) to promote reproducibility in the data analysis of microsaccades. It allows users to interactively vary the parametric values for microsaccade filters and evaluate the resulting influence on microsaccade behavior across individuals and on a group level. In particular, we exploit brushing-and-linking techniques that allow the microsaccadic properties of space, time, and movement direction to be extracted, visualized, and compared across multiple views. We demonstrate in a case study the use of our visual analytics system on data sets collected from natural scene viewing and show in a qualitative usability study the usefulness of this approach for eye tracking researchers. We believe that interactive tools such as VisME will promote greater transparency in eye movement research by providing researchers with the ability to easily understand complex eye tracking data sets; such tools can also serve as teaching systems. VisME is provided as open source software.

Motion direction representation in multivariate electroencephalography activity for smooth pursuit eye movements

Visually-guided smooth pursuit eye movements are composed of initial open-loop and later steady-state periods. Feedforward sensory information dominates the motor behavior during the open-loop pursuit, and a more complex feedback loop regulates the steady-state pursuit. To understand the neural representations of motion direction during open-loop and steady-state smooth pursuits, we recorded electroencephalography (EEG) responses from human observers while they tracked random-dot kinematograms as pursuit targets. We estimated population direction tuning curves from multivariate EEG activity using an inverted encoding model. We found significant direction tuning curves as early as about 60 ms from stimulus onset. Direction tuning responses were generalized to later times during the open-loop smooth pursuit, but they became more dynamic during the later steady-state pursuit. The encoding quality of retinal motion direction information estimated from the early direction tuning curves was predictive of trial-by-trial variation in initial pursuit directions. These results suggest that the movement directions of open-loop smooth pursuit are guided by the representation of the retinal motion present in the multivariate EEG activity.

Optimizing the ICA-based removal of ocular EEG artifacts from free viewing experiments

Combining EEG with eye-tracking is a promising approach to study neural correlates of natural vision, but the resulting recordings are also heavily contaminated by activity of the eye balls, eye lids, and extraocular muscles. While Independent Component Analysis (ICA) is commonly used to suppress these ocular artifacts, its performance under free viewing conditions has not been systematically evaluated and many published reports contain residual artifacts. Here I evaluated and optimized ICA-based correction for two tasks with unconstrained eye movements: visual search in images and sentence reading. In a first step, four parameters of the ICA pipeline were varied orthogonally: the (1) high-pass and (2) low-pass filter applied to the training data, (3) the proportion of training data containing myogenic saccadic spike potentials (SP), and (4) the threshold for eye tracker-based component rejection. In a second step, the eye-tracker was used to objectively quantify the correction quality of each ICA solution, both in terms of undercorrection (residual artifacts) and overcorrection (removal of neurogenic activity). As a benchmark, results were compared to those obtained with an alternative spatial filter, Multiple Source Eye Correction (MSEC). With commonly used settings, Infomax ICA not only left artifacts in the data, but also distorted neurogenic activity during eye movement-free intervals. However, correction results could be strongly improved by training the ICA on optimally filtered data in which SPs were massively overweighted. With optimized procedures, ICA removed virtually all artifacts, including the SP and its associated spectral broadband artifact from both viewing paradigms, with little distortion of neural activity. It also outperformed MSEC in terms of SP correction. Matlab code is provided.

Unfold: an integrated toolbox for overlap correction, non-linear modeling, and regression-based EEG analysis

Electrophysiological research with event-related brain potentials (ERPs) is increasingly moving from simple, strictly orthogonal stimulation paradigms towards more complex, quasi-experimental designs and naturalistic situations that involve fast, multisensory stimulation and complex motor behavior. As a result, electrophysiological responses from subsequent events often overlap with each other. In addition, the recorded neural activity is typically modulated by numerous covariates, which influence the measured responses in a linear or non-linear fashion. Examples of paradigms where systematic temporal overlap variations and low-level confounds between conditions cannot be avoided include combined electroencephalogram (EEG)/eye-tracking experiments during natural vision, fast multisensory stimulation experiments, and mobile brain/body imaging studies. However, even "traditional," highly controlled ERP datasets often contain a hidden mix of overlapping activity (e.g., from stimulus onsets, involuntary microsaccades, or button presses) and it is helpful or even necessary to disentangle these components for a correct interpretation of the results. In this paper, we introduce unfold, a powerful, yet easy-to-use MATLAB toolbox for regression-based EEG analyses that combines existing concepts of massive univariate modeling ("regression-ERPs"), linear deconvolution modeling, and non-linear modeling with the generalized additive model into one coherent and flexible analysis framework. The toolbox is modular, compatible with EEGLAB and can handle even large datasets efficiently. It also includes advanced options for regularization and the use of temporal basis functions (e.g., Fourier sets). We illustrate the advantages of this approach for simulated data as well as data from a standard face recognition experiment. In addition to traditional and non-conventional EEG/ERP designs, unfold can also be applied to other overlapping physiological signals, such as pupillary or electrodermal responses. It is available as open-source software at http://www.unfoldtoolbox.org.

Distinct modes of top-down cognitive processing in the ventral visual cortex

Top-down cognitive control leads to changes in the sensory processing of the brain. In visual perception such changes can take place in the ventral visual cortex altering the functional asymmetry in forward and backward connections. Here we used fixation-related evoked responses of EEG measurement and dynamic causal modeling to examine hierarchical forward-backward asymmetry, while twenty-six healthy adults performed cognitive tasks that require different types of top-down cognitive control (memorizing or searching visual objects embedded in a natural scene image). The generative model revealed an enhanced asymmetry toward forward connections during memorizing, whereas enhanced backward connections were found during searching. This task-dependent modulation of forward and backward connections suggests two distinct modes of top-down cognitive processing in cortical networks. The alteration in forward-backward asymmetry might underlie the functional role in the cognitive control of visual information processing.

Transient and sustained effects of stimulus properties on the generation of microsaccades

Saccades shift the gaze rapidly every few hundred milliseconds from one fixated location to the next, producing a flow of visual input into the visual system even in the absence of changes in the environment. During fixation, small saccades called microsaccades are produced 1-3 times per second, generating a flow of visual input. The characteristics of this visual flow are determined by the timings of the saccades and by the characteristics of the visual stimuli on which they are performed. Previous models of microsaccade generation have accounted for the effects of external stimulation on the production of microsaccades, but they have not considered the effects of the prolonged background stimulus on which microsaccades are performed. The effects of this stimulus on the process of microsaccade generation could be sustained, following its prolonged presentation, or transient, through the visual transients produced by the microsaccades themselves. In four experiments, we varied the properties of the constant displays and examined the resulting modulation of microsaccade properties: their sizes, their timings, and the correlations between properties of consecutive microsaccades. Findings show that displays of higher spatial frequency and contrast produce smaller microsaccades and longer minimal intervals between consecutive microsaccades; and smaller microsaccades are followed by smaller and delayed microsaccades. We explain these findings in light of previous models and suggest a conceptual model by which both sustained and transient effects of the stimulus have central roles in determining the generation of microsaccades.

Refixation control in free viewing: a specialized mechanism divulged by eye-movement-related brain activity

In free viewing, the eyes return to previously visited locations rather frequently, even though the attentional and memory-related processes controlling eye-movement show a strong antirefixation bias. To overcome this bias, a special refixation triggering mechanism may have to be recruited. We probed the neural evidence for such a mechanism by combining eye tracking with EEG recording. A distinctive signal associated with refixation planning was observed in the EEG during the presaccadic interval: the presaccadic potential was reduced in amplitude before a refixation compared with normal fixations. The result offers direct evidence for a special refixation mechanism that operates in the saccade planning stage of eye movement control. Once the eyes have landed on the revisited location, acquisition of visual information proceeds indistinguishably from ordinary fixations. NEW & NOTEWORTHY A substantial proportion of eye fixations in human natural viewing behavior are revisits of recently visited locations, i.e., refixations. Our recently developed methods enabled us to study refixations in a free viewing visual search task, using combined eye movement and EEG recording. We identified in the EEG a distinctive refixation-related signal, signifying a control mechanism specific to refixations as opposed to ordinary eye fixations.