Combining EEG and eye tracking: identification, characterization, and correction of eye movement artifacts in electroencephalographic data

EEG and Eye Tracking Demonstrate Vigilance Enhancement with Challenge Integration

Maintaining vigilance is possibly the first requirement for surveillance tasks where personnel are faced with monotonous yet intensive monitoring tasks. Decrement in vigilance in such situations could result in dangerous consequences such as accidents, loss of life and system failure. In this paper, we investigate the possibility to enhance vigilance or sustained attention using “challenge integration,” a strategy that integrates a primary task with challenging stimuli. A primary surveillance task (identifying an intruder in a simulated factory environment) and a challenge stimulus (periods of rain obscuring the surveillance scene) were employed to test the changes in vigilance levels. The effect of integrating challenging events (resulting from artificially simulated rain) into the task were compared to the initial monotonous phase. EEG and eye tracking data is collected and analyzed for n = 12 subjects. Frontal midline theta power and frontal theta to parietal alpha power ratio which are used as measures of engagement and attention allocation show an increase due to challenge integration (p < 0.05 in each case). Relative delta band power of EEG also shows statistically significant suppression on the frontoparietal and occipital cortices due to challenge integration (p < 0.05). Saccade amplitude, saccade velocity and blink rate obtained from eye tracking data exhibit statistically significant changes during the challenge phase of the experiment (p < 0.05 in each case). From the correlation analysis between the statistically significant measures of eye tracking and EEG, we infer that saccade amplitude and saccade velocity decrease with vigilance decrement along with frontal midline theta and frontal theta to parietal alpha ratio. Conversely, blink rate and relative delta power increase with vigilance decrement. However, these measures exhibit a reverse trend when challenge stimulus appears in the task suggesting vigilance enhancement. Moreover, the mean reaction time is lower for the challenge integrated phase (RTmean = 3.65 ± 1.4s) compared to initial monotonous phase without challenge (RTmean = 4.6 ± 2.7s). Our work shows that vigilance level, as assessed by response of these vital signs, is enhanced by challenge integration.

Visual processing during natural reading

Reading is a unique human ability that plays a pivotal role in the development and functioning of our modern society. However, its neural basis remains poorly understood since previous research was focused on reading words with fixed gaze. Here we developed a methodological framework for single-trial analysis of fixation onset-related EEG activity (FOREA) that enabled us to investigate visual information processing during natural reading. To reveal the effect of reading skills on orthographic processing during natural reading, we measured how altering the configural properties of the written text by modifying inter-letter spacing affects FOREA. We found that orthographic processing is reflected in FOREA in three consecutive time windows (120–175 ms, 230–265 ms, 345–380 ms after fixation onset) and the magnitude of FOREA effects in the two later time intervals showed a close association with the participants’ reading speed: FOREA effects were larger in fast than in slow readers. Furthermore, these expertise-driven configural effects were clearly dissociable from the FOREA signatures of visual perceptual processes engaged to handle the increased crowding (155–220 ms) as a result of decreasing letter spacing. Our findings revealed that with increased reading skills orthographic processing becomes more sensitive to the configural properties of the written text.

Hybrid ICA-Regression: Automatic Identification and Removal of Ocular Artifacts from Electroencephalographic Signals

Electroencephalography (EEG) is a portable brain-imaging technique with the advantage of high-temporal resolution that can be used to record electrical activity of the brain. However, it is difficult to analyze EEG signals due to the contamination of ocular artifacts, and which potentially results in misleading conclusions. Also, it is a proven fact that the contamination of ocular artifacts cause to reduce the classification accuracy of a brain-computer interface (BCI). It is therefore very important to remove/reduce these artifacts before the analysis of EEG signals for applications like BCI. In this paper, a hybrid framework that combines independent component analysis (ICA), regression and high-order statistics has been proposed to identify and eliminate artifactual activities from EEG data. We used simulated, experimental and standard EEG signals to evaluate and analyze the effectiveness of the proposed method. Results demonstrate that the proposed method can effectively remove ocular artifacts as well as it can preserve the neuronal signals present in EEG data. A comparison with four methods from literature namely ICA, regression analysis, wavelet-ICA (wICA), and regression-ICA (REGICA) confirms the significantly enhanced performance and effectiveness of the proposed method for removal of ocular activities from EEG, in terms of lower mean square error and mean absolute error values and higher mutual information between reconstructed and original EEG.

Oscillatory Brain Dynamics during Sentence Reading: A Fixation-Related Spectral Perturbation Analysis

The present study investigated oscillatory brain dynamics during self-paced sentence-level processing. Participants read fully correct sentences, sentences containing a semantic violation and "sentences" in which the order of the words was randomized. At the target word level, fixations on semantically unrelated words elicited a lower-beta band (13-18 Hz) desynchronization. At the sentence level, gamma power (31-55 Hz) increased linearly for syntactically correct sentences, but not when the order of the words was randomized. In the 300-900 ms time window after sentence onsets, theta power (4-7 Hz) was greater for syntactically correct sentences as compared to sentences where no syntactic structure was preserved (random words condition). We interpret our results as conforming with a recently formulated predictive-coding framework for oscillatory neural dynamics during sentence-level language comprehension. Additionally, we discuss how our results relate to previous findings with serial visual presentation vs. self-paced reading.

Direction and viewing area-sensitive influence of EOG artifacts revealed in the EEG topographic pattern analysis

The influence of eye movement-related artifacts on electroencephalography (EEG) signals of human subjects, who were requested to perform a direction or viewing area dependent saccade task, was investigated by using a simultaneous recording with ocular potentials as electro-oculography (EOG). In the past, EOG artifact removals have been studied in tasks with a single fixation point in the screen center, with less attention to the sensitivity of cornea-retinal dipole orientations to the EEG head map. In the present study, we hypothesized the existence of a systematic EOG influence that differs according to coupling conditions of eye-movement directions with viewing areas including different fixation points. The effect was validated in the linear regression analysis by using 12 task conditions combining horizontal/vertical eye-movement direction and three segregated zones of gaze in the screen. In the first place, event-related potential topographic patterns were analyzed to compare the 12 conditions and propagation coefficients of the linear regression analysis were successively calculated in each condition. As a result, the EOG influences were significantly different in a large number of EEG channels, especially in the case of horizontal eye-movements. In the cross validation, the linear regression analysis using the appropriate dataset of the target direction/viewing area combination demonstrated an improved performance compared with the traditional methods using a single fixation at the center. This result may open a potential way to improve artifact correction methods by considering the systematic EOG influence that can be predicted according to the view angle such as using eye-tracker systems.

Hybrid EEG--Eye Tracker: Automatic Identification and Removal of Eye Movement and Blink Artifacts from Electroencephalographic Signal

Contamination of eye movement and blink artifacts in Electroencephalogram (EEG) recording makes the analysis of EEG data more difficult and could result in mislead findings. Efficient removal of these artifacts from EEG data is an essential step in improving classification accuracy to develop the brain-computer interface (BCI). In this paper, we proposed an automatic framework based on independent component analysis (ICA) and system identification to identify and remove ocular artifacts from EEG data by using hybrid EEG and eye tracker system. The performance of the proposed algorithm is illustrated using experimental and standard EEG datasets. The proposed algorithm not only removes the ocular artifacts from artifactual zone but also preserves the neuronal activity related EEG signals in non-artifactual zone. The comparison with the two state-of-the-art techniques namely ADJUST based ICA and REGICA reveals the significant improved performance of the proposed algorithm for removing eye movement and blink artifacts from EEG data. Additionally, results demonstrate that the proposed algorithm can achieve lower relative error and higher mutual information values between corrected EEG and artifact-free EEG data.

Classification of Eye Fixation Related Potentials for Variable Stimulus Saliency

OBJECTIVE

Electroencephalography (EEG) and eye tracking can possibly provide information about which items displayed on the screen are relevant for a person. Exploiting this implicit information promises to enhance various software applications. The specific problem addressed by the present study is that items shown in real applications are typically diverse. Accordingly, the saliency of information, which allows to discriminate between relevant and irrelevant items, varies. As a consequence, recognition can happen in foveal or in peripheral vision, i.e., either before or after the saccade to the item. Accordingly, neural processes related to recognition are expected to occur with a variable latency with respect to the eye movements. The aim was to investigate if relevance estimation based on EEG and eye tracking data is possible despite of the aforementioned variability.

APPROACH

Sixteen subjects performed a search task where the target saliency was varied while the EEG was recorded and the unrestrained eye movements were tracked. Based on the acquired data, it was estimated which of the items displayed were targets and which were distractors in the search task.

RESULTS

Target prediction was possible also when the stimulus saliencies were mixed. Information contained in EEG and eye tracking data was found to be complementary and neural signals were captured despite of the unrestricted eye movements. The classification algorithm was able to cope with the experimentally induced variable timing of neural activity related to target recognition.

SIGNIFICANCE

It was demonstrated how EEG and eye tracking data can provide implicit information about the relevance of items on the screen for potential use in online applications.

Neural Patterns of the Implicit Association Test

The Implicit Association Test (IAT) is a reaction time based categorization task that measures the differential associative strength between bipolar targets and evaluative attribute concepts as an approach to indexing implicit beliefs or biases. An open question exists as to what exactly the IAT measures, and here EEG (Electroencephalography) has been used to investigate the time course of ERPs (Event-related Potential) indices and implicated brain regions in the IAT. IAT-EEG research identifies a number of early (250-450 ms) negative ERPs indexing early-(pre-response) processing stages of the IAT. ERP activity in this time range is known to index processes related to cognitive control and semantic processing. A central focus of these efforts has been to use IAT-ERPs to delineate the implicit and explicit factors contributing to measured IAT effects. Increasing evidence indicates that cognitive control (and related top-down modulation of attention/perceptual processing) may be components in the effective measurement of IAT effects, as factors such as physical setting or task instruction can change an IAT measurement. In this study we further implicate the role of proactive cognitive control and top-down modulation of attention/perceptual processing in the IAT-EEG. We find statistically significant relationships between D-score (a reaction-time based measure of the IAT-effect) and early ERP-time windows, indicating where more rapid word categorizations driving the IAT effect are present, they are at least partly explainable by neural activity not significantly correlated with the IAT measurement itself. Using LORETA, we identify a number of brain regions driving these ERP-IAT relationships notably involving left-temporal, insular, cingulate, medial frontal and parietal cortex in time regions corresponding to the N2- and P3-related activity. The identified brain regions involved with reduced reaction times on congruent blocks coincide with those of previous studies.

Informed decomposition of electroencephalographic data

BACKGROUND

Blind source separation techniques have become the de facto standard for decomposing electroencephalographic (EEG) data. These methods are poorly suited for incorporating prior information into the decomposition process. While alternative techniques to this problem, such as the use of constrained optimization techniques, have been proposed, these alternative techniques tend to only minimally satisfy the prior constraints. In addition, the experimenter must preset a number of parameters describing both this minimal limit as well as the size of the target subspaces.

NEW METHOD

We propose an informed decomposition approach that builds upon the constrained optimization approaches for independent components analysis to better model and separate distinct subspaces within EEG data. We use a likelihood function to adaptively determine the optimal model size for each target subspace.

RESULTS

Using our method we are able to produce ordered independent subspaces that exhibit less residual mixing than those obtained with other methods. The results show an improvement in modeling specific features of the EEG space, while also showing a simultaneous reduction in the number of components needed for each model.

COMPARISON WITH EXISTING METHOD(S)

We first compare our approach to common methods in the field of EEG decomposition, such as Infomax, FastICA, PCA, JADE, and SOBI for the task of modeling and removing both EOG and EMG artifacts. We then demonstrate the utility of our approach for the more complex problem of modeling neural activity.

CONCLUSIONS

By working in a one-size-fits-all fashion current EEG decomposition methods do not adapt to the specifics of each data set and are not well designed to incorporate additional information about the decomposition problem. However, by adding specific information about the problem to the decomposition task, we improve the identification and separation of distinct subspaces within the original data and show better preservation of the remaining data.

Predictions of Visual Content across Eye Movements and Their Modulation by Inferred Information

The brain is proposed to operate through probabilistic inference, testing and refining predictions about the world. Here, we search for neural activity compatible with the violation of active predictions, learned from the contingencies between actions and the consequent changes in sensory input. We focused on vision, where eye movements produce stimuli shifts that could, in principle, be predicted. We compared, in humans, error signals to saccade-contingent changes of veridical and inferred inputs by contrasting the electroencephalographic activity after saccades to a stimulus presented inside or outside the blind spot. We observed early (<250 ms) and late (>250 ms) error signals after stimulus change, indicating the violation of sensory and associative predictions, respectively. Remarkably, the late response was diminished for blind-spot trials. These results indicate that predictive signals occur across multiple levels of the visual hierarchy, based on generative models that differentiate between signals that originate from the outside world and those that are inferred.

Spectral Signatures of Saccade Target Selection

Action generation relies on a widely distributed network of brain areas. However, little is known about the spatiotemporal dynamics of neuronal activity in the network that gives rise to voluntary action in humans. Here, we used magnetoencephalography (MEG) and source analysis (n = 15, 7 female subjects) to investigate the spectral signatures of human cortical networks engaged in active and intrinsically motivated viewing behavior. We compared neuronal activity of externally cued saccades with saccades to freely chosen targets. For planning and execution of both saccade types, we found an increase in gamma band (~64-128 Hz) activity and a concurrent decrease in beta band (~12-32 Hz) activity in saccadic control areas, including the intraparietal sulcus and the frontal eye fields. Guided compared to voluntary actions were accompanied by stronger transient increases in the gamma and low frequency (<16 Hz) range immediately following the instructional cue. In contrast, action selection between competing alternatives was reflected by stronger sustained fronto-parietal gamma increases that occurred later in time and persisted until movement execution. This sustained enhancement for free target selection was accompanied by a spatially widespread reduction of lower frequency power (~8-45 Hz) in parietal and extrastriate areas. Our results suggest that neuronal population activity in the gamma frequency band in a distributed network of fronto-parietal areas reflects the intrinsically driven process of selection among competing behavioral alternatives.

Neuronal oscillations form parietal/frontal networks during contour integration

The ability to integrate visual features into a global coherent percept that can be further categorized and manipulated are fundamental abilities of the neural system. While the processing of visual information involves activation of early visual cortices, the recruitment of parietal and frontal cortices has been shown to be crucial for perceptual processes. Yet is it not clear how both cortical and long-range oscillatory activity leads to the integration of visual features into a coherent percept. Here, we will investigate perceptual grouping through the analysis of a contour categorization task, where the local elements that form contour must be linked into a coherent structure, which is then further processed and manipulated to perform the categorization task. The contour formation in our visual stimulus is a dynamic process where, for the first time, visual perception of contours is disentangled from the onset of visual stimulation or from motor preparation, cognitive processes that until now have been behaviorally attached to perceptual processes. Our main finding is that, while local and long-range synchronization at several frequencies seem to be an ongoing phenomena, categorization of a contour could only be predicted through local oscillatory activity within parietal/frontal sources, which in turn, would synchronize at gamma (>30 Hz) frequency. Simultaneously, fronto-parietal beta (13–30 Hz) phase locking forms a network spanning across neural sources that are not category specific. Both long range networks, i.e., the gamma network that is category specific, and the beta network that is not category specific, are functionally distinct but spatially overlapping. Altogether, we show that a critical mechanism underlying contour categorization involves oscillatory activity within parietal/frontal cortices, as well as its synchronization across distal cortical sites.

Telemonitoring with respect to mood disorders and information and communication technologies: overview and presentation of the PSYCHE project

This paper reviews what we know about prediction in relation to mood disorders from the perspective of clinical, biological, and physiological markers. It then also presents how information and communication technologies have developed in the field of mood disorders, from the first steps, for example, the transition from paper and pencil to more sophisticated methods, to the development of ecological momentary assessment methods and, more recently, wearable systems. These recent developments have paved the way for the use of integrative approaches capable of assessing multiple variables. The PSYCHE project stands for Personalised monitoring SYstems for Care in mental HEalth.

ERPLAB: an open-source toolbox for the analysis of event-related potentials

ERPLAB toolbox is a freely available, open-source toolbox for processing and analyzing event-related potential (ERP) data in the MATLAB environment. ERPLAB is closely integrated with EEGLAB, a popular open-source toolbox that provides many EEG preprocessing steps and an excellent user interface design. ERPLAB adds to EEGLAB's EEG processing functions, providing additional tools for filtering, artifact detection, re-referencing, and sorting of events, among others. ERPLAB also provides robust tools for averaging EEG segments together to create averaged ERPs, for creating difference waves and other recombinations of ERP waveforms through algebraic expressions, for filtering and re-referencing the averaged ERPs, for plotting ERP waveforms and scalp maps, and for quantifying several types of amplitudes and latencies. ERPLAB's tools can be accessed either from an easy-to-learn graphical user interface or from MATLAB scripts, and a command history function makes it easy for users with no programming experience to write scripts. Consequently, ERPLAB provides both ease of use and virtually unlimited power and flexibility, making it appropriate for the analysis of both simple and complex ERP experiments. Several forms of documentation are available, including a detailed user's guide, a step-by-step tutorial, a scripting guide, and a set of video-based demonstrations.

The neurophysiological index of visual working memory maintenance is not due to load dependent eye movements

The Contralateral Delayed Activity (CDA) is slow negative potential found during a variety of tasks, providing an important measure of the representation of information in visual working memory. However, it is studied using stimulus arrays in which the to-be-remembered objects are shown in the periphery of the left or the right visual field. Our goal was to determine whether fixational eye movements in the direction of the memoranda might underlie the CDA. We found that subjects' gaze was shifted toward the visual field of the memoranda during the retention interval, with its magnitude increasing with the set size. However, the CDA was clearly observed even when the subjects' gaze shifts were absent. In addition, the magnitude of the subjects' gaze shifts was unrelated to their visual working memory capacity measured with behavioral data, unlike the CDA. Finally, the onset latency of the set size dependent eye movements followed the onset of the set size dependent CDA. Thus, our findings clearly show that the CDA does not represent a simple inability to maintain fixation during visual working memory maintenance, but that this neural index of representation in working memory appears to induce eye movements toward the locations of the objects being remembered.

Methodological aspects of EEG and body dynamics measurements during motion

EEG involves the recording, analysis, and interpretation of voltages recorded on the human scalp which originate from brain gray matter. EEG is one of the most popular methods of studying and understanding the processes that underlie behavior. This is so, because EEG is relatively cheap, easy to wear, light weight and has high temporal resolution. In terms of behavior, this encompasses actions, such as movements that are performed in response to the environment. However, there are methodological difficulties which can occur when recording EEG during movement such as movement artifacts. Thus, most studies about the human brain have examined activations during static conditions. This article attempts to compile and describe relevant methodological solutions that emerged in order to measure body and brain dynamics during motion. These descriptions cover suggestions on how to avoid and reduce motion artifacts, hardware, software and techniques for synchronously recording EEG, EMG, kinematics, kinetics, and eye movements during motion. Additionally, we present various recording systems, EEG electrodes, caps and methods for determinating real/custom electrode positions. In the end we will conclude that it is possible to record and analyze synchronized brain and body dynamics related to movement or exercise tasks.

Motor-related brain activity during action observation: a neural substrate for electrocorticographic brain-computer interfaces after spinal cord injury

After spinal cord injury (SCI), motor commands from the brain are unable to reach peripheral nerves and muscles below the level of the lesion. Action observation (AO), in which a person observes someone else performing an action, has been used to augment traditional rehabilitation paradigms. Similarly, AO can be used to derive the relationship between brain activity and movement kinematics for a motor-based brain-computer interface (BCI) even when the user cannot generate overt movements. BCIs use brain signals to control external devices to replace functions that have been lost due to SCI or other motor impairment. Previous studies have reported congruent motor cortical activity during observed and overt movements using magnetoencephalography (MEG) and functional magnetic resonance imaging (fMRI). Recent single-unit studies using intracortical microelectrodes also demonstrated that a large number of motor cortical neurons had similar firing rate patterns between overt and observed movements. Given the increasing interest in electrocorticography (ECoG)-based BCIs, our goal was to identify whether action observation-related cortical activity could be recorded using ECoG during grasping tasks. Specifically, we aimed to identify congruent neural activity during observed and executed movements in both the sensorimotor rhythm (10–40 Hz) and the high-gamma band (65–115 Hz) which contains significant movement-related information. We observed significant motor-related high-gamma band activity during AO in both able-bodied individuals and one participant with a complete C4 SCI. Furthermore, in able-bodied participants, both the low and high frequency bands demonstrated congruent activity between action execution and observation. Our results suggest that AO could be an effective and critical procedure for deriving the mapping from ECoG signals to intended movement for an ECoG-based BCI system for individuals with paralysis.

Correlations between the signal complexity of cerebral and cardiac electrical activity: a multiscale entropy analysis

The heart begins to beat before the brain is formed. Whether conventional hierarchical central commands sent by the brain to the heart alone explain all the interplay between these two organs should be reconsidered. Here, we demonstrate correlations between the signal complexity of brain and cardiac activity. Eighty-seven geriatric outpatients with healthy hearts and varied cognitive abilities each provided a 24-hour electrocardiography (ECG) and a 19-channel eye-closed routine electroencephalography (EEG). Multiscale entropy (MSE) analysis was applied to three epochs (resting-awake state, photic stimulation of fast frequencies (fast-PS), and photic stimulation of slow frequencies (slow-PS)) of EEG in the 1–58 Hz frequency range, and three RR interval (RRI) time series (awake-state, sleep and that concomitant with the EEG) for each subject. The low-to-high frequency power (LF/HF) ratio of RRI was calculated to represent sympatho-vagal balance. With statistics after Bonferroni corrections, we found that: (a) the summed MSE value on coarse scales of the awake RRI (scales 11–20, RRI-MSE-coarse) were inversely correlated with the summed MSE value on coarse scales of the resting-awake EEG (scales 6–20, EEG-MSE-coarse) at Fp2, C4, T6 and T4; (b) the awake RRI-MSE-coarse was inversely correlated with the fast-PS EEG-MSE-coarse at O1, O2 and C4; (c) the sleep RRI-MSE-coarse was inversely correlated with the slow-PS EEG-MSE-coarse at Fp2; (d) the RRI-MSE-coarse and LF/HF ratio of the awake RRI were correlated positively to each other; (e) the EEG-MSE-coarse at F8 was proportional to the cognitive test score; (f) the results conform to the cholinergic hypothesis which states that cognitive impairment causes reduction in vagal cardiac modulation; (g) fast-PS significantly lowered the EEG-MSE-coarse globally. Whether these heart-brain correlations could be fully explained by the central autonomic network is unknown and needs further exploration.

Epileptic nystagmus and vertigo associated with bilateral temporal and frontal lobe epilepsy

Epileptic nystagmus is defined as a quick, repetitive jerky movement of the eyeball associated with seizure activity. In cases of epileptic nystagmus associated with ictal discharge from multiple brain areas, localization of the exact epileptogenic zone could be extremely difficult. In a nine-year-old patient with epileptic nystagmus and vertigo associated with bilateral temporal and frontal lobe epilepsy, we could infer the epileptic focus by interpreting the patient's clinical picture, characteristics of nystagmus, and findings of electroencephalography.

Eye movement related brain responses to emotional scenes during free viewing

Emotional stimuli are preferentially processed over neutral stimuli. Previous studies, however, disagree on whether emotional stimuli capture attention preattentively or whether the processing advantage is dependent on allocation of attention. The present study investigated attention and emotion processes by measuring brain responses related to eye movement events while 11 participants viewed images selected from the International Affective Picture System (IAPS). Brain responses to emotional stimuli were compared between serial and parallel presentation. An "emotional" set included one image with high positive or negative valence among neutral images. A "neutral" set comprised four neutral images. The participants were asked to indicate which picture-if any-was emotional and to rate that picture on valence and arousal. In the serial condition, the event-related potentials (ERPs) were time-locked to the stimulus onset. In the parallel condition, the ERPs were time-locked to the first eye entry on an image. The eye movement results showed facilitated processing of emotional, especially unpleasant information. The EEG results in both presentation conditions showed that the LPP ("late positive potential") amplitudes at 400-500 ms were enlarged for the unpleasant and pleasant pictures as compared to neutral pictures. Moreover, the unpleasant scenes elicited stronger responses than pleasant scenes. The ERP results did not support parafoveal emotional processing, although the eye movement results suggested faster attention capture by emotional stimuli. Our findings, thus, suggested that emotional processing depends on overt attentional resources engaged in the processing of emotional content. The results also indicate that brain responses to emotional images can be analyzed time-locked to eye movement events, although the response amplitudes were larger during serial presentation.

Gyroscope-driven mouse pointer with an EMOTIV® EEG headset and data analysis based on Empirical Mode Decomposition

This paper presents a project on the development of a cursor control emulating the typical operations of a computer-mouse, using gyroscope and eye-blinking electromyographic signals which are obtained through a commercial 16-electrode wireless headset, recently released by Emotiv. The cursor position is controlled using information from a gyroscope included in the headset. The clicks are generated through the user's blinking with an adequate detection procedure based on the spectral-like technique called Empirical Mode Decomposition (EMD). EMD is proposed as a simple and quick computational tool, yet effective, aimed to artifact reduction from head movements as well as a method to detect blinking signals for mouse control. Kalman filter is used as state estimator for mouse position control and jitter removal. The detection rate obtained in average was 94.9%. Experimental setup and some obtained results are presented.

Parafoveal X-masks interfere with foveal word recognition: evidence from fixation-related brain potentials

The boundary paradigm, in combination with parafoveal masks, is the main technique for studying parafoveal preprocessing during reading. The rationale is that the masks (e.g., strings of X's) prevent parafoveal preprocessing, but do not interfere with foveal processing. A recent study, however, raised doubts about the neutrality of parafoveal masks. In the present study, we explored this issue by means of fixation-related brain potentials (FRPs). Two FRP conditions presented rows of five words. The task of the participant was to judge whether the final word of a list was a "new" word, or whether it was a repeated (i.e., "old") word. The critical manipulation was that the final word was X-masked during parafoveal preview in one condition, whereas another condition presented a valid preview of the word. In two additional event-related brain potential (ERP) conditions, the words were presented serially with no parafoveal preview available; in one of the conditions with a fixed timing, in the other word presentation was self-paced by the participants. Expectedly, the valid-preview FRP condition elicited the shortest processing times. Processing times did not differ between the two ERP conditions indicating that "cognitive readiness" during self-paced processing can be ruled out as an alternative explanation for differences in processing times between the ERP and the FRP conditions. The longest processing times were found in the X-mask FRP condition indicating that parafoveal X-masks interfere with foveal word recognition.

Co-registration of eye movements and event-related potentials in connected-text paragraph reading

Eyetracking during reading has provided a critical source of on-line behavioral data informing basic theory in language processing. Similarly, event-related potentials (ERPs) have provided an important on-line measure of the neural correlates of language processing. Recently there has been strong interest in co-registering eyetracking and ERPs from simultaneous recording to capitalize on the strengths of both techniques, but a challenge has been devising approaches for controlling artifacts produced by eye movements in the EEG waveform. In this paper we describe our approach to correcting for eye movements in EEG and demonstrate its applicability to reading. The method is based on independent components analysis, and uses three criteria for identifying components tied to saccades: (1) component loadings on the surface of the head are consistent with eye movements; (2) source analysis localizes component activity to the eyes, and (3) the temporal activation of the component occurred at the time of the eye movement and differed for right and left eye movements. We demonstrate this method's applicability to reading by comparing ERPs time-locked to fixation onset in two reading conditions. In the text-reading condition, participants read paragraphs of text. In the pseudo-reading control condition, participants moved their eyes through spatially similar pseudo-text that preserved word locations, word shapes, and paragraph spatial structure, but eliminated meaning. The corrected EEG, time-locked to fixation onsets, showed effects of reading condition in early ERP components. The results indicate that co-registration of eyetracking and EEG in connected-text paragraph reading is possible, and has the potential to become an important tool for investigating the cognitive and neural bases of on-line language processing in reading.

Dissociating neuronal gamma-band activity from cranial and ocular muscle activity in EEG

EEG is the most common technique for studying neuronal dynamics of the human brain. However, electromyogenic artifacts from cranial muscles and ocular muscles executing involuntary microsaccades compromise estimates of neuronal activity in the gamma band (>30 Hz). Yet, the relative contributions and practical consequences of these artifacts remain unclear. Here, we systematically dissected the effects of these different artifacts on studying visual gamma-band activity with EEG on the sensor and source level, and show strategies to cope with these confounds. We found that cranial muscle activity prevented a direct investigation of neuronal gamma-band activity at the sensor level. Furthermore, we found prolonged microsaccade-related artifacts beyond the well-known transient EEG confounds. We then show that if electromyogenic artifacts are carefully accounted for, the EEG nonetheless allows for studying visual gamma-band activity even at the sensor level. Furthermore, we found that source analysis based on spatial filtering does not only map the EEG signals to the cortical space of interest, but also efficiently accounts for cranial and ocular muscle artifacts. Together, our results clarify the relative contributions and characteristics of myogenic artifacts confounding visual gamma-band activity in EEG, and provide practical guidelines for future experiments.

Visual encoding and fixation target selection in free viewing: presaccadic brain potentials

In scrutinizing a scene, the eyes alternate between fixations and saccades. During a fixation, two component processes can be distinguished: visual encoding and selection of the next fixation target. We aimed to distinguish the neural correlates of these processes in the electrical brain activity prior to a saccade onset. Participants viewed color photographs of natural scenes, in preparation for a change detection task. Then, for each participant and each scene we computed an image heat map, with temperature representing the duration and density of fixations. The temperature difference between the start and end points of saccades was taken as a measure of the expected task-relevance of the information concentrated in specific regions of a scene. Visual encoding was evaluated according to whether subsequent change was correctly detected. Saccades with larger temperature difference were more likely to be followed by correct detection than ones with smaller temperature differences. The amplitude of presaccadic activity over anterior brain areas was larger for correct detection than for detection failure. This difference was observed for short "scrutinizing" but not for long "explorative" saccades, suggesting that presaccadic activity reflects top-down saccade guidance. Thus, successful encoding requires local scanning of scene regions which are expected to be task-relevant. Next, we evaluated fixation target selection. Saccades "moving up" in temperature were preceded by presaccadic activity of higher amplitude than those "moving down". This finding suggests that presaccadic activity reflects attention deployed to the following fixation location. Our findings illustrate how presaccadic activity can elucidate concurrent brain processes related to the immediate goal of planning the next saccade and the larger-scale goal of constructing a robust representation of the visual scene.

To perceive or not perceive: the role of gamma-band activity in signaling object percepts

The relation of gamma-band synchrony to holistic perception in which concerns the effects of sensory processing, high level perceptual gestalt formation, motor planning and response is still controversial. To provide a more direct link to emergent perceptual states we have used holistic EEG/ERP paradigms where the moment of perceptual “discovery” of a global pattern was variable. Using a rapid visual presentation of short-lived Mooney objects we found an increase of gamma-band activity locked to perceptual events. Additional experiments using dynamic Mooney stimuli showed that gamma activity increases well before the report of an emergent holistic percept. To confirm these findings in a data driven manner we have further used a support vector machine classification approach to distinguish between perceptual vs. non perceptual states, based on time-frequency features. Sensitivity, specificity and accuracy were all above 95%. Modulations in the 30–75 Hz range were larger for perception states. Interestingly, phase synchrony was larger for perception states for high frequency bands. By focusing on global gestalt mechanisms instead of local processing we conclude that gamma-band activity and synchrony provide a signature of holistic perceptual states of variable onset, which are separable from sensory and motor processing.