Combining EEG and eye tracking: identification, characterization, and correction of eye movement artifacts in electroencephalographic data

Influence of reading skill and word length on fixation-related brain activity in school-aged children during natural reading

Word length is one of the main determinants of eye movements during reading and has been shown to influence slow readers more strongly than typical readers. The influence of word length on reading in individuals with different reading skill levels has been shown in separate eye-tracking and electroencephalography studies. However, the influence of reading difficulty on cortical correlates of word length effect during natural reading is unknown. To investigate how reading skill is related to brain activity during natural reading, we performed an exploratory analysis on our data set from a previous study, where slow reading (N = 27) and typically reading (N = 65) 12-to-13.5-year-old children read sentences while co-registered ET-EEG was recorded. We extracted fixation-related potentials (FRPs) from the sentences using the linear deconvolution approach. We examined standard eye-movement variables and deconvoluted FRP estimates: intercept of the response, categorical effect of first fixation versus additional fixation and continuous effect of word length. We replicated the pattern of stronger word length effect in eye movements for slow readers. We found a difference between typical readers and slow readers in the FRP intercept, which contains activity that is common to all fixations, within a fixation time-window of 50-300 ms. For both groups, the word length effect was present in brain activity during additional fixations; however, this effect was not different between groups. This suggests that stronger word length effect in the eye movements of slow readers might be mainly due re-fixations, which are more probable due to the lower efficiency of visual processing.

Optimizing the ICA-based removal of ocular EEG artifacts from free viewing experiments

Combining EEG with eye-tracking is a promising approach to study neural correlates of natural vision, but the resulting recordings are also heavily contaminated by activity of the eye balls, eye lids, and extraocular muscles. While Independent Component Analysis (ICA) is commonly used to suppress these ocular artifacts, its performance under free viewing conditions has not been systematically evaluated and many published reports contain residual artifacts. Here I evaluated and optimized ICA-based correction for two tasks with unconstrained eye movements: visual search in images and sentence reading. In a first step, four parameters of the ICA pipeline were varied orthogonally: the (1) high-pass and (2) low-pass filter applied to the training data, (3) the proportion of training data containing myogenic saccadic spike potentials (SP), and (4) the threshold for eye tracker-based component rejection. In a second step, the eye-tracker was used to objectively quantify the correction quality of each ICA solution, both in terms of undercorrection (residual artifacts) and overcorrection (removal of neurogenic activity). As a benchmark, results were compared to those obtained with an alternative spatial filter, Multiple Source Eye Correction (MSEC). With commonly used settings, Infomax ICA not only left artifacts in the data, but also distorted neurogenic activity during eye movement-free intervals. However, correction results could be strongly improved by training the ICA on optimally filtered data in which SPs were massively overweighted. With optimized procedures, ICA removed virtually all artifacts, including the SP and its associated spectral broadband artifact from both viewing paradigms, with little distortion of neural activity. It also outperformed MSEC in terms of SP correction. Matlab code is provided.

Artifacts in Simultaneous hdEEG/fMRI Imaging: A Nonlinear Dimensionality Reduction Approach

Simultaneous recordings of electroencephalogram (EEG) and functional magnetic resonance imaging (fMRI) are at the forefront of technologies of interest to physicians and scientists because they combine the benefits of both modalities—better time resolution (hdEEG) and space resolution (fMRI). However, EEG measurements in the scanner contain an electromagnetic field that is induced in leads as a result of gradient switching slight head movements and vibrations, and it is corrupted by changes in the measured potential because of the Hall phenomenon. The aim of this study is to design and test a methodology for inspecting hidden EEG structures with respect to artifacts. We propose a top-down strategy to obtain additional information that is not visible in a single recording. The time-domain independent component analysis algorithm was employed to obtain independent components and spatial weights. A nonlinear dimension reduction technique t-distributed stochastic neighbor embedding was used to create low-dimensional space, which was then partitioned using the density-based spatial clustering of applications with noise (DBSCAN). The relationships between the found data structure and the used criteria were investigated. As a result, we were able to extract information from the data structure regarding electrooculographic, electrocardiographic, electromyographic and gradient artifacts. This new methodology could facilitate the identification of artifacts and their residues from simultaneous EEG in fMRI.

The peripheral preview effect with faces: Combined EEG and eye-tracking suggests multiple stages of trans-saccadic predictive and non-predictive processing

The world appears stable despite saccadic eye-movements. One possible explanation for this phenomenon is that the visual system predicts upcoming input across saccadic eye-movements based on peripheral preview of the saccadic target. We tested this idea using concurrent electroencephalography (EEG) and eye-tracking. Participants made cued saccades to peripheral upright or inverted face stimuli that changed orientation (invalid preview) or maintained orientation (valid preview) while the saccade was completed. Experiment 1 demonstrated better discrimination performance and a reduced fixation-locked N170 component (fN170) with valid than with invalid preview, demonstrating integration of pre- and post-saccadic information. Moreover, the early fixation-related potentials (FRP) showed a preview face inversion effect suggesting that some pre-saccadic input was represented in the brain until around 170 ms post fixation-onset. Experiment 2 replicated Experiment 1 and manipulated the proportion of valid and invalid trials to test whether the preview effect reflects context-based prediction across trials. A whole-scalp Bayes factor analysis showed that this manipulation did not alter the fN170 preview effect but did influence the face inversion effect before the saccade. The pre-saccadic inversion effect declined earlier in the mostly invalid block than in the mostly valid block, which is consistent with the notion of pre-saccadic expectations. In addition, in both studies, we found strong evidence for an interaction between the pre-saccadic preview stimulus and the post-saccadic target as early as 50 ms (Experiment 2) or 90 ms (Experiment 1) into the new fixation. These findings suggest that visual stability may involve three temporal stages: prediction about the saccadic target, integration of pre-saccadic and post-saccadic information at around 50-90 ms post fixation onset, and post-saccadic facilitation of rapid categorization.

Co-registration of eye movements and neuroimaging for studying contextual predictions in natural reading

Sixteen years ago, Sereno and Rayner (2003. Measuring word recognition in reading: eye movements and event-related potentials. Trends in Cognitive Sciences, 7(11), 489-493) illustrated how "by means of review and comparison" eye movement (EM) and event-related potential (ERP) studies may advance our understanding of visual word recognition. Attempts to simultaneously record EMs and ERPs soon followed. Recently, this co-registration approach has also been transferred to fMRI and oscillatory EEG. With experimental settings close to natural reading, co-registration enables us to directly integrate insights from EM and neuroimaging studies. This should extend current experimental paradigms by moving the field towards studying sentence-level processing including effects of context and parafoveal preview. This article will introduce the basic principles and applications of co-registration and selectively review how this approach may shed light on one of the most controversially discussed issues in reading research, contextual predictions in online language processing.

Parafoveal previews and lexical frequency in natural reading: Evidence from eye movements and fixation-related potentials

Participants' eye movements and electroencephalogram (EEG) signal were recorded as they read sentences displayed according to the gaze-contingent boundary paradigm. Two target words in each sentence were manipulated for lexical frequency (high vs. low frequency) and parafoveal preview of each target word (identical vs. string of random letters vs. string of Xs). Eye movement data revealed visual parafoveal-on-foveal (PoF) effects, as well as foveal visual and orthographic preview effects and word frequency effects. Fixation-related potentials (FRPs) showed visual and orthographic PoF effects as well as foveal visual and orthographic preview effects. Our results replicated the early preview positivity effect (Dimigen, Kliegl, & Sommer, 2012) in the X-string preview condition, and revealed different neural correlates associated with a preview comprised of a string of random letters relative to a string of Xs. The former effects seem likely to reflect difficulty associated with the integration of parafoveal and foveal information, as well as feature overlap, while the latter reflect inhibition, and potentially disruption, to processing underlying reading. Interestingly, and consistent with Kretzschmar, Schlesewsky, and Staub (2015), no frequency effect was reflected in the FRP measures. The findings provide insight into the neural correlates of parafoveal processing and written word recognition in reading and demonstrate the value of utilizing ecologically valid paradigms to study well established phenomena that occur as text is read naturally. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Classification of Movement Intention Using Independent Components of Premovement EEG

Many previous studies on brain-machine interfaces (BMIs) have focused on electroencephalography (EEG) signals elicited during motor-command execution to generate device commands. However, exploiting pre-execution brain activity related to movement intention could improve the practical applicability of BMIs. Therefore, in this study we investigated whether EEG signals occurring before movement execution could be used to classify movement intention. Six subjects performed reaching tasks that required them to move a cursor to one of four targets distributed horizontally and vertically from the center. Using independent components of EEG acquired during a premovement phase, two-class classifications were performed for left vs. right trials and top vs. bottom trials using a support vector machine. Instructions were presented visually (test) and aurally (condition). In the test condition, accuracy for a single window was about 75%, and it increased to 85% in classification using two windows. In the control condition, accuracy for a single window was about 73%, and it increased to 80% in classification using two windows. Classification results showed that a combination of two windows from different time intervals during the premovement phase improved classification performance in the both conditions compared to a single window classification. By categorizing the independent components according to spatial pattern, we found that information depending on the modality can improve classification performance. We confirmed that EEG signals occurring during movement preparation can be used to control a BMI.

Detection Analysis of Epileptic EEG Using a Novel Random Forest Model Combined With Grid Search Optimization

In the automatic detection of epileptic seizures, the monitoring of critically ill patients with time varying EEG signals is an essential procedure in intensive care units. There is an increasing interest in using EEG analysis to detect seizure, and in this study we aim to get a better understanding of how to visualize the information in the EEG time-frequency feature, and design and train a novel random forest algorithm for EEG decoding, especially for multiple-levels of illness. Here, we propose an automatic detection framework for epileptic seizure based on multiple time-frequency analysis approaches; it involves a novel random forest model combined with grid search optimization. The short-time Fourier transformation visualizes seizure features after normalization. The dimensionality of features is reduced through principal component analysis before feeding them into the classification model. The training parameters are optimized using grid search optimization to improve detection performance and diagnostic accuracy by in the recognition of three different levels epileptic of conditions (healthy subjects, seizure-free intervals, seizure activity). Our proposed model was used to classify 500 samples of raw EEG data, and multiple cross-validations were adopted to boost the modeling accuracy. Experimental results were evaluated by an accuracy, a confusion matrix, a receiver operating characteristic curve, and an area under the curve. The evaluations indicated that our model achieved the more effective classification than some previous typical methods. Such a scheme for computer-assisted clinical diagnosis of seizures has a potential guiding significance, which not only relieves the suffering of patient with epilepsy to improve quality of life, but also helps neurologists reduce their workload.

Automated Recognition of Epileptic EEG States Using a Combination of Symlet Wavelet Processing, Gradient Boosting Machine, and Grid Search Optimizer

Automatic recognition methods for non-stationary electroencephalogram (EEG) data collected from EEG sensors play an essential role in neurological detection. The integrated approaches proposed in this study consist of Symlet wavelet processing, a gradient boosting machine, and a grid search optimizer for a three-class classification scheme for normal subjects, intermittent epilepsy, and continuous epilepsy. Fourth-order Symlet wavelets are adopted to decompose the EEG data into five frequencies sub-bands, such as gamma, beta, alpha, theta, and delta, whose statistical features were computed and used as classification features. The grid search optimizer is used to automatically find the optimal parameters for training the classifier. The classification accuracy of the gradient boosting machine was compared with that of a conventional support vector machine and a random forest classifier constructed according to previous descriptions. Multiple performance indices were used to evaluate the proposed classification scheme, which provided better classification accuracy and detection effectiveness than has been recently reported in other studies on three-class classification of EEG data.

Beta and Theta Oscillations Differentially Support Free Versus Forced Control over Multiple-Target Search

Many important situations require human observers to simultaneously search for more than one object. Despite a long history of research into visual search, the behavioral and neural mechanisms associated with multiple-target search are poorly understood. Here we test the novel theory that the efficiency of looking for multiple targets critically depends on the mode of cognitive control the environment affords to the observer. We used an innovative combination of electroencephalogram (EEG) and eye tracking while participants searched for two targets, within two different contexts: either both targets were present in the search display and observers were free to prioritize either one of them, thus enabling proactive control over selection; or only one of the two targets would be present in each search display, which requires reactive control to reconfigure selection when the wrong target has been prioritized. During proactive control, both univariate and multivariate signals of beta-band (15-35 Hz) power suppression before display onset predicted switches between target selections. This signal originated over midfrontal and sensorimotor regions and has previously been associated with endogenous state changes. In contrast, imposed target selections requiring reactive control elicited prefrontal power enhancements in the delta/theta band (2-8 Hz), but only after display onset. This signal predicted individual differences in associated oculomotor switch costs, reflecting reactive reconfiguration of target selection. The results provide compelling evidence that multiple target representations are differentially prioritized during visual search, and for the first time reveal distinct neural mechanisms underlying proactive and reactive control over multiple-target search.SIGNIFICANCE STATEMENT Searching for more than one object in complex visual scenes can be detrimental for search performance. Although perhaps annoying in daily life, this can have severe consequences in professional settings such as medical and security screening. Previous research has not yet resolved whether multiple-target search involves changing priorities in what people attend to, and how such changes are controlled. We approached these questions by concurrently measuring cortical activity and eye movements using EEG and eye tracking while observers searched for multiple possible targets. Our findings provide the first unequivocal support for the existence of two modes of control during multiple-target search, which are expressed in qualitatively distinct time-frequency signatures of the EEG both before and after visual selection.

Semantic anomaly detection in school-aged children during natural sentence reading - A study of fixation-related brain potentials

In this study, we investigated the effects of context-related semantic anomalies on the fixation-related brain potentials of 12-13-year-old Finnish children in grade 6 during sentence reading. The detection of such anomalies is typically reflected in the N400 event-related potential. We also examined whether the representation invoked by the sentence context extends to the orthographic representation level by replacing the final words of the sentence with an anomalous word neighbour of a plausible word. The eye-movement results show that the anomalous word neighbours of plausible words cause similar first-fixation and gaze duration reactions, as do other anomalous words. Similarly, we observed frontal negativity in the fixation-related potential of the unrelated anomalous words and in the anomalous word neighbours. This frontal negativity was larger in both anomalous conditions than in the response elicited by the plausible condition. We thus show that the brain successfully uses context to separate anomalous words from plausible words on a single letter level during free reading. From the P600 response of the scalp waveform, we observed that the P600 was delayed in the anomalous word neighbour condition. We performed group-level decomposition on the data with ICA (independent component analysis) and analysed the time course and source structure of the decomposed data. This analysis of decomposed brain signals not only confirmed the delay of the P600 response but also revealed that the frontal negativity concealed s more typical and separate N400 response, which was similarly delayed in the anomalous word neighbour condition, as was the P600 response. Source analysis of these independent components implicated the right frontal eye field as the cortical source for the frontal negativity and the middle temporal and parietal regions as cortical sources for the components resembling the N400 and P600 responses. We interpret the delays present in N400 and P600 responses to anomalous word neighbours to reflect competition with the representation of the plausible word just one letter different.

Improving free-viewing fixation-related EEG potentials with continuous-time regression

BACKGROUND

In the analysis of combined ET-EEG data, there are several issues with estimating FRPs by averaging. Neural responses associated with fixations will likely overlap with one another in the EEG recording and neural responses change as a function of eye movement characteristics. Especially in tasks that do not constrain eye movements in any way, these issues can become confounds.

NEW METHOD

Here, we propose the use of regression based estimates as an alternative to averaging. Multiple regression can disentangle different influences on the EEG and correct for overlap. It thereby accounts for potential confounds in a way that averaging cannot. Specifically, we test the applicability of the rERP framework, as proposed by Smith and Kutas (2015b), (2017), or Sassenhagen (2018) to combined eye tracking and EEG data from a visual search and a scene memorization task.

RESULTS

Results show that the method successfully estimates eye movement related confounds in real experimental data, so that these potential confounds can be accounted for when estimating experimental effects.

COMPARISON WITH EXISTING METHODS

The rERP method successfully corrects for overlapping neural responses in instances where averaging does not. As a consequence, baselining can be applied without risking distortions. By estimating a known experimental effect, we show that rERPs provide an estimate with less variance and more accuracy than averaged FRPs. The method therefore provides a practically feasible and favorable alternative to averaging.

CONCLUSIONS

We conclude that regression based ERPs provide novel opportunities for estimating fixation related EEG in free-viewing experiments.

Distraction by deviant sounds during reading: An eye-movement study

Oddball studies have shown that sounds unexpectedly deviating from an otherwise repeated sequence capture attention away from the task at hand. While such distraction is typically regarded as potentially important in everyday life, previous work has so far not examined how deviant sounds affect performance on more complex daily tasks. In this study, we developed a new method to examine whether deviant sounds can disrupt reading performance by recording participants’ eye movements. Participants read single sentences in silence and while listening to task-irrelevant sounds. In the latter condition, a 50-ms sound was played contingent on the fixation of five target words in the sentence. On most occasions, the same tone was presented (standard sound), whereas on rare and unexpected occasions it was replaced by white noise (deviant sound). The deviant sound resulted in significantly longer fixation durations on the target words relative to the standard sound. A time-course analysis showed that the deviant sound began to affect fixation durations around 180 ms after fixation onset. Furthermore, deviance distraction was not modulated by the lexical frequency of target words. In summary, fixation durations on the target words were longer immediately after the presentation of the deviant sound, but there was no evidence that it interfered with the lexical processing of these words. The present results are in line with the recent proposition that deviant sounds yield a temporary motor suppression and suggest that deviant sounds likely inhibit the programming of the next saccade.

Eye Movement-Related Confounds in Neural Decoding of Visual Working Memory Representations

A relatively new analysis technique, known as neural decoding or multivariate pattern analysis (MVPA), has become increasingly popular for cognitive neuroimaging studies over recent years. These techniques promise to uncover the representational contents of neural signals, as well as the underlying code and the dynamic profile thereof. A field in which these techniques have led to novel insights in particular is that of visual working memory (VWM). In the present study, we subjected human volunteers to a combined VWM/imagery task while recording their neural signals using magnetoencephalography (MEG). We applied multivariate decoding analyses to uncover the temporal profile underlying the neural representations of the memorized item. Analysis of gaze position however revealed that our results were contaminated by systematic eye movements, suggesting that the MEG decoding results from our originally planned analyses were confounded. In addition to the eye movement analyses, we also present the original analyses to highlight how these might have readily led to invalid conclusions. Finally, we demonstrate a potential remedy, whereby we train the decoders on a functional localizer that was specifically designed to target bottom-up sensory signals and as such avoids eye movements. We conclude by arguing for more awareness of the potentially pervasive and ubiquitous effects of eye movement-related confounds.

Electrooculogram and submandibular montage to distinguish different eye, eyelid, and tongue movements in electroencephalographic studies

OBJECTIVE

We propose an electrooculogram and submandibular montage that helps to discriminate eye/eyelid/tongue movements and to differentiate them from epileptiform activity or slowing on electroencephalography (EEG).

METHODS

We analyzed different eye/eyelid and tongue movements in 6 and 4 patients, respectively. Six peri-orbitally and one submandibular electrodes were placed. We referred these electrodes to an indifferent reference (Cz/Pz) and we recorded eye/eyelid and tongue movements simultaneously with the 10-20 system EEG. Additionally, we analyzed 2 seizures with the electrooculogram montage.

RESULTS

The electrooculogram deflections always showed an opposite phase direction when eye/eyelid movements occurred. Conversely, epileptiform activity produced deflections in the same phase direction in all electrooculogram electrodes. The electrooculogram montage was able to distinguish eye ictal semiology. Vertical tongue movements showed opposite phase deflections between the submandibular and the inferior ocular electrodes. Horizontal tongue movements revealed opposite phase reversal deflections between both inferior ocular electrodes.

CONCLUSIONS

The proposed montage accurately defines different eye/eyelid and tongue movements from brainwave activity. Additionally, it is useful to differentiate eye/eyelid movements from epileptiform activity and to characterize ictal ocular semiology, which can help localize or lateralize the epileptogenic zone.

SIGNIFICANCE

We propose this new montage to provide added value to prolonged video-EEG studies.

Target probability modulates fixation-related potentials in visual search

This study investigated the influence of target probability on the neural response to target detection in free viewing visual search. Participants were asked to indicate the number of targets (one or two) among distractors in a visual search task while EEG and eye movements were co-registered. Target probability was manipulated by varying the set size of the displays between 10, 22, and 30 items. Fixation-related potentials time-locked to first target fixations revealed a pronounced P300 at the centro-parietal cortex with larger amplitudes for set sizes 22 and 30 than for set size 10. With increasing set size, more distractor fixations preceded the detection of the target, resulting in a decreased target probability and, consequently, a larger P300. For distractors, no increase of P300 amplitude with set size was observed. The findings suggest that set size specifically affects target but not distractor processing in overt serial visual search.

An error-aware gaze-based keyboard by means of a hybrid BCI system

Gaze-based keyboards offer a flexible way for human-computer interaction in both disabled and able-bodied people. Besides their convenience, they still lead to error-prone human-computer interaction. Eye tracking devices may misinterpret user's gaze resulting in typesetting errors, especially when operated in fast mode. As a potential remedy, we present a novel error detection system that aggregates the decision from two distinct subsystems, each one dealing with disparate data streams. The first subsystem operates on gaze-related measurements and exploits the eye-transition pattern to flag a typo. The second, is a brain-computer interface that utilizes a neural response, known as Error-Related Potentials (ErrPs), which is inherently generated whenever the subject observes an erroneous action. Based on the experimental data gathered from 10 participants under a spontaneous typesetting scenario, we first demonstrate that ErrP-based Brain Computer Interfaces can be indeed useful in the context of gaze-based typesetting, despite the putative contamination of EEG activity from the eye-movement artefact. Then, we show that the performance of this subsystem can be further improved by considering also the error detection from the gaze-related subsystem. Finally, the proposed bimodal error detection system is shown to significantly reduce the typesetting time in a gaze-based keyboard.

Brain-computer interface combining eye saccade two-electrode EEG signals and voice cues to improve the maneuverability of wheelchair

Brain-computer interfaces (BCIs) largely augment human capabilities by translating brain wave signals into feasible commands to operate external devices. However, many issues face the development of BCIs such as the low classification accuracy of brain signals and the tedious human-learning procedures. To solve these problems, we propose to use signals associated with eye saccades and blinks to control a BCI interface. By extracting existing physiological eye signals, the user does not need to adapt his/her brain waves to the device. Furthermore, using saccade signals to control an external device frees the limbs to perform other tasks. In this research, we use two electrodes placed on top of the left and right ears of thirteen participants. Then we use Independent Component Analysis (ICA) to extract meaningful EEG signals associated with eye movements. A sliding-window technique was implemented to collect relevant features. Finally, we classified the features as horizontal or blink eye movements using KNN and SVM. We were able to achieve a mean classification accuracy of about 97%. The two electrodes were then integrated with off-the-shelf earbuds to control a wheelchair. The earbuds can generate voice cues to indicate when to rotate the eyeballs to certain locations (i.e., left or right) or blink, so that the user can select directional commands to drive the wheelchair. In addition, through properly designing the contents of voice menus, we can generate as many commands as possible, even though we only have limited numbers of states of the identified eye saccade movements.

The effect of panic disorder versus anxiety sensitivity on event-related potentials during anticipation of threat

Attention-related abnormalities are key components of the abnormal defensive responding observed in panic disorder (PD). Although behavioral studies have found aberrant attentional biases towards threat in PD, psychophysiological studies have been mixed. Predictability of threat, an important feature of threat processing, may have contributed to these mixed findings. Additionally, anxiety sensitivity, a dimensional trait associated with PD, may yield stronger associations with cognitive processes than categorical diagnoses of PD. In this study, 171 participants with PD and/or depression and healthy controls completed a task that differentiated anticipation of predictable vs. unpredictable shocks, while startle eyeblink and event-related potentials (ERPs [N100, P300]) were recorded. In all participants, relative to the control condition, probe N100 was enhanced to both predictable and unpredictable threat, whereas P300 suppression was unique to predictable threat. Probe N100, but not P300, was associated with startle eyeblink during both threatening conditions, and was strongest for unpredictable threat. PD was not associated with ERPs, but anxiety sensitivity (physical concerns) was positively associated with probe N100 (indicating reduced responding) in the unpredictable condition independent of PD diagnosis. Vulnerability to panic-related psychopathology may be characterized by aberrant early processing of threat, which may be especially evident during anticipation of unpredictable threats.

Sociality Mental Modes Modulate the Processing of Advice-Giving: An Event-Related Potentials Study

People have different motivations to get along with others in different sociality mental modes (i.e., communal mode and market mode), which might affect social decision-making. The present study examined how these two types of sociality mental modes affect the processing of advice-giving using the event-related potentials (ERPs). After primed with the communal mode and market mode, participants were instructed to decide whether or not give an advice (profitable or damnous) to a stranger without any feedback. The behavioral results showed that participants preferred to give the profitable advice to the stranger more slowly compared with the damnous advice, but this difference was only observed in the market mode condition. The ERP results indicated that participants demonstrated more negative N1 amplitude for the damnous advice compared with the profitable advice, and larger P300 was elicited in the market mode relative to both the communal mode and the control group. More importantly, participants in the market mode demonstrated larger P300 for the profitable advice than the damnous advice, whereas this difference was not observed at the communal mode and the control group. These findings are consistent with the dual-process system during decision-making and suggest that market mode may lead to deliberate calculation for costs and benefits when giving the profitable advice to others.

Large-Scale Traveling Waves in EEG Activity Following Eye Movement

In spontaneous, stimulus-evoked, and eye-movement evoked EEG, the oscillatory signal shows large scale, dynamically organized patterns of phase. We investigated eye-movement evoked patterns in free-viewing conditions. Participants viewed photographs of natural scenes in anticipation of a memory test. From 200 ms intervals following saccades, we estimated the EEG phase gradient over the entire scalp, and the wave activity, i.e. the goodness of fit of a wave model involving a phase gradient assumed to be smooth over the scalp. In frequencies centered at 6.5 Hz, large-scale phase organization occurred, peaking around 70 ms after fixation onset and taking the form of a traveling wave. According to the wave gradient, most of the times the wave spreads from the posterior-inferior to anterior-superior direction. In these directions, the gradients depended on the size and direction of the saccade. Wave propagation velocity decreased in the course of the fixation, particularly in the interval from 50 to 150 ms after fixation onset. This interval corresponds to the fixation-related lambda activity, which reflects early perceptual processes following fixation onset. We conclude that lambda activity has a prominent traveling wave component. This component consists of a short-term whole-head phase pattern of specific direction and velocity, which may reflect feedforward propagation of visual information at fixation.

Spectral fingerprints of large-scale cortical dynamics during ambiguous motion perception

Ambiguous stimuli have been widely used to study the neuronal correlates of consciousness. Recently, it has been suggested that conscious perception might arise from the dynamic interplay of functionally specialized but widely distributed cortical areas. While previous research mainly focused on phase coupling as a correlate of cortical communication, more recent findings indicated that additional coupling modes might coexist and possibly subserve distinct cortical functions. Here, we studied two coupling modes, namely phase and envelope coupling, which might differ in their origins, putative functions and dynamics. Therefore, we recorded 128-channel EEG while participants performed a bistable motion task and utilized state-of-the-art source-space connectivity analysis techniques to study the functional relevance of different coupling modes for cortical communication. Our results indicate that gamma-band phase coupling in extrastriate visual cortex might mediate the integration of visual tokens into a moving stimulus during ambiguous visual stimulation. Furthermore, our results suggest that long-range fronto-occipital gamma-band envelope coupling sustains the horizontal percept during ambiguous motion perception. Additionally, our results support the idea that local parieto-occipital alpha-band phase coupling controls the inter-hemispheric information transfer. These findings provide correlative evidence for the notion that synchronized oscillatory brain activity reflects the processing of sensory input as well as the information integration across several spatiotemporal scales. The results indicate that distinct coupling modes are involved in different cortical computations and that the rich spatiotemporal correlation structure of the brain might constitute the functional architecture for cortical processing and specific multi-site communication. Hum Brain Mapp 37:4099-4111, 2016. © 2016 Wiley Periodicals, Inc.

Electroencephalographic Correlates of Suicidal Ideation in the Theta Band

Previous studies have indicated that the abnormalities of brain activity are associated with suicidal behavior. However, few studies have explored neurophysiologic biomarkers of suicidal ideation (SI) in healthy subjects. Here, we studied alterations of theta quantitative EEG (QEEG) in frontal regions associated with SI in young healthy subjects. Resting QEEG data were recorded from 90 young healthy subjects. The subjects were divided into two group based on Scale for Suicidal Ideation (SSI) scores: a high SSI group (n = 33) and a low SSI group (n = 57). Individual frontal electrodes (Fp1, Fp2, F7, F3, Fz, F4, F8) and central midline electrodes (FCz, Cz) were examined for absolute and relative power in the theta band. Clinical state and social support was assessed using the Hospital Anxiety and Depression Scale and Functional Social Support Questionnaire. We found that theta absolute and relative power in all channels was positively correlated with SSI, except Fp1, F7 and F8. The high SSI group showed higher theta relative power at F3, Fz, FCz, and Cz than the low SSI group. Theta relative power in the fronto-central region was significantly increased in the high SSI group compared to that in the low SSI group. Hierarchical regression analysis showed that SSI score was the most powerful predictor of fronto-central theta power. The findings suggest that brain electrical activity at the fronto-central region may be associated with differences in SI in young healthy subjects. EEG activity in theta band has clinical potential as a biomarker for preventing suicide.

Task modulates functional connectivity networks in free viewing behavior

In free visual exploration, eye-movement is immediately followed by dynamic reconfiguration of brain functional connectivity. We studied the task-dependency of this process in a combined visual search-change detection experiment. Participants viewed two (nearly) same displays in succession. First time they had to find and remember multiple targets among distractors, so the ongoing task involved memory encoding. Second time they had to determine if a target had changed in orientation, so the ongoing task involved memory retrieval. From multichannel EEG recorded during 200 ms intervals time-locked to fixation onsets, we estimated the functional connectivity using a weighted phase lag index at the frequencies of theta, alpha, and beta bands, and derived global and local measures of the functional connectivity graphs. We found differences between both memory task conditions for several network measures, such as mean path length, radius, diameter, closeness and eccentricity, mainly in the alpha band. Both the local and the global measures indicated that encoding involved a more segregated mode of operation than retrieval. These differences arose immediately after fixation onset and persisted for the entire duration of the lambda complex, an evoked potential commonly associated with early visual perception. We concluded that encoding and retrieval differentially shape network configurations involved in early visual perception, affecting the way the visual input is processed at each fixation. These findings demonstrate that task requirements dynamically control the functional connectivity networks involved in early visual perception.

10-Month-Old Infants Are Sensitive to the Time Course of Perceived Actions: Eye-Tracking and EEG Evidence

Research has shown that infants are able to track a moving target efficiently - even if it is transiently occluded from sight. This basic ability allows prediction of when and where events happen in everyday life. Yet, it is unclear whether, and how, infants internally represent the time course of ongoing movements to derive predictions. In this study, 10-month-old crawlers observed the video of a same-aged crawling baby that was transiently occluded and reappeared in either a temporally continuous or non-continuous manner (i.e., delayed by 500 ms vs. forwarded by 500 ms relative to the real-time movement). Eye movement and rhythmic neural brain activity (EEG) were measured simultaneously. Eye movement analyses showed that infants were sensitive to slight temporal shifts in movement continuation after occlusion. Furthermore, brain activity associated with sensorimotor processing differed between observation of continuous and non-continuous movements. Early sensitivity to an action's timing may hence be explained within the internal real-time simulation account of action observation. Overall, the results support the hypothesis that 10-month-old infants are well prepared for internal representation of the time course of observed movements that are within the infants' current motor repertoire.

Isolating Discriminant Neural Activity in the Presence of Eye Movements and Concurrent Task Demands

A growing number of studies use the combination of eye-tracking and electroencephalographic (EEG) measures to explore the neural processes that underlie visual perception. In these studies, fixation-related potentials (FRPs) are commonly used to quantify early and late stages of visual processing that follow the onset of each fixation. However, FRPs reflect a mixture of bottom-up (sensory-driven) and top-down (goal-directed) processes, in addition to eye movement artifacts and unrelated neural activity. At present there is little consensus on how to separate this evoked response into its constituent elements. In this study we sought to isolate the neural sources of target detection in the presence of eye movements and over a range of concurrent task demands. Here, participants were asked to identify visual targets (Ts) amongst a grid of distractor stimuli (Ls), while simultaneously performing an auditory N-back task. To identify the discriminant activity, we used independent components analysis (ICA) for the separation of EEG into neural and non-neural sources. We then further separated the neural sources, using a modified measure-projection approach, into six regions of interest (ROIs): occipital, fusiform, temporal, parietal, cingulate, and frontal cortices. Using activity from these ROIs, we identified target from non-target fixations in all participants at a level similar to other state-of-the-art classification techniques. Importantly, we isolated the time course and spectral features of this discriminant activity in each ROI. In addition, we were able to quantify the effect of cognitive load on both fixation-locked potential and classification performance across regions. Together, our results show the utility of a measure-projection approach for separating task-relevant neural activity into meaningful ROIs within more complex contexts that include eye movements.

Dynamic spatiotemporal variability of alpha-BOLD relationships during the resting-state and task-evoked responses

Accurate characterization of the spatiotemporal relationship between two of the most prominent neuroimaging measures of neuronal activity, the 8-13Hz, occipito-parietal EEG alpha oscillation and the BOLD fMRI signal, must encompass the intrinsically dynamic nature of both alpha power and brain function. Here, during the eyes-open resting state, we use a 16s sliding-window analysis and demonstrate that the mean spatial network of dynamic alpha-BOLD correlations is highly comparable to the static network calculated over six minutes. However, alpha-BOLD correlations showed substantial spatiotemporal variability within-subjects and passed through many different configurations such that the static network was fully represented in only ~10% of 16s epochs, with visual and parietal regions (coherent on average) often opposingly correlated with each other or with alpha. We find that the common assumption of static-alpha BOLD correlations greatly oversimplifies temporal variation in brain network dynamics. Fluctuations in alpha-BOLD coupling significantly depended upon the instantaneous amplitude of alpha power, and primary and lateral visual areas were most strongly negatively correlated with alpha during different alpha power states, possibly suggesting the action of multiple alpha mechanisms. Dynamic alpha-BOLD correlations could not be explained by eye-blinks/movements, head motion or non-neuronal physiological variability. Individual's mean alpha power and frequency were found to contribute to between-subject variability in alpha-BOLD correlations. Additionally, application to a visual stimulation dataset showed that dynamic alpha-BOLD correlations provided functional information pertaining to the brain's response to stimulation by exhibiting spatiotemporal fluctuations related to variability in the trial-by-trial BOLD response magnitude. Significantly weaker visual alpha-BOLD correlations were found both preceding and following small amplitude BOLD response trials compared to large response trials.

Concealed, Unobtrusive Ear-Centered EEG Acquisition: cEEGrids for Transparent EEG

Electroencephalography (EEG) is an important clinical tool and frequently used to study the brain-behavior relationship in humans noninvasively. Traditionally, EEG signals are recorded by positioning electrodes on the scalp and keeping them in place with glue, rubber bands, or elastic caps. This setup provides good coverage of the head, but is impractical for EEG acquisition in natural daily-life situations. Here, we propose the transparent EEG concept. Transparent EEG aims for motion tolerant, highly portable, unobtrusive, and near invisible data acquisition with minimum disturbance of a user's daily activities. In recent years several ear-centered EEG solutions that are compatible with the transparent EEG concept have been presented. We discuss work showing that miniature electrodes placed in and around the human ear are a feasible solution, as they are sensitive enough to pick up electrical signals stemming from various brain and non-brain sources. We also describe the cEEGrid flex-printed sensor array, which enables unobtrusive multi-channel EEG acquisition from around the ear. In a number of validation studies we found that the cEEGrid enables the recording of meaningful continuous EEG, event-related potentials and neural oscillations. Here, we explain the rationale underlying the cEEGrid ear-EEG solution, present possible use cases and identify open issues that need to be solved on the way toward transparent EEG.

BLINKER: Automated Extraction of Ocular Indices from EEG Enabling Large-Scale Analysis

Electroencephalography (EEG) offers a platform for studying the relationships between behavioral measures, such as blink rate and duration, with neural correlates of fatigue and attention, such as theta and alpha band power. Further, the existence of EEG studies covering a variety of subjects and tasks provides opportunities for the community to better characterize variability of these measures across tasks and subjects. We have implemented an automated pipeline (BLINKER) for extracting ocular indices such as blink rate, blink duration, and blink velocity-amplitude ratios from EEG channels, EOG channels, and/or independent components (ICs). To illustrate the use of our approach, we have applied the pipeline to a large corpus of EEG data (comprising more than 2000 datasets acquired at eight different laboratories) in order to characterize variability of certain ocular indicators across subjects. We also investigate dependence of ocular indices on task in a shooter study. We have implemented our algorithms in a freely available MATLAB toolbox called BLINKER. The toolbox, which is easy to use and can be applied to collections of data without user intervention, can automatically discover which channels or ICs capture blinks. The tools extract blinks, calculate common ocular indices, generate a report for each dataset, dump labeled images of the individual blinks, and provide summary statistics across collections. Users can run BLINKER as a script or as a plugin for EEGLAB. The toolbox is available at https://github.com/VisLab/EEG-Blinks. User documentation and examples appear at http://vislab.github.io/EEG-Blinks/.

A Removal of Eye Movement and Blink Artifacts from EEG Data Using Morphological Component Analysis

EEG signals contain a large amount of ocular artifacts with different time-frequency properties mixing together in EEGs of interest. The artifact removal has been substantially dealt with by existing decomposition methods known as PCA and ICA based on the orthogonality of signal vectors or statistical independence of signal components. We focused on the signal morphology and proposed a systematic decomposition method to identify the type of signal components on the basis of sparsity in the time-frequency domain based on Morphological Component Analysis (MCA), which provides a way of reconstruction that guarantees accuracy in reconstruction by using multiple bases in accordance with the concept of “dictionary.” MCA was applied to decompose the real EEG signal and clarified the best combination of dictionaries for this purpose. In our proposed semirealistic biological signal analysis with iEEGs recorded from the brain intracranially, those signals were successfully decomposed into original types by a linear expansion of waveforms, such as redundant transforms: UDWT, DCT, LDCT, DST, and DIRAC. Our result demonstrated that the most suitable combination for EEG data analysis was UDWT, DST, and DIRAC to represent the baseline envelope, multifrequency wave-forms, and spiking activities individually as representative types of EEG morphologies.

Neural Differences between Covert and Overt Attention Studied using EEG with Simultaneous Remote Eye Tracking

Research on neural mechanisms of attention has generally instructed subjects to direct attention covertly while maintaining a fixed gaze. This study combined simultaneous eye tracking and electroencephalogram (EEG) to measure neural attention responses during exogenous cueing in overt attention shifts (with saccadic eye movements to a target) and compared these with covert attention shifts (responding manually while maintaining central fixation). EEG analysis of the period preceding the saccade latency showed similar occipital response amplitudes for overt and covert shifts, although response latencies differed. However, a frontal positivity was greater during covert attention shifts, possibly reflecting saccade inhibition to maintain fixation. The results show that combined EEG and eye tracking can be successfully used to study natural overt shifts of attention (applicable to non-verbal infants) and that requiring inhibition of saccades can lead to additional frontal responses. Such data can be used to refine current neural models of attention that have been mainly based on covert shifts.

Representational Dynamics of Facial Viewpoint Encoding

Faces provide a wealth of information, including the identity of the seen person and social cues, such as the direction of gaze. Crucially, different aspects of face processing require distinct forms of information encoding. Another person's attentional focus can be derived based on a view-dependent code. In contrast, identification benefits from invariance across all viewpoints. Different cortical areas have been suggested to subserve these distinct functions. However, little is known about the temporal aspects of differential viewpoint encoding in the human brain. Here, we combine EEG with multivariate data analyses to resolve the dynamics of face processing with high temporal resolution. This revealed a distinct sequence of viewpoint encoding. Head orientations were encoded first, starting after around 60 msec of processing. Shortly afterward, peaking around 115 msec after stimulus onset, a different encoding scheme emerged. At this latency, mirror-symmetric viewing angles elicited highly similar cortical responses. Finally, about 280 msec after visual onset, EEG response patterns demonstrated a considerable degree of viewpoint invariance across all viewpoints tested, with the noteworthy exception of the front-facing view. Taken together, our results indicate that the processing of facial viewpoints follows a temporal sequence of encoding schemes, potentially mirroring different levels of computational complexity.

Analysing concurrent transcranial magnetic stimulation and electroencephalographic data: A review and introduction to the open-source TESA software

The concurrent use of transcranial magnetic stimulation with electroencephalography (TMS-EEG) is growing in popularity as a method for assessing various cortical properties such as excitability, oscillations and connectivity. However, this combination of methods is technically challenging, resulting in artifacts both during recording and following typical EEG analysis methods, which can distort the underlying neural signal. In this article, we review the causes of artifacts in EEG recordings resulting from TMS, as well as artifacts introduced during analysis (e.g. as the result of filtering over high-frequency, large amplitude artifacts). We then discuss methods for removing artifacts, and ways of designing pipelines to minimise analysis-related artifacts. Finally, we introduce the TMS-EEG signal analyser (TESA), an open-source extension for EEGLAB, which includes functions that are specific for TMS-EEG analysis, such as removing and interpolating the TMS pulse artifact, removing and minimising TMS-evoked muscle activity, and analysing TMS-evoked potentials. The aims of TESA are to provide users with easy access to current TMS-EEG analysis methods and to encourage direct comparisons of these methods and pipelines. It is hoped that providing open-source functions will aid in both improving and standardising analysis across the field of TMS-EEG research.

Method for Improving EEG Based Emotion Recognition by Combining It with Synchronized Biometric and Eye Tracking Technologies in a Non-invasive and Low Cost Way

Technical advances, particularly the integration of wearable and embedded sensors, facilitate tracking of physiological responses in a less intrusive way. Currently, there are many devices that allow gathering biometric measurements from human beings, such as EEG Headsets or Health Bracelets. The massive data sets generated by tracking of EEG and physiology may be used, among other things, to infer knowledge about human moods and emotions. Apart from direct biometric signal measurement, eye tracking systems are nowadays capable of determining the point of gaze of the users when interacting in ICT environments, which provides an added value research on many different areas, such as psychology or marketing. We present a process in which devices for eye tracking, biometric, and EEG signal measurements are synchronously used for studying both basic and complex emotions. We selected the least intrusive devices for different signal data collection given the study requirements and cost constraints, so users would behave in the most natural way possible. On the one hand, we have been able to determine basic emotions participants were experiencing by means of valence and arousal. On the other hand, a complex emotion such as empathy has also been detected. To validate the usefulness of this approach, a study involving forty-four people has been carried out, where they were exposed to a series of affective stimuli while their EEG activity, biometric signals, and eye position were synchronously recorded to detect self-regulation. The hypothesis of the work was that people who self-regulated would show significantly different results when analyzing their EEG data. Participants were divided into two groups depending on whether Electro Dermal Activity (EDA) data indicated they self-regulated or not. The comparison of the results obtained using different machine learning algorithms for emotion recognition shows that using EEG activity alone as a predictor for self-regulation does not allow properly determining whether a person in self-regulation its emotions while watching affective stimuli. However, adequately combining different data sources in a synchronous way to detect emotions makes it possible to overcome the limitations of single detection methods.

Comparing the Performance of Popular MEG/EEG Artifact Correction Methods in an Evoked-Response Study

We here compared results achieved by applying popular methods for reducing artifacts in magnetoencephalography (MEG) and electroencephalography (EEG) recordings of the auditory evoked Mismatch Negativity (MMN) responses in healthy adult subjects. We compared the Signal Space Separation (SSS) and temporal SSS (tSSS) methods for reducing noise from external and nearby sources. Our results showed that tSSS reduces the interference level more reliably than plain SSS, particularly for MEG gradiometers, also for healthy subjects not wearing strongly interfering magnetic material. Therefore, tSSS is recommended over SSS. Furthermore, we found that better artifact correction is achieved by applying Independent Component Analysis (ICA) in comparison to Signal Space Projection (SSP). Although SSP reduces the baseline noise level more than ICA, SSP also significantly reduces the signal—slightly more than it reduces the artifacts interfering with the signal. However, ICA also adds noise, or correction errors, to the waveform when the signal-to-noise ratio (SNR) in the original data is relatively low—in particular to EEG and to MEG magnetometer data. In conclusion, ICA is recommended over SSP, but one should be careful when applying ICA to reduce artifacts on neurophysiological data with relatively low SNR.

Emotional interference modulates performance monitoring in patients with obsessive-compulsive disorder

BACKGROUND

Enhanced performance monitoring in patients with obsessive-compulsive disorder (OCD), typically measured by error-related negativity (ERN), provides evidence for the fronto-striatal model of OCD. Here, we examined whether performance monitoring in OCD patients is modulated by emotional interference induced by task-irrelevant emotional stimuli.

METHODS

A modified version of the flanker task with emotional face stimuli (fearful vs. neutral faces) was performed by 22 OCD patients and 22 healthy control subjects while electroencephalogram signals were recorded. Response-locked ERN was defined as the mean amplitude from 20 to 120msec after the response.

RESULTS

During trials with fearful face stimuli, OCD patients showed larger ERN amplitude than control subjects, but there was no difference between groups during trials with neutral face stimuli. Whereas OCD patients exhibited enhanced ERN amplitude in the fearful face condition compared with the neutral face condition, control subjects showed no variation between conditions. OCD patients also exhibited larger correct response negativity amplitude than control subjects in both fearful and neutral face conditions.

CONCLUSIONS

These results support the theory that OCD involves overactive performance monitoring and indicate that emotional interference modulates performance monitoring in patients with OCD, thus implying that affective function in the fronto-striatal network forms part of the neural basis of OCD.

The Impact of Task Demands on Fixation-Related Brain Potentials during Guided Search

Recording synchronous data from EEG and eye-tracking provides a unique methodological approach for measuring the sensory and cognitive processes of overt visual search. Using this approach we obtained fixation related potentials (FRPs) during a guided visual search task specifically focusing on the lambda and P3 components. An outstanding question is whether the lambda and P3 FRP components are influenced by concurrent task demands. We addressed this question by obtaining simultaneous eye-movement and electroencephalographic (EEG) measures during a guided visual search task while parametrically modulating working memory load using an auditory N-back task. Participants performed the guided search task alone, while ignoring binaurally presented digits, or while using the auditory information in a 0, 1, or 2-back task. The results showed increased reaction time and decreased accuracy in both the visual search and N-back tasks as a function of auditory load. Moreover, high auditory task demands increased the P3 but not the lambda latency while the amplitude of both lambda and P3 was reduced during high auditory task demands. The results show that both early and late stages of visual processing indexed by FRPs are significantly affected by concurrent task demands imposed by auditory working memory.