Combining EEG and eye tracking: identification, characterization, and correction of eye movement artifacts in electroencephalographic data

Visual search training benefits from the integrative effect of enhanced covert attention and optimized overt eye movements

Training serves as an effective approach to improve visual search performance when the target does not automatically pop out from the distractors. In the present study, we trained participants on a conjunction visual search task and examined the training effects in behavior and eye movement. The results of Experiments 1 to 4 showed that training improved behavioral performance and reduced the number of saccades and overall scanning time. Training also increased the search initiation time before the first saccade and the proportion of trials in which the participants correctly identified the target without any saccade, but these effects were modulated by stimulus’ parameters. In Experiment 5, we simultaneously recorded eye movements and electroencephalography signals and the results revealed significant N2 posterior contralateral (N2pc) components after the stimulus onset (i.e., stimulus-locked) and before the first saccade (i.e., saccade-locked) when the search target was the trained one. These N2pc components can be considered as the neural signatures for the enhanced covert attention to the trained target. Together with the training-induced increase in functional visual field, these mechanisms could support the beneficial effects of increased search initiation time and reduced number of saccades. These findings suggest that visual search training enhanced covert attention to target and optimized overt eye movements to facilitate search performance.

Recommendations and publication guidelines for studies using frequency domain and time-frequency domain analyses of neural time series

Since its beginnings in the early 20th century, the psychophysiological study of human brain function has included research into the spectral properties of electrical and magnetic brain signals. Now, dramatic advances in digital signal processing, biophysics, and computer science have enabled increasingly sophisticated methodology for neural time series analysis. Innovations in hardware and recording techniques have further expanded the range of tools available to researchers interested in measuring, quantifying, modeling, and altering the spectral properties of neural time series. These tools are increasingly used in the field, by a growing number of researchers who vary in their training, background, and research interests. Implementation and reporting standards also vary greatly in the published literature, causing challenges for authors, readers, reviewers, and editors alike. The present report addresses this issue by providing recommendations for the use of these methods, with a focus on foundational aspects of frequency domain and time-frequency analyses. It also provides publication guidelines, which aim to (1) foster replication and scientific rigor, (2) assist new researchers who wish to enter the field of brain oscillations, and (3) facilitate communication among authors, reviewers, and editors.

Naturalistic viewing conditions can increase task engagement and aesthetic preference but have only minimal impact on EEG quality

Free gaze and moving images are typically avoided in EEG experiments due to the expected generation of artifacts and noise. Yet for a growing number of research questions, loosening these rigorous restrictions would be beneficial. Among these is research on visual aesthetic experiences, which often involve open-ended exploration of highly variable stimuli. Here we systematically compare the effect of conservative vs. more liberal experimental settings on various measures of behavior, brain activity and physiology in an aesthetic rating task. Our primary aim was to assess EEG signal quality. 43 participants either maintained fixation or were allowed to gaze freely, and viewed either static images or dynamic (video) stimuli consisting of dance performances or nature scenes. A passive auditory background task (auditory steady-state response; ASSR) was added as a proxy measure for overall EEG recording quality. We recorded EEG, ECG and eyetracking data, and participants rated their aesthetic preference and state of boredom on each trial. Whereas both behavioral ratings and gaze behavior were affected by task and stimulus manipulations, EEG SNR was barely affected and generally robust across all conditions, despite only minimal preprocessing and no trial rejection. In particular, we show that using video stimuli does not necessarily result in lower EEG quality and can, on the contrary, significantly reduce eye movements while increasing both the participants' aesthetic response and general task engagement. We see these as encouraging results indicating that - at least in the lab - more liberal experimental conditions can be adopted without significant loss of signal quality.

Lucid dreaming occurs in activated rapid eye movement sleep, not a mixture of sleep and wakefulness

STUDY OBJECTIVES

(1) To critically test whether a previously reported increase in frontolateral 40 Hz power in lucid REM sleep, used to justify the claim that lucid dreaming is a "hybrid state" mixing sleep and wakefulness, is attributable to the saccadic spike potential (SP) artifact as a corollary of heightened REM density. (2) To replicate the finding that lucid dreams are associated with physiological activation, including heightened eye movement density, during REM sleep. (3) To conduct an exploratory analysis of changes in EEG features during lucid REM sleep.

METHODS

We analyzed 14 signal-verified lucid dreams (SVLDs) and baseline REM sleep segments from the same REM periods from six participants derived from the Stanford SVLD database. Participants marked lucidity onset with standard left-right-left-right-center (LR2c) eye-movement signals in polysomnography recordings.

RESULTS

Compared to baseline REM sleep, lucid REM sleep had higher REM density (β = 0.85, p = 0.002). Bayesian analysis supported the null hypothesis of no differences in frontolateral 40 Hz power after removal of the SP artifact (BH = 0.18) and ICA correction (BH = 0.01). Compared to the entire REM sleep period, lucid REM sleep showed small reductions in low-frequency and beta band spectral power as well as increased signal complexity (all p < 0.05), which were within the normal variance of baseline REM sleep.

CONCLUSIONS

Lucid dreams are associated with higher-than-average levels of physiological activation during REM sleep, including measures of both subcortical and cortical activation. Increases in 40 Hz power in periorbital channels reflect saccadic and microsaccadic SPs as a result of higher REM density accompanying heightened activation.

Distinct aspects of emotion dysregulation differentially correspond to magnitude and slope of the late positive potential to affective stimuli

ABSTRACTEmotion dysregulation (ED) is a transdiagnostic risk factor for several forms of psychopathology. One established, integrative conceptualisation of ED that has informed our understanding of psychopathology (Gratz, K. L., & Roemer, L. (2004). Multidimensional assessment of emotion regulation and dysregulation : development, factor structure, and initial validation of the difficulties in emotion regulation scale. Journal of Psychopathology and Behavioral Assessment, 26(1), 41-54) was recently extended to account for state-level and specifically physiological aspects of ED. As such, the goal of the current study was to assess the degree to which this conceptualisation could be captured using a physiological, state-level index (i.e. the late positive potential; LPP). Participants (N=41; M_age=27.5 years, SD=11.0) completed a trait-based rating scale measure of the extended conceptualisation of ED and viewed a series of images that ranged in valence (i.e. unpleasant, pleasant, and neutral). Multilevel modelling indicated some points of convergence between rating scale scores and the LPP, and other points of divergence. Findings underscore the utility of a multi-method approach to improve understanding of key transdiagnostic characteristics across levels of analysis. Further, results are novel evidence supporting validity of the extended conceptualisation of ED.

Handling EEG artifacts and searching individually optimal experimental parameter in real time: a system development and demonstration

Objective.Neuroadaptive paradigms that systematically assess event-related potential (ERP) features across many different experimental parameters have the potential to improve the generalizability of ERP findings and may help to accelerate ERP-based biomarker discovery by identifying the exact experimental conditions for which ERPs differ most for a certain clinical population. Obtaining robust and reliable ERPs online is a prerequisite for ERP-based neuroadaptive research. One of the key steps involved is to correctly isolate electroencephalography artifacts in real time because they contribute a large amount of variance that, if not removed, will greatly distort the ERP obtained. Another key factor of concern is the computational cost of the online artifact handling method. This work aims to develop and validate a cost-efficient system to support ERP-based neuroadaptive research.Approach.We developed a simple online artifact handling method, single trial PCA-based artifact removal (SPA), based on variance distribution dichotomies to distinguish between artifacts and neural activity. We then applied this method in an ERP-based neuroadaptive paradigm in which Bayesian optimization was used to search individually optimal inter-stimulus-interval (ISI) that generates ERP with the highest signal-to-noise ratio.Main results.SPA was compared to other offline and online algorithms. The results showed that SPA exhibited good performance in both computational efficiency and preservation of ERP pattern. Based on SPA, the Bayesian optimization procedure was able to quickly find individually optimal ISI.Significance.The current work presents a simple yet highly cost-efficient method that has been validated in its ability to extract ERP, preserve ERP effects, and better support ERP-based neuroadaptive paradigm.

Measuring the temporal dynamics of inter-personal neural entrainment in continuous child-adult EEG hyperscanning data

Current approaches to analysing EEG hyperscanning data in the developmental literature typically consider interpersonal entrainment between interacting physiological systems as a time-invariant property. This approach obscures crucial information about how entrainment between interacting systems is established and maintained over time. Here, we describe methods, and present computational algorithms, that will allow researchers to address this gap in the literature. We focus on how two different approaches to measuring entrainment, namely concurrent (e.g., power correlations, phase locking) and sequential (e.g., Granger causality) measures, can be applied to three aspects of the brain signal: amplitude, power, and phase. We guide the reader through worked examples using simulated data on how to leverage these methods to measure changes in interbrain entrainment. For each, we aim to provide a detailed explanation of the interpretation and application of these analyses when studying neural entrainment during early social interactions.

Electrocortical Activity in Older Adults Is More Influenced by Cognitive Task Complexity Than Concurrent Walking

Human cognitive-motor performance largely depends on how brain resources are allocated during simultaneous tasks. Nonetheless, little is known regarding the age-related changes in electrocortical activity when dual-task during walking presents higher complexity levels. Thus, the aim of this study was to investigate whether there are distinct changes in walking performance and electrocortical activation between young and older adults performing simple and complex upper limb response time tasks. Physically active young (23 ± 3 years, n = 21) and older adults (69 ± 5 years, n = 19) were asked to respond as fast as possible to a single stimuli or a double stimuli appearing on a touch screen during standing and walking. Response time, step frequency, step frequency variability and electroencephalographic (EEG) N200 and P300 amplitudes and latencies from frontal central and parietal brain regions were recorded. The results demonstrated that older adults were 23% slower to respond to double stimuli, whereas younger adults were only 12% slower (p < 0.01). The longer response time for older adults was accompanied by greater step frequency variability following double-stimuli presentations (p < 0.01). Older adults presented reduced N200 and P300 amplitudes compared to younger participants across all conditions (p < 0.001), with no effects of posture (standing vs walking) on both groups (p > 0.05). More importantly, the P300 amplitude was significantly reduced for older adults when responding to double stimuli regardless of standing or walking tasks (p < 0.05), with no changes in younger participants. Therefore, physically active older adults can attenuate potential walking deficits experienced during dual-task walking in simple cognitive tasks. However, cognitive tasks involving decision making influence electrocortical activation due to reduced cognitive resources to cope with the task demands.

Theta oscillations: A rhythm difference comparison between major depressive disorder and anxiety disorder

BACKGROUND

Due to substantial comorbidities of major depressive disorder (MDD) and anxiety disorder (AN), these two disorders must be distinguished. Accurate identification and diagnosis facilitate effective and prompt treatment. EEG biomarkers are a potential research hotspot for neuropsychiatric diseases. The purpose of this study was to investigate the differences in EEG power spectrum at theta oscillations between patients with MDD and patients with AN.

METHODS

Spectral analysis was used to study 66 patients with MDD and 43 patients with AN. Participants wore 16-lead EEG caps to measure resting EEG signals. The EEG power spectrum was measured using the fast Fourier transform. Independent samples t-test was used to analyze the EEG power values of the two groups, and p < 0.05 was statistically significant.

RESULTS

EEG power spectrum of the MDD group significantly differed from the AN group in the theta oscillation on 4-7 Hz at eight electrode points at F3, O2, T3, P3, P4, FP1, FP2, and F8.

CONCLUSION

Participants with anxiety demonstrated reduced power in the prefrontal cortex, left temporal lobe, and right occipital regions. Confirmed by further studies, theta oscillations could be another biomarker that distinguishes MDD from AN.

Semantic parafoveal processing in natural reading: Insight from fixation-related potentials & eye movements

Prior research suggests that we may access the meaning of parafoveal words during reading. We explored how semantic-plausibility parafoveal processing takes place in natural reading through the co-registration of eye movements (EM) and fixation-related potentials (FRPs), using the boundary paradigm. We replicated previous evidence of semantic parafoveal processing from highly controlled reading situations, extending their findings to more ecologically valid reading scenarios. Additionally, and exploring the time-course of plausibility preview effects, we found distinct but complementary evidence from EM and FRPs measures. FRPs measures, showing a different trend than EM evidence, revealed that plausibility preview effects may be long-lasting. We highlight the importance of a co-registration set-up in ecologically valid scenarios to disentangle the mechanisms related to semantic-plausibility parafoveal processing.

Mental State Detection Using Riemannian Geometry on Electroencephalogram Brain Signals

The goal of this study was to implement a Riemannian geometry (RG)-based algorithm to detect high mental workload (MWL) and mental fatigue (MF) using task-induced electroencephalogram (EEG) signals. In order to elicit high MWL and MF, the participants performed a cognitively demanding task in the form of the letter n-back task. We analyzed the time-varying characteristics of the EEG band power (BP) features in the theta and alpha frequency band at different task conditions and cortical areas by employing a RG-based framework. MWL and MF were considered as too high, when the Riemannian distances of the task-run EEG reached or surpassed the threshold of the baseline EEG. The results of this study showed a BP increase in the theta and alpha frequency bands with increasing experiment duration, indicating elevated MWL and MF that impedes/hinders the task performance of the participants. High MWL and MF was detected in 8 out of 20 participants. The Riemannian distances also showed a steady increase toward the threshold with increasing experiment duration, with the most detections occurring toward the end of the experiment. To support our findings, subjective ratings (questionnaires concerning fatigue and workload levels) and behavioral measures (performance accuracies and response times) were also considered.

Exposure to money modulates the processing of charitable donation: An event-related potentials study

Exposure to monetary cues might affect charitable donations, but little is known about how monetary cues affect charitable donations from the neural perspective. The present study examined the effect of monetary cues on charitable donations by means of event-related potentials (ERPs). Participants primed with monetary or neutral images decided whether to accept donation offers with the high, moderate, and low costs. The behavioural results showed that in the money-primed condition, participants took more time to decide for the high-cost than for the moderate and low-cost donation offers. The ERP results showed that the P2 and P3 were larger in the money-primed condition relative to the neutral images condition. Notably, participants primed with money demonstrated larger P3 for the high-cost donation offers than for the moderate and low-cost offers, but this difference was not observed in the control condition. These findings indicate that people primed with money may pay more attention to the cost-relevant information related to their self-interests when conducting prosocial behaviours.

Detecting lucid dreams only by submentalis electromyography

Lucid dreams (LDs) occur when people become aware that they are dreaming. This phenomenon has a wide range of possible applications from the perspectives of psychology, training physical movements, and controlling computers while asleep, among others. However, research on LDs might lack efficiency because the standard LD verification protocol uses polysomnography (PSG), which requires an expensive apparatus and skilled staff. The standard protocol also may reduce LD-induction efficiency. The current study examines whether humans can send phasic signals through submentalis electromyography (EMG) during muscle atonia via pre-agreed chin movements (PACM). This ability would manifest both REM sleep and consciousness, which are the main features of LDs. In laboratory conditions volunteers were instructed to open their jaws three times while in an LD right after the standard verification protocol to achieve the research goal. Results: 4 of 5 volunteers proved to be in an LD using the standard protocol, and then all of them made PACM. The outcomes show that dream signals cannot be blocked in the submentalis area during muscle atonia. Also, this finding can be considered to develop a simplified, reliable LD protocol that needs only one EMG sensor. The cost of this protocol could be only a small percentage of the current protocol, making it more convenient for researchers and volunteers. It can also be used remotely by inbuilt in wearable gadgets. Considering PACM could speed up LD research and provide many discoveries and new opportunities. Also, it can be used in sleep paralysis studies.

Automatic classification of ICA components from infant EEG using MARA

Automated systems for identifying and removing non-neural ICA components are growing in popularity among EEG researchers of adult populations. Infant EEG data differs in many ways from adult EEG data, but there exists almost no specific system for automated classification of source components from paediatric populations. Here, we adapt one of the most popular systems for adult ICA component classification for use with infant EEG data. Our adapted classifier significantly outperformed the original adult classifier on samples of naturalistic free play EEG data recorded from 10 to 12-month-old infants, achieving agreement rates with the manual classification of over 75% across two validation studies (n = 44, n = 25). Additionally, we examined both classifiers’ ability to remove stereotyped ocular artifact from a basic visual processing ERP dataset compared to manual ICA data cleaning. Here, the new classifier performed on level with expert manual cleaning and was again significantly better than the adult classifier at removing artifact whilst retaining a greater amount of genuine neural signal operationalised through comparing ERP activations in time and space. Our new system (iMARA) offers developmental EEG researchers a flexible tool for automatic identification and removal of artifactual ICA components.

•

Currently, few tools are available for ICA based data correction that are designed around infant EEG.

•

ICA correction is necessary for naturalistic paradigms when movement artifacts covary with cognitive processes.

•

We present a system for automated ICA classification designed around infant EEG data.

•

This system offers increased performance compared to systems of automatic ICA classification designed around adult EEG.

Strategic Distractor Suppression Improves Selective Control in Human Vision

Our visual environment is complicated, and our cognitive capacity is limited. As a result, we must strategically ignore some stimuli to prioritize others. Common sense suggests that foreknowledge of distractor characteristics, like location or color, might help us ignore these objects. But empirical studies have provided mixed evidence, often showing that knowing about a distractor before it appears counterintuitively leads to its attentional selection. What has looked like strategic distractor suppression in the past is now commonly explained as a product of prior experience and implicit statistical learning, and the long-standing notion the distractor suppression is reflected in α band oscillatory brain activity has been challenged by results appearing to link α to target resolution. Can we strategically, proactively suppress distractors? And, if so, does this involve α? Here, we use the concurrent recording of human EEG and eye movements in optimized experimental designs to identify behavior and brain activity associated with proactive distractor suppression. Results from three experiments show that knowing about distractors before they appear causes a reduction in electrophysiological indices of covert attentional selection of these objects and a reduction in the overt deployment of the eyes to the location of the objects. This control is established before the distractor appears and is predicted by the power of cue-elicited α activity over the visual cortex. Foreknowledge of distractor characteristics therefore leads to improved selective control, and α oscillations in visual cortex reflect the implementation of this strategic, proactive mechanism.SIGNIFICANCE STATEMENT To behave adaptively and achieve goals we often need to ignore visual distraction. Is it easier to ignore distracting objects when we know more about them? We recorded eye movements and electrical brain activity to determine whether foreknowledge of distractor characteristics can be used to limit processing of these objects. Results show that knowing the location or color of a distractor stops us from attentionally selecting it. A neural signature of this inhibition emerges in oscillatory alpha band brain activity, and when this signal is strong, selective processing of the distractor decreases. Knowing about the characteristics of task-irrelevant distractors therefore increases our ability to focus on task-relevant information, in this way gating information processing in the brain.

Yet another artefact rejection study: an exploration of cleaning methods for biological and neuromodulatory noise

Objective. Electroencephalography (EEG) cleaning has been a longstanding issue in the research community. In recent times, huge leaps have been made in the field, resulting in very promising techniques to address the issue. The most widespread ones rely on a family of mathematical methods known as blind source separation (BSS), ideally capable of separating artefactual signals from the brain originated ones. However, corruption of EEG data still remains a problem, especially in real life scenario where a mixture of artefact components affects the signal and thus correctly choosing the correct cleaning procedure can be non trivial. Our aim is here to evaluate and score the plethora of available BSS-based cleaning methods, providing an overview of their advantages and downsides and of their best field of application.Approach. To address this, we here first characterized and modeled different types of artefact, i.e. arising from muscular or blinking activity as well as from transcranial alternate current stimulation. We then tested and scored several BSS-based cleaning procedures on semi-synthetic datasets corrupted by the previously modeled noise sources. Finally, we built a lifelike dataset affected by many artefactual components. We tested an iterative multistep approach combining different BSS steps, aimed at sequentially removing each specific artefactual component.Main results. We did not find an overall best method, as different scenarios require different approaches. We therefore provided an overview of the performance in terms of both reconstruction accuracy and computational burden of each method in different use cases.Significance. Our work provides insightful guidelines for signal cleaning procedures in the EEG related field.

Eye tracking in human interaction: Possibilities and limitations

There is a long history of interest in looking behavior during human interaction. With the advance of (wearable) video-based eye trackers, it has become possible to measure gaze during many different interactions. We outline the different types of eye-tracking setups that currently exist to investigate gaze during interaction. The setups differ mainly with regard to the nature of the eye-tracking signal (head- or world-centered) and the freedom of movement allowed for the participants. These features place constraints on the research questions that can be answered about human interaction. We end with a decision tree to help researchers judge the appropriateness of specific setups.

Exposure to money modulates neural responses to outcome evaluations involving social reward

Recent research suggests that exposure to monetary cues strengthens an individual's motivation to pursue monetary rewards by inducing the 'market mode' (i.e. thinking and behaving in accordance with market principles). Here, we examined the effect of market mode on social reward processes by means of event-related potentials (ERPs). Participants primed with monetary images or neutral images acted as advisors who selected one of two options for a putative advisee. Subsequently, all participants passively observed the advisee accepting or rejecting their advice and receiving a gain or loss outcome. After money priming, the feedback-related negativity (FRN) to the advisee's gain/loss outcome was larger following incorrect as compared to correct advice irrespective of whether the advice had been accepted or rejected. A smaller P3 following incorrect advice showed only when the advice was rejected. After neutral priming, the FRN was larger for incorrect relative to correct advice only when the advice had been rejected. However, the P3 was larger for correct relative to incorrect advice irrespective of the advisee's final choice. These findings suggest that the market mode facilitates early and automatic feedback processing but reduces later and controlled responding to outcomes that had been accepted.

A Simple and Effective Way to Study Executive Functions by Using 360° Videos

Executive dysfunctions constitute a significant public health problem due to their high impact on everyday life and personal independence. Therefore, the identification of early strategies to assess and rehabilitate these impairments appears to be a priority. The ecological limitations of traditional neuropsychological tests and the numerous difficulties in administering tests in real-life scenarios have led to the increasing use of virtual reality (VR) and 360° environment-based tools for assessing executive functions (EFs) in real life. This perspective aims at proposing the development and implementation of Executive-functions Innovative Tool 360° (EXIT 360°), an innovative, enjoyable, and ecologically valid tool for a multidimensional and multicomponent evaluation of executive dysfunctions. EXIT 360° allows a complete and integrated executive functioning assessment through an original task for EFs delivered via a mobile-powered VR headset combined with eye tracker (ET) and electroencephalograms (EEG). Our tool is born as a 360°-based instrument, easily accessible and clinically usable, that will radically transform clinicians’ and patient’s assessment experience. In EXIT 360°, patients are engaged in a “game for health,” where they must perform everyday subtasks in 360° daily life environments. In this way, the clinicians can obtain quickly more ecologically valid information about several aspects of EFs (e.g., planning, problem-solving). Moreover, the multimodal approach allows completing the assessment of EFs by integrating verbal responses, reaction times, and physiological data (eye movements and brain activation). Overall, EXIT 360° will allow obtaining simultaneously and in real time more information about executive dysfunction and its impact in real life, allowing clinicians to tailor the rehabilitation to the subject’s needs.

A neurophysiological approach to spatial filter selection for adaptive brain–computer interfaces

Objective. The common spatial patterns (CSP) algorithm is an effective method to extract discriminatory features from electroencephalography (EEG) to be used by a brain–computer interface (BCI). However, informed selection of CSP filters typically requires oversight from a BCI expert to accept or reject filters based on the neurophysiological plausibility of their activation patterns. Our goal was to identify, analyze and automatically classify prototypical CSP patterns to enhance the prediction of motor imagery states in a BCI. Approach. A data-driven approach that used four publicly available EEG datasets was adopted. Cluster analysis revealed recurring, visually similar CSP patterns and a convolutional neural network was developed to distinguish between established CSP pattern classes. Furthermore, adaptive spatial filtering schemes that utilize the categorization of CSP patterns were proposed and evaluated. Main results. Classes of common neurophysiologically probable and improbable CSP patterns were established. Analysis of the relationship between these categories of CSP patterns and classification performance revealed discarding neurophysiologically improbable filters can decrease decoder performance. Further analysis revealed that the spatial orientation of EEG modulations can evolve over time, and that the features extracted from the original CSP filters can become inseparable. Importantly, it was shown through a novel adaptive CSP technique that adaptation in response to these emerging patterns can restore feature separability. Significance. These findings highlight the importance of considering and reporting on spatial filter activation patterns in both online and offline studies. They also emphasize to researchers in the field the importance of spatial filter adaptation in BCI decoder design, particularly for online studies with a focus on training users to develop stable and suitable brain patterns.

Cortical Activity Underlying Gait Improvements Achieved With Dopaminergic Medication During Usual Walking and Obstacle Avoidance in Parkinson Disease

BACKGROUND

Dopaminergic medication improves gait in people with Parkinson disease (PD). However, it remains unclear if dopaminergic medication modulates cortical activity while walking.

OBJECTIVE

We investigated the effects of dopaminergic medication on cortical activity during unobstructed walking and obstacle avoidance in people with PD.

METHODS

A total of 23 individuals with PD, in both off (PD_OFF) and on (PD_ON) medication states, and 30 healthy older adults (control group [CG]) performed unobstructed walking and obstacle avoidance conditions. Cortical activity was acquired through a combined functional near-infrared spectroscopy electroencephalography (EEG) system, along with gait parameters, through an electronic carpet. Prefrontal cortex (PFC) oxygenated hemoglobin (HbO₂) and EEG absolute power from FCz, Cz, and CPz channels were calculated.

RESULTS

HbO₂ concentration reduced for people with PD_OFF during obstacle avoidance compared with unobstructed walking. In contrast, both people with PD_ON and the CG had increased HbO₂ concentration when avoiding obstacles compared with unobstructed walking. Dopaminergic medication increased step length, step velocity, and β and γ power in the CPz channel, regardless of walking condition. Moreover, dopaminergic-related changes (ie, on-off) in FCz/CPz γ power were associated with dopaminergic-related changes in step length for both walking conditions.

CONCLUSIONS

PD compromises the activation of the PFC during obstacle avoidance, and dopaminergic medication facilitates its recruitment. In addition, PD medication increases sensorimotor integration during walking by increasing posterior parietal cortex (CPz) activity. Increased γ power in the CPz and FCz channels is correlated with step length improvements achieved with dopaminergic medication during unobstructed walking and obstacle avoidance in PD.

Effects of rTMS on the brain: is there value in variability?

The ability of repetitive transcranial magnetic stimulation (rTMS) to non-invasively induce neuroplasticity in the human cortex has opened exciting possibilities for its application in both basic and clinical research. Changes in the amplitude of motor evoked potentials (MEPs) elicited by single-pulse transcranial magnetic stimulation has so far provided a convenient model for exploring the neurophysiology of rTMS effects on the brain, influencing the ways in which these stimulation protocols have been applied therapeutically. However, a growing number of studies have reported large inter-individual variability in the mean MEP response to rTMS, raising legitimate questions about the usefulness of this model for guiding therapy. Although the increasing application of different neuroimaging approaches has made it possible to probe rTMS-induced neuroplasticity outside the motor cortex to measure changes in neural activity that impact other aspects of human behaviour, the high variability of rTMS effects on these measurements remains an important issue for the field to address. In this review, we seek to move away from the conventional facilitation/inhibition dichotomy that permeates much of the rTMS literature, presenting a non-standard approach for measuring rTMS-induced neuroplasticity. We consider the evidence that rTMS is able to modulate an individual's moment-to-moment variability of neural activity, and whether this could have implications for guiding the therapeutic application of rTMS.

Psychophysical dual-task setups do not measure pre-saccadic attention but saccade-related strengthening of sensory representations

Visual attention and saccadic eye movements are linked in a tight, yet flexible fashion. In humans, this link is typically studied with dual‐task setups. Participants are instructed to execute a saccade to some target location, while a discrimination target is flashed on a screen before the saccade can be made. Participants are also instructed to report a specific feature of this discrimination target at the trial end. Discrimination performance is usually better if the discrimination target occurred at the same location as the saccade target compared to when it occurred at a different location, which is explained by the mandatory shift of attention to the saccade target location before saccade onset. This pre‐saccadic shift of attention presumably enhances the perception of the discrimination target if it occurred at the same, but not if it occurred at a different location. It is, however, known that a dual‐task setup can alter the primary process under investigation. Here, we directly compared pre‐saccadic attention in single‐task versus dual‐task setups using concurrent electroencephalography (EEG) and eye‐tracking. Our results corroborate the idea of a pre‐saccadic shift of attention. They, however, question that this shift leads to the same‐position discrimination advantage. The relation of saccade and discrimination target position affected the EEG signal only after saccade onset. Our results, thus, favor an alternative explanation based on the role of saccades for the consolidation of sensory and short‐term memory. We conclude that studies with dual‐task setups arrived at a valid conclusion despite not measuring exactly what they intended to measure.

In humans, the relation between visual attention and saccadic eye movements is usually studied with psychophysical dual‐task setups. Here, we employ concurrent EEG and eye‐tracking to directly compare dual‐task to single‐task setups and conclude in line with previous research that attention precedes saccades. However, our results suggest that dual‐task setups do not measure what they are supposed to measure, that is, the pre‐saccadic shift of attention.

Real-Time Artifacts Reduction during TMS-EEG Co-Registration: A Comprehensive Review on Technologies and Procedures

Transcranial magnetic stimulation (TMS) excites neurons in the cortex, and neural activity can be simultaneously recorded using electroencephalography (EEG). However, TMS-evoked EEG potentials (TEPs) do not only reflect transcranial neural stimulation as they can be contaminated by artifacts. Over the last two decades, significant developments in EEG amplifiers, TMS-compatible technology, customized hardware and open source software have enabled researchers to develop approaches which can substantially reduce TMS-induced artifacts. In TMS-EEG experiments, various physiological and external occurrences have been identified and attempts have been made to minimize or remove them using online techniques. Despite these advances, technological issues and methodological constraints prevent straightforward recordings of early TEPs components. To the best of our knowledge, there is no review on both TMS-EEG artifacts and EEG technologies in the literature to-date. Our survey aims to provide an overview of research studies in this field over the last 40 years. We review TMS-EEG artifacts, their sources and their waveforms and present the state-of-the-art in EEG technologies and front-end characteristics. We also propose a synchronization toolbox for TMS-EEG laboratories. We then review subject preparation frameworks and online artifacts reduction maneuvers for improving data acquisition and conclude by outlining open challenges and future research directions in the field.

Regression-based analysis of combined EEG and eye-tracking data: Theory and applications

Fixation-related potentials (FRPs), neural responses aligned to the end of saccades, are a promising tool for studying the dynamics of attention and cognition under natural viewing conditions. In the past, four methodological problems have complicated the analysis of such combined eye-tracking/electroencephalogram experiments: (1) the synchronization of data streams, (2) the removal of ocular artifacts, (3) the condition-specific temporal overlap between the brain responses evoked by consecutive fixations, and (4) the fact that numerous low-level stimulus and saccade properties also influence the postsaccadic neural responses. Although effective solutions exist for the first two problems, the latter two are only beginning to be addressed. In the current paper, we present and review a unified regression-based framework for FRP analysis that allows us to deconvolve overlapping potentials while also controlling for both linear and nonlinear confounds on the FRP waveform. An open software implementation is provided for all procedures. We then demonstrate the advantages of this proposed (non)linear deconvolution modeling approach for data from three commonly studied paradigms: face perception, scene viewing, and reading. First, for a traditional event-related potential (ERP) face recognition experiment, we show how this technique can separate stimulus ERPs from overlapping muscle and brain potentials produced by small (micro)saccades on the face. Second, in natural scene viewing, we model and isolate multiple nonlinear effects of saccade parameters on the FRP. Finally, for a natural sentence reading experiment using the boundary paradigm, we show how it is possible to study the neural correlates of parafoveal preview after removing spurious overlap effects caused by the associated difference in average fixation time. Our results suggest a principal way of measuring reliable eye movement-related brain activity during natural vision.

Validation of SOBI-DANS method for automatic identification of horizontal and vertical eye movement components from EEG

Neurophysiological investigation of neural processes are hindered by the presence of large artifacts associated with eye movement. Although blind source separation (BSS)-based hybrid algorithms are useful for separating, identifying, and removing these artifacts from EEG, it remains unexamined to what extent neural signals can remain mixed with these artifact components, potentially resulting in unintended removal of critical neural signals. Here, we present a novel validation approach to quantitatively evaluate to what extent horizontal and vertical saccadic eye movement-related artifact components (H and V Comps) are indeed ocular in origin. To automate the identification of the H and V Comps recovered by the second-order blind identification (SOBI), we introduced a novel Discriminant ANd Similarity (DANS)-based method. Through source localization, we showed that over 95% of variance in the SOBI-DANS identified H and V Comps' scalp projections were ocular in origin. Through the analysis of saccade-related potentials (SRPs), we found that the H and V Comps' SRP amplitudes were finely modulated by eye movement direction and distance jointly. SOBI-DANS' component selection was in 100% agreement with human experts' selection and was 100% successful in component identification across all participants indicating a high cross-individual consistency or robustness. These results set the stage for future work to transform the to-be-thrown-away artifacts into signals indicative of gaze position, thereby providing readily co-registered eye movement and neural signal without using a separate eye tracker.

Toward Measuring Target Perception: First-Order and Second-Order Deep Network Pipeline for Classification of Fixation-Related Potentials

The topdown determined visual object perception refers to the ability of a person to identify a prespecified visual target. This paper studies the technical foundation for measuring the target-perceptual ability in a guided visual search task, using the EEG-based brain imaging technique. Specifically, it focuses on the feature representation learning problem for single-trial classification of fixation-related potentials (FRPs). The existing methods either capture only first-order statistics while ignoring second-order statistics in data, or directly extract second-order statistics with covariance matrices estimated with raw FRPs that suffer from low signal-to-noise ratio. In this paper, we propose a new representation learning pipeline involving a low-level convolution subnetwork followed by a high-level Riemannian manifold subnetwork, with a novel midlevel pooling layer bridging them. In this way, the discriminative power of the first-order features can be increased by the convolution subnetwork, while the second-order information in the convolutional features could further be deeply learned with the subsequent Riemannian subnetwork. In particular, the temporal ordering of FRPs is well preserved for the components in our pipeline, which is considered to be a valuable source of discriminant information. The experimental results show that proposed approach leads to improved classification performance and robustness to lack of data over the state-of-the-art ones, thus making it appealing for practical applications in measuring the target-perceptual ability of cognitively impaired patients with the FRP technique.

No Evidence for a Role of Spatially Modulated α-Band Activity in Tactile Remapping and Short-Latency, Overt Orienting Behavior

Oscillatory α-band activity is commonly associated with spatial attention and multisensory prioritization. It has also been suggested to reflect the automatic transformation of tactile stimuli from a skin-based, somatotopic reference frame into an external one. Previous research has not convincingly separated these two possible roles of α-band activity. Previous experimental paradigms have used artificially long delays between tactile stimuli and behavioral responses to aid relating oscillatory activity to these different events. However, this strategy potentially blurs the temporal relationship of α-band activity relative to behavioral indicators of tactile-spatial transformations. Here, we assessed α-band modulation with massive univariate deconvolution, an analysis approach that disentangles brain signals overlapping in time and space. Thirty-one male and female human participants performed a delay-free, visual search task in which saccade behavior was unrestricted. A tactile cue to uncrossed or crossed hands was either informative or uninformative about visual target location. α-Band suppression following tactile stimulation was lateralized relative to the stimulated hand over central-parietal electrodes but relative to its external location over parieto-occipital electrodes. α-Band suppression reflected external touch location only after informative cues, suggesting that posterior α-band lateralization does not index automatic tactile transformation. Moreover, α-band suppression occurred at the time of, or after, the production of the saccades guided by tactile stimulation. These findings challenge the idea that α-band activity is directly involved in tactile-spatial transformation and suggest instead that it reflects delayed, supramodal processes related to attentional reorienting.SIGNIFICANCE STATEMENT Localizing a touch in space requires integrating somatosensory information about skin location and proprioceptive or visual information about posture. The automatic remapping between skin-based tactile information to a location in external space has been proposed to rely on the modulation of oscillatory brain activity in the α-band range, across the multiple cortical areas that are involved in tactile, multisensory, and spatial processing. We report two findings that are inconsistent with this view. First, α-band activity reflected the remapped stimulus location only when touch was task relevant. Second, α-band modulation occurred too late to account for spatially directed behavioral responses and, thus, only after remapping must have taken place. These characteristics contradict the idea that α-band directly reflects automatic tactile remapping processes.

Parameter clustering in Bayesian functional principal component analysis of neuroscientific data

The extraordinary advancements in neuroscientific technology for brain recordings over the last decades have led to increasingly complex spatiotemporal data sets. To reduce oversimplifications, new models have been developed to be able to identify meaningful patterns and new insights within a highly demanding data environment. To this extent, we propose a new model called parameter clustering functional principal component analysis (PCl-fPCA) that merges ideas from functional data analysis and Bayesian nonparametrics to obtain a flexible and computationally feasible signal reconstruction and exploration of spatiotemporal neuroscientific data. In particular, we use a Dirichlet process Gaussian mixture model to cluster functional principal component scores within the standard Bayesian functional PCA framework. This approach captures the spatial dependence structure among smoothed time series (curves) and its interaction with the time domain without imposing a prior spatial structure on the data. Moreover, by moving the mixture from data to functional principal component scores, we obtain a more general clustering procedure, thus allowing a higher level of intricate insight and understanding of the data. We present results from a simulation study showing improvements in curve and correlation reconstruction compared with different Bayesian and frequentist fPCA models and we apply our method to functional magnetic resonance imaging and electroencephalogram data analyses providing a rich exploration of the spatiotemporal dependence in brain time series.

Neurophysiological indicators of internal attention: An electroencephalography-eye-tracking coregistration study

INTRODUCTION

Many goal-directed and spontaneous everyday activities (e.g., planning, mind wandering) rely on an internal focus of attention. Internally directed cognition (IDC) was shown to differ from externally directed cognition in a range of neurophysiological indicators such as electroencephalogram (EEG) alpha activity and eye behavior.

METHODS

In this EEG-eye-tracking coregistration study, we investigated effects of attention direction on EEG alpha activity and various relevant eye parameters. We used an established paradigm to manipulate internal attention demands in the visual domain within tasks by means of conditional stimulus masking.

RESULTS

Consistent with previous research, IDC involved relatively higher EEG alpha activity (lower alpha desynchronization) at posterior cortical sites. Moreover, IDC was characterized by greater pupil diameter (PD), fewer microsaccades, fixations, and saccades. These findings show that internal versus external cognition is associated with robust differences in several indicators at the neural and perceptual level. In a second line of analysis, we explored the intrinsic temporal covariation between EEG alpha activity and eye parameters during rest. This analysis revealed a positive correlation of EEG alpha power with PD especially in bilateral parieto-occipital regions.

CONCLUSION

Together, these findings suggest that EEG alpha activity and PD represent time-sensitive indicators of internal attention demands, which may be involved in a neurophysiological gating mechanism serving to shield internal cognition from irrelevant sensory information.

Is Cortical Activation During Walking Different Between Parkinson’s Disease Motor Subtypes?

Parkinson’s disease (PD) is often classified into tremor dominant (TD) and postural instability gait disorder (PIGD) subtypes. Degeneration of subcortical/cortical pathways is different between PD subtypes, which leads to differences in motor behavior. However, the influence of PD subtype on cortical activity during walking remains poorly understood. Therefore, we aimed to investigate the influence of PD motor subtypes on cortical activity during unobstructed walking and obstacle avoidance. Seventeen PIGD and 19 TD patients performed unobstructed walking and obstacle avoidance conditions. Brain activity was measured using a mobile functional near-infrared spectroscopy–electroencephalography (EEG) systems, and gait parameters were analyzed using an electronic carpet. Concentrations of oxygenated hemoglobin (HbO2) of the prefrontal cortex (PFC) and EEG absolute power from alpha, beta, and gamma bands in FCz, Cz, CPz, and Oz channels were calculated. These EEG channels correspond to supplementary motor area, primary motor cortex, posterior parietal cortex, and visual cortex, respectively. Postural instability gait disorder patients presented higher PFC activity than TD patients, regardless of the walking condition. Tremor dominant patients presented reduced beta power in the Cz channel during obstacle avoidance compared to unobstructed walking. Both TD and PIGD patients decreased alpha and beta power in the FCz and CPz channels. In conclusion, PIGD patients need to recruit additional cognitive resources from the PFC for walking. Both TD and PIGD patients presented changes in the activation of brain areas related to motor/sensorimotor areas in order to maintain balance control during obstacle avoidance, being that TD patients presented further changes in the motor area (Cz channel) to avoid obstacles.

Best practices in eye tracking research

This guide describes best practices in using eye tracking technology for research in a variety of disciplines. A basic outline of the anatomy and physiology of the eyes and of eye movements is provided, along with a description of the sorts of research questions eye tracking can address. We then explain how eye tracking technology works and what sorts of data it generates, and provide guidance on how to select and use an eye tracker as well as selecting appropriate eye tracking measures. Challenges to the validity of eye tracking studies are described, along with recommendations for overcoming these challenges. We then outline correct reporting standards for eye tracking studies.

Microsaccades in Applied Environments: Real-World Applications of Fixational Eye Movement Measurements

Across a wide variety of research environments, the recording of microsaccades and other fixational eye movements has provided insight and solutions into practical problems. Here we review the literature on fixational eye movements-especially microsaccades-in applied and ecologically-valid scenarios. Recent technical advances allow noninvasive fixational eye movement recordings in real-world contexts, while observers perform a variety of tasks. Thus, fixational eye movement measures have been obtained in a host of real-world scenarios, such as in connection with driver fatigue, vestibular sensory deprivation in astronauts, and elite athletic training, among others. Here we present the state of the art in the practical applications of fixational eye movement research, examine its potential future uses, and discuss the benefits of including microsaccade measures in existing eye movement detection technologies. Current evidence supports the inclusion of fixational eye movement measures in real-world contexts, as part of the development of new or improved oculomotor assessment tools. The real-world applications of fixational eye movement measurements will only grow larger and wider as affordable high-speed and high-spatial resolution eye trackers become increasingly prevalent.

Toward the Understanding of Topographical and Spectral Signatures of Infant Movement Artifacts in Naturalistic EEG

Electroencephalography (EEG) is perhaps the most widely used brain-imaging technique for pediatric populations. However, EEG signals are prone to distortion by motion. Compared to adults, infants' motion is both more frequent and less stereotypical yet motion effects on the infant EEG signal are largely undocumented. Here, we present a systematic assessment of naturalistic motion effects on the infant EEG signal. EEG recordings were performed with 14 infants (12 analyzed) who passively watched movies whilst spontaneously producing periods of bodily movement and rest. Each infant produced an average of 38.3 s (SD = 14.7 s) of rest and 18.8 s (SD = 17.9 s) of single motion segments for the final analysis. Five types of infant motions were analyzed: Jaw movements, and Limb movements of the Hand, Arm, Foot, and Leg. Significant movement-related distortions of the EEG signal were detected using cluster-based permutation analysis. This analysis revealed that, relative to resting state, infants' Jaw and Arm movements produced significant increases in beta (∼15 Hz) power, particularly over peripheral sites. Jaw movements produced more anteriorly located effects than Arm movements, which were most pronounced over posterior parietal and occipital sites. The cluster analysis also revealed trends toward decreased power in the theta and alpha bands observed over central topographies for all motion types. However, given the very limited quantity of infant data in this study, caution is recommended in interpreting these findings before subsequent replications are conducted. Nonetheless, this work is an important first step to inform future development of methods for addressing EEG motion-related artifacts. This work also supports wider use of naturalistic paradigms in social and developmental neuroscience.

Investigating the Use of Pretrained Convolutional Neural Network on Cross-Subject and Cross-Dataset EEG Emotion Recognition

The electroencephalogram (EEG) has great attraction in emotion recognition studies due to its resistance to deceptive actions of humans. This is one of the most significant advantages of brain signals in comparison to visual or speech signals in the emotion recognition context. A major challenge in EEG-based emotion recognition is that EEG recordings exhibit varying distributions for different people as well as for the same person at different time instances. This nonstationary nature of EEG limits the accuracy of it when subject independency is the priority. The aim of this study is to increase the subject-independent recognition accuracy by exploiting pretrained state-of-the-art Convolutional Neural Network (CNN) architectures. Unlike similar studies that extract spectral band power features from the EEG readings, raw EEG data is used in our study after applying windowing, pre-adjustments and normalization. Removing manual feature extraction from the training system overcomes the risk of eliminating hidden features in the raw data and helps leverage the deep neural network’s power in uncovering unknown features. To improve the classification accuracy further, a median filter is used to eliminate the false detections along a prediction interval of emotions. This method yields a mean cross-subject accuracy of 86.56% and 78.34% on the Shanghai Jiao Tong University Emotion EEG Dataset (SEED) for two and three emotion classes, respectively. It also yields a mean cross-subject accuracy of 72.81% on the Database for Emotion Analysis using Physiological Signals (DEAP) and 81.8% on the Loughborough University Multimodal Emotion Dataset (LUMED) for two emotion classes. Furthermore, the recognition model that has been trained using the SEED dataset was tested with the DEAP dataset, which yields a mean prediction accuracy of 58.1% across all subjects and emotion classes. Results show that in terms of classification accuracy, the proposed approach is superior to, or on par with, the reference subject-independent EEG emotion recognition studies identified in literature and has limited complexity due to the elimination of the need for feature extraction.

Fixation-related Brain Potentials during Semantic Integration of Object–Scene Information

In vision science, a particularly controversial topic is whether and how quickly the semantic information about objects is available outside foveal vision. Here, we aimed at contributing to this debate by coregistering eye movements and EEG while participants viewed photographs of indoor scenes that contained a semantically consistent or inconsistent target object. Linear deconvolution modeling was used to analyze the ERPs evoked by scene onset as well as the fixation-related potentials (FRPs) elicited by the fixation on the target object (t) and by the preceding fixation (t − 1). Object–scene consistency did not influence the probability of immediate target fixation or the ERP evoked by scene onset, which suggests that object–scene semantics was not accessed immediately. However, during the subsequent scene exploration, inconsistent objects were prioritized over consistent objects in extrafoveal vision (i.e., looked at earlier) and were more effortful to process in foveal vision (i.e., looked at longer). In FRPs, we demonstrate a fixation-related N300/N400 effect, whereby inconsistent objects elicit a larger frontocentral negativity than consistent objects. In line with the behavioral findings, this effect was already seen in FRPs aligned to the pretarget fixation t − 1 and persisted throughout fixation t, indicating that the extraction of object semantics can already begin in extrafoveal vision. Taken together, the results emphasize the usefulness of combined EEG/eye movement recordings for understanding the mechanisms of object–scene integration during natural viewing.

Parafoveal-on-foveal repetition effects in sentence reading: A co-registered eye-tracking and electroencephalogram study

When reading, can the next word in the sentence (word n + 1) influence how you read the word you are currently looking at (word n)? Serial models of sentence reading state that this generally should not be the case, whereas parallel models predict that this should be the case. Here we focus on perhaps the simplest and the strongest Parafoveal‐on‐Foveal (PoF) manipulation: word n + 1 is either the same as word n or a different word. Participants read sentences for comprehension and when their eyes left word n, the repeated or unrelated word at position n + 1 was swapped for a word that provided a syntactically correct continuation of the sentence. We recorded electroencephalogram and eye‐movements, and time‐locked the analysis of fixation‐related potentials (FRPs) to fixation of word n. We found robust PoF repetition effects on gaze durations on word n, and also on the initial landing position on word n. Most important is that we also observed significant effects in FRPs, reaching significance at 260 ms post‐fixation of word n. Repetition of the target word n at position n + 1 caused a widely distributed reduced negativity in the FRPs. Given the timing of this effect, we argue that it is driven by orthographic processing of word n + 1, while readers were still looking at word n, plus the spatial integration of orthographic information extracted from these two words in parallel.

A hotly debated issue in reading research concerns the serial versus parallel nature of word identification processes. Evidence in favor of parallel processing has been criticized because it has often been obtained with artificial reading paradigms. Here we show that fixation‐related potentials elicited by a target word embedded in a normal sentence are impacted by the nature of the word immediately to the right of the fixated target word (n), and that this impact became significant 260 ms after fixation of word n. This is in line with parallel orthographic processing across adjacent words during sentence reading.

Relative Average Look Duration and its Association with Neurophysiological Activity in Young Children with Autism Spectrum Disorder

Autism Spectrum Disorder (ASD) is characterized by early attentional differences that often precede the hallmark symptoms of social communication impairments. Development of novel measures of attentional behaviors may lead to earlier identification of children at risk for ASD. In this work, we first introduce a behavioral measure, Relative Average Look Duration (RALD), indicating attentional preference to different stimuli, such as social versus nonsocial stimuli; and then study its association with neurophysiological activity. We show that (1) ASD and typically developing (TD) children differ in both (absolute) Average Look Duration (ALD) and RALD to stimuli during an EEG experiment, with the most pronounced differences in looking at social stimuli; and (2) associations between looking behaviors and neurophysiological activity, as measured by EEG, are different for children with ASD versus TD. Even when ASD children show attentional engagement to social content, our results suggest that their underlying brain activity is different than TD children. This study therefore introduces a new measure of social/nonsocial attentional preference in ASD and demonstrates the value of incorporating attentional variables measured simultaneously with EEG into the analysis pipeline.

Eye Movements and Fixation-Related Potentials in Reading: A Review

The present review is addressed to researchers in the field of reading and psycholinguistics who are both familiar with and new to co-registration research of eye movements (EMs) and fixation related-potentials (FRPs) in reading. At the outset, we consider a conundrum relating to timing discrepancies between EM and event related potential (ERP) effects. We then consider the extent to which the co-registration approach might allow us to overcome this and thereby discriminate between formal theoretical and computational accounts of reading. We then describe three phases of co-registration research before evaluating the existing body of such research in reading. The current, ongoing phase of co-registration research is presented in comprehensive tables which provide a detailed summary of the existing findings. The thorough appraisal of the published studies allows us to engage with issues such as the reliability of FRP components as correlates of cognitive processing in reading and the advantages of analysing both data streams (i.e., EMs and FRPs) simultaneously relative to each alone, as well as the current, and limited, understanding of the relationship between EM and FRP measures. Finally, we consider future directions and in particular the potential of analytical methods involving deconvolution and the potential of measurement of brain oscillatory activity.

Post-Saccadic Face Processing Is Modulated by Pre-Saccadic Preview: Evidence from Fixation-Related Potentials

Humans actively sample their environment with saccadic eye movements to bring relevant information into high-acuity foveal vision. Despite being lower in resolution, peripheral information is also available before each saccade. How the pre-saccadic extrafoveal preview of a visual object influences its post-saccadic processing is still an unanswered question. The current study investigated this question by simultaneously recording behavior and fixation-related brain potentials while human subjects made saccades to face stimuli. We manipulated the relationship between pre-saccadic "previews" and post-saccadic images to explicitly isolate the influences of the former. Subjects performed a gender discrimination task on a newly foveated face under three preview conditions: scrambled face, incongruent face (different identity from the foveated face), and congruent face (same identity). As expected, reaction times were faster after a congruent-face preview compared with a scrambled-face preview. Importantly, intact face previews (either incongruent or congruent) resulted in a massive reduction of post-saccadic neural responses. Specifically, we analyzed the classic face-selective N170 component at occipitotemporal electroencephalogram electrodes, which was still present in our experiments with active looking. However, the post-saccadic N170 was strongly attenuated following intact-face previews compared with the scrambled condition. This large and long-lasting decrease in evoked activity is consistent with a trans-saccadic mechanism of prediction that influences category-specific neural processing at the start of a new fixation. These findings constrain theories of visual stability and show that the extrafoveal preview methodology can be a useful tool to investigate its underlying mechanisms.SIGNIFICANCE STATEMENT Neural correlates of object recognition have traditionally been studied by flashing stimuli to the central visual field. This procedure differs in fundamental ways from natural vision, where viewers actively sample the environment with eye movements and also obtain a low-resolution preview of soon-to-be-fixated objects. Here we show that the N170, a classic electrophysiological marker of the structural encoding of faces, also occurs during a more natural viewing condition but is strongly reduced due to extrafoveal preprocessing (preview benefit). Our results therefore highlight the importance of peripheral vision during trans-saccadic processing in building a coherent and stable representation of the world around us.

Learning to see the threat: temporal dynamics of ERPs of motivated attention in fear conditioning

Social threat detection is important in everyday life. Studies of cortical activity have shown that event-related potentials (ERPs) of motivated attention are modulated during fear conditioning. The time course of motivated attention in learning and extinction of fear is, however, still largely unknown. We aimed to study temporal dynamics of learning processes in classical fear conditioning to social cues (neutral faces) by selecting an experimental setup that produces large effects on well-studied ERP components (early posterior negativity, EPN; late positive potential, LPP; stimulus preceding negativity, SPN) and then exploring small consecutive groups of trials. EPN, LPP, and SPN markedly and quickly increased during the acquisition phase in response to the CS+ but not the CS-. These changes were visible even at high temporal resolution and vanished completely during extinction. Moreover, some evidence was found for component differences in extinction learning, with differences between CS+ and CS- extinguishing faster for late as compared to early ERP components. Results demonstrate that fear learning to social cues is a very fast and highly plastic process and conceptually different ERPs of motivated attention are sensitive to these changes at high temporal resolution, pointing to specific neurocognitive and affective processes of social fear learning.

(Early) context effects on event-related potentials over natural inputs

Language understanding requires the integration of the input with preceding context. Event-related potentials (ERPs) have contributed significantly to our understanding of what contextual information is accessed and when. Much of this research has, however, been limited to experimenter-designed stimuli with highly atypical lexical and context statistics. This raises questions about the extent to which previous findings generalize to everyday language processing of natural stimuli with typical linguistic statistics. We ask whether context can affect ERPs over natural stimuli early, before the N400 time window. We re-analyzed a data set of ERPs over ~700 visually presented content words in sentences from English novels. To increase power, we employed linear mixed effects regression simultaneously modeling random variance by subject and by item. To reduce concerns about Type I error inflation common to any type of time series analysis, we introduced a simple approach to model and discount auto-correlations at multiple, empirically determined, time lags. We compared this approach to Bonferroni correction. Planned follow-up analyses used Generalized Additive Mixture Models to assess the linearity of contextual effects, including lexical surprisal, found within the N400 time window. We found that contextual information affects ERPs in both early (~200ms after word onset) and late (N400) time windows, supporting a cascading, interactive account of lexical access.

A co-registration investigation of inter-word spacing and parafoveal preview: Eye movements and fixation-related potentials

Participants' eye movements (EMs) and EEG signal were simultaneously recorded to examine foveal and parafoveal processing during sentence reading. All the words in the sentence were manipulated for inter-word spacing (intact spaces vs. spaces replaced by a random letter) and parafoveal preview (identical preview vs. random letter string preview). We observed disruption for unspaced text and invalid preview conditions in both EMs and fixation-related potentials (FRPs). Unspaced and invalid preview conditions received longer reading times than spaced and valid preview conditions. In addition, the FRP data showed that unspaced previews disrupted reading in earlier time windows of analysis, compared to string preview conditions. Moreover, the effect of parafoveal preview was greater for spaced relative to unspaced conditions, in both EMs and FRPs. These findings replicate well-established preview effects, provide novel insight into the neural correlates of reading with and without inter-word spacing and suggest that spatial selection precedes lexical processing.

14 challenges and their solutions for conducting social neuroscience and longitudinal EEG research with infants

The use of electroencephalography (EEG) to study infant brain development is a growing trend. In addition to classical longitudinal designs that study the development of neural, cognitive and behavioural functions, new areas of EEG application are emerging, such as novel social neuroscience paradigms using dual infant-adult EEG recordings. However, most of the experimental designs, analysis methods, as well as EEG hardware were originally developed for single-person adult research. When applied to study infant development, adult-based solutions often pose unique problems that may go unrecognised. Here, we identify 14 challenges that infant EEG researchers may encounter when designing new experiments, collecting data, and conducting data analysis. Challenges related to the experimental design are: (1) small sample size and data attrition, and (2) varying arousal in younger infants. Challenges related to data acquisition are: (3) determining the optimal location for reference and ground electrodes, (4) control of impedance when testing with the high-density sponge electrode nets, (5) poor fit of standard EEG caps to the varying infant head shapes, and (6) ensuring a high degree of temporal synchronisation between amplifiers and recording devices during dual-EEG acquisition. Challenges related to the analysis of longitudinal and social neuroscience datasets are: (7) developmental changes in head anatomy, (8) prevalence and diversity of infant myogenic artefacts, (9) a lack of stereotypical topography of eye movements needed for the ICA-based data cleaning, (10) and relatively high inter-individual variability of EEG responses in younger cohorts. Additional challenges for the analysis of dual EEG data are: (11) developmental shifts in canonical EEG rhythms and difficulties in differentiating true inter-personal synchrony from spurious synchrony due to (12) common intrinsic properties of the signal and (13) shared external perturbation. Finally, (14) there is a lack of test-retest reliability studies of infant EEG. We describe each of these challenges and suggest possible solutions. While we focus specifically on the social neuroscience and longitudinal research, many of the issues we raise are relevant for all fields of infant EEG research.