Combining EEG and eye tracking: identification, characterization, and correction of eye movement artifacts in electroencephalographic data

Eye-movement artifact correction in infant EEG: A systematic comparison between ICA and Artifact Blocking

BACKGROUND

Independent Component Analysis (ICA) is a well-established approach to clean EEG and remove the impact of signals of non-neural origin, such as those from muscular activity and eye movements. However, evidence suggests that ICA removes artifacts less effectively in infants than in adults. This study systematically compares ICA and Artifact Blocking (AB), an alternative approach proposed to improve eye-movement artifact correction in infant EEG.

METHODS

We analyzed EEG collected from 50 infants between 6 and 18 months of age as part of the International Infant EEG Data Integration Platform (EEG-IP), a longitudinal multi-study dataset. EEG was recorded while infants sat on their caregivers' laps and watched videos. We used ICA and AB to correct for eye-movement artifacts in the EEG and calculated the proportion of effectively corrected segments, signal-to-noise ratio (SNR), power-spectral density (PSD), and multiscale entropy (MSE) in manually selected EEG segments with and without eye-movement artifacts.

RESULTS

On the one hand, the proportion of effectively corrected segments indicated that ICA corrected eye-movement artifacts (sensitivity) better than AB. SNR and PSD indicated that both AB and ICA correct eye-movement artifacts with equal sensitivity. MSE gave mixed results. On the other hand, AB caused less distortion to the clean segments (specificity) for SNR, PSD, and MSE.

CONCLUSION

Our results suggest that ICA is more sensitive (i.e., it better removes artifacts) but less specific (it distorts clean signals) than AB for correcting eye-movement artifacts in infant EEG.

The influence of empathic concern on evaluative processing in self and charity outcomes

Empathy plays a crucial role in determining how one understands others' emotional experiences and behavioral decisions. This study aimed to explore whether empathic concern affects the processing of self-related and charity-related outcome evaluations. In this study, participants performed gambling tasks for themselves and low- and high-empathy charities. The behavioral results showed that low-empathy charities had a significantly higher risk rate than the self, whereas there was no significant difference between low-and high-empathy charities. The event-related potential (ERP) results showed that the P300 valence difference (d-P300) of the self was significantly higher for high-empathy charitable activities than for low-empathy charitable activities, and the d-P300 of high-empathy charitable activities was significantly higher than that of low-empathy charitable activities. The P300 valence differences primarily originated from activation difference in the posterior mid-cingulate cortex (pMCC) and medial prefrontal cortex (mPFC). The time-frequency analysis showed that positive outcomes induced greater β2 event-related desynchronization (ERD) amplitudes for high-empathy charitable activities compared to negative outcomes. These findings suggest that empathic concern increased the distinction between good and bad outcomes for charities and promoted greater cognitive effort allocation toward prosocial rewards. The d-P300 is closely linked to activations in the pMCC and mPFC.

Neural correlates of how egoistic, altruistic, and mixed motivations modulate the influence of self-deception on deceptive behavior

Using event-related potentials, this study investigated the interplay between self-deception and varying motivations in influencing deceptive behavior. Participants in either the self-deception or nonself-deception condition were instructed to make deceptive decisions under different motivational contexts. Behavioral responses and neural activity were recorded throughout the decision-making process. Behaviorally, there were no significant differences in deception proportions between the self-deception and nonself-deception conditions under egoistic or mixed motivations. However, under altruistic motivation, participants in the self-deception condition engaged in more deceptive behavior compared to those in the nonself-deception condition. Event-related potential results revealed no significant differences in P2 and N2 amplitudes between the self-deception and nonself-deception conditions under egoistic or mixed motivations. However, under altruistic motivation, the self-deception condition was associated with reduced P2 amplitudes and more negative N2 amplitudes relative to the nonself-deception condition. Additionally, no significant interaction in P3 amplitudes was found between self-deception and different motivations. These findings suggest that the effect of self-deception on deceptive behavior is modulated by different motivations. Specifically, egoistic or mixed motivations might be associated with a self-serving tendency in deceptive behavior, while altruistic motivation appeared to enhance deceptive behavior in the self-deception condition, potentially to foster a prosocial image.

EEG time-frequency dynamics of early cognitive control development

Cognitive control is crucial for goal-directed behavior, and essential for other aspects of cognitive and socioemotional development. This review examines when and how the neural dynamics of cognitive control emerge and develop, focusing on electroencephalography measures used to study cognitive control in infants and children. We argue that time-frequency analyses are uniquely able to capture two distinct components of cognitive control: 1) the detection that control is needed, and 2) the instantiation of control. Starting in infancy and increasing across childhood and adolescence, studies suggest the signal strength and consistency of midfrontal theta and delta oscillations are involved in processes that detect the need for control. For control instantiation, there is evidence that theta band connectivity between midfrontal and lateral-frontal cortices is present from early childhood. There is also evidence for the involvement of midfrontal theta power in the instantiation of control in infancy. We further review emerging evidence that indicates individual differences in midfrontal theta are not only proximally related to behavior, but also sensitive to variations in early experience and risk for psychopathology, providing a neural mechanism linking early adversity to future psychopathology. We discuss needed future steps, including novel paradigms, computational models, and aperiodic/periodic modeling of EEG.

Protocol for semi-automatic EEG preprocessing incorporating independent component analysis and principal component analysis

Preprocessing is a critical yet challenging step in electroencephalography (EEG) research due to its significant potential impact on results. We present a protocol for semi-automatic EEG preprocessing incorporating independent component analysis (ICA) and principal component analysis (PCA) with step-by-step quality checking to ensure removal of large-amplitude artifacts. We describe steps for interpolating bad channels, removal of major artifacts by ICA and PCA correction, and exporting processed data. This protocol produced consistent results from users with a broad range of experience.

The Representation of Stimulus Features during Stable Fixation and Active Vision

Predictive updating of an object's spatial coordinates from presaccade to postsaccade contributes to stable visual perception. Whether object features are predictively remapped remains contested. We set out to characterize the spatiotemporal dynamics of feature processing during stable fixation and active vision. To do so, we applied multivariate decoding methods to EEG data collected while human participants (male and female) viewed brief visual stimuli. Stimuli appeared at different locations across the visual field at either high or low spatial frequency (SF). During fixation, classifiers were trained to decode SF presented at one parafoveal location and cross-tested on SF from either the same, adjacent, or more peripheral locations. When training and testing on the same location, SF was classified shortly after stimulus onset (∼79 ms). Decoding of SF at locations farther from the trained location emerged later (∼144-295 ms), with decoding latency modulated by eccentricity. This analysis provides a detailed time course for the spread of feature information across the visual field. Next, we investigated how active vision impacts the emergence of SF information. In the presence of a saccade, the decoding time of peripheral SF at parafoveal locations was earlier, indicating predictive anticipation of SF due to the saccade. Crucially, however, this predictive effect was not limited to the specific remapped location. Rather, peripheral SF was correctly classified, at an accelerated time course, at all parafoveal positions. This indicates spatially coarse, predictive anticipation of stimulus features during active vision, likely enabling a smooth transition on saccade landing.

Neural correlates of social observation and socioeconomic status in influencing environmental donations

With the increasing urgency of environmental degradation, it is crucial to investigate whether and why individuals from different socioeconomic statuses (SES) engage in environmental donations within social contexts to develop targeted strategies that promote environmental sustainability. However, the psychological mechanisms and neural activities underlying environmental donations across SES in social contexts remain unclear. The current study randomly assigned participants to high (low) SES groups and asked them to complete the environmental donation task in the (non) observable contexts while the electroencephalogram was recorded. Behaviorally, this study revealed that the presence of others promoted environmental donations in the low-cost condition and the average donations and acceptance rates were influenced by costliness. The ERP results indicated that high-SES individuals exhibited more negative N2 in the observable condition than in the non-observable condition. At the later stage, social observation modulated the neural activity evoked by donation decision making, with high-SES individuals showing larger P3 and late positive potentials (LPP) in the observable (vs. non-observable) condition. These findings suggest that public service institutions can implement targeted fundraising strategies that consider motivational differences among individuals with different SES, such as creating cost-effective environmental scenarios for low-SES individuals and designing reputation-enhancing situations for high-SES individuals.

The influence of social status and promise levels in trust games: An Event-Related Potential (ERP) study

Promises are widely used to increase trust in social status; yet how promise levels and social status influence trust behavior and its underlying neurophysiological mechanisms remain unclear. We used a modified trust game to investigate the effects of promise levels and social status on trust behavior. Participants, as investors paired with trustees of varying social status who were given the opportunity to promise to return different levels of money, were required to decide to whether trust the trustees. In Experiment 1, trustees promised to return high-, medium-, or low-level money to participants, and no return feedback was provided. In Experiment 2, trustees promised to return high- and low-level money to participants. Return feedback was provided and event-related potential (ERP) data were recorded. The behavioral results indicated that participants trusted high-status partners more than low-status partners, regardless of the promise level. The ERP results showed that with low-status partners, the N2 was more negative and the P3 was smaller under low-level promise conditions than under high-level promise conditions. However, with high-status partner, there were no differences in N2 and P3 between high- and low-level promise conditions. Our findings suggest that social status may affect the perception of potential risks across different promise levels in trust.

Alternate Electrode Placements to Facilitate Frontal Electroencephalography Monitoring in Anesthetized and Critically Ill Patients

BACKGROUND

Frontal electroencephalography (EEG) monitoring can be useful in guiding the titration of anesthetics, but it is not always feasible to place electrodes in the standard configuration in some circumstances, including during neurosurgery. This study compares 5 alternate configurations of the Masimo Sedline Sensor.

METHODS

Ten stably sedated patients in the intensive care unit were recruited. Frontal EEG was monitored in the standard configuration (bifrontal upright) and 5 alternate configurations: bifrontal inverse, infraorbital, lateral upright, lateral inverse, and semilateral. Average power spectral densities (PSDs) with 95% CIs in the alternate configurations were compared to PSDs in the standard configuration. Two-one-sided-testing with Wilcoxon signed-rank tests assessed equivalence in the spectral edge frequency (SEF-95), EEG power, and relative delta (0.5 to 3.5 Hz), alpha (8 to 12 Hz), and beta (20 to 30 Hz) power between each alternate and standard configurations.

RESULTS

After the removal of unanalyzable tracings, 7 patients were included for analysis in the infraorbital configuration and 9 in all other configurations. In the lateral upright and lateral inverse configurations, PSDs significantly differed from the standard configuration within the 15 to 20 Hz band. The greatest decrease in EEG power was in the lateral inverse configuration (median: -97 dB; IQR: -130, -62 dB). The largest change in frequency distribution of EEG power was in the infraorbital configuration; median SEF-95 change of -1.4 Hz (IQR: -2.8, 0.7 Hz), median relative delta power change of +7.3% (IQR: 1.4%, 7.9%), and median relative alpha power change of -0.6% (IQR: -5.7%, 0.0%).

CONCLUSIONS

These 5 alternate Sedline electrode configurations are suitable options for monitoring frontal EEG when the standard configuration is not possible.

Parafoveal N400 effects reveal that word skipping is associated with deeper lexical processing in the presence of context-driven expectations

Readers are able to begin processing upcoming words before directly fixating them, and in some cases skip words altogether (i.e., never fixated). However, the exact mechanisms and recognition thresholds underlying skipping decisions are not entirely clear. In the current study, we test whether skipping decisions reflect instances of more extensive lexical processing by recording neural language processing (via electroencephalography; EEG) and eye movements simultaneously, and we split trials based on target word-skipping behavior. To test lexical processing of the words, we manipulated the orthographic and phonological relationship between upcoming preview words and a semantically correct (and in some cases, expected) target word using the gaze-contingent display change paradigm. We also manipulated the constraint of the sentences to investigate the extent to which the identification of sublexical features of words depends on a reader's expectations. We extracted fixation-related brain potentials (FRPs) during the fixation on the preceding word (i.e., in response to parafoveal viewing of the manipulated previews). We found that word skipping is associated with larger neural responses (i.e., N400 amplitudes) to semantically incongruous words that did not share a phonological representation with the correct word, and this effect was only observed in high-constraint sentences. These findings suggest that word skipping can be reflective of more extensive linguistic processing, but in the absence of expectations, word skipping may occur based on less fine-grained linguistic processing and be more reflective of identification of plausible or expected sublexical features rather than higher-level lexical processing (e.g., semantic access).

Neural mechanism underlying preview effects and masked priming effects in visual word processing

Two classic experimental paradigms - masked repetition priming and the boundary paradigm - have played a pivotal role in understanding the process of visual word recognition. Traditionally, these paradigms have been employed by different communities of researchers, with their own long-standing research traditions. Nevertheless, a review of the literature suggests that the brain-electric correlates of word processing established with both paradigms may show interesting similarities, in particular with regard to the location, timing, and direction of N1 and N250 effects. However, as of yet, no direct comparison has been undertaken between the two paradigms. In the current study, we used combined eye-tracking/EEG to perform such a within-subject comparison using the same materials (single Chinese characters) as stimuli. To facilitate direct comparisons, we used a simplified version of the boundary paradigm - the single word boundary paradigm. Our results show the typical early repetition effects of N1 and N250 for both paradigms. However, repetition effects in N250 (i.e., a reduced negativity following identical-word primes/previews as compared to different-word primes/previews) were larger with the single word boundary paradigm than with masked priming. For N1 effects, repetition effects were similar across the two paradigms, showing a larger N1 after repetitions as compared to alternations. Therefore, the results indicate that at the neural level, a briefly presented and masked foveal prime produces qualitatively similar facilitatory effects on visual word recognition as a parafoveal preview before a single saccade, although such effects appear to be stronger in the latter case.

Fixation-related potentials during a virtual navigation task: The influence of image statistics on early cortical processing

Historically, electrophysiological correlates of scene processing have been studied with experiments using static stimuli presented for discrete timescales where participants maintain a fixed eye position. Gaps remain in generalizing these findings to real-world conditions where eye movements are made to select new visual information and where the environment remains stable but changes with our position and orientation in space, driving dynamic visual stimulation. Co-recording of eye movements and electroencephalography (EEG) is an approach to leverage fixations as time-locking events in the EEG recording under free-viewing conditions to create fixation-related potentials (FRPs), providing a neural snapshot in which to study visual processing under naturalistic conditions. The current experiment aimed to explore the influence of low-level image statistics-specifically, luminance and a metric of spatial frequency (slope of the amplitude spectrum)-on the early visual components evoked from fixation onsets in a free-viewing visual search and navigation task using a virtual environment. This research combines FRPs with an optimized approach to remove ocular artifacts and deconvolution modeling to correct for overlapping neural activity inherent in any free-viewing paradigm. The results suggest that early visual components-namely, the lambda response and N1-of the FRPs are sensitive to luminance and spatial frequency around fixation, separate from modulation due to underlying differences in eye-movement characteristics. Together, our results demonstrate the utility of studying the influence of image statistics on FRPs using a deconvolution modeling approach to control for overlapping neural activity and oculomotor covariates.

Refixation behavior in naturalistic viewing: Methods, mechanisms, and neural correlates

When freely viewing a scene, the eyes often return to previously visited locations. By tracking eye movements and coregistering eye movements and EEG, such refixations are shown to have multiple roles: repairing insufficient encoding from precursor fixations, supporting ongoing viewing by resampling relevant locations prioritized by precursor fixations, and aiding the construction of memory representations. All these functions of refixation behavior are understood to be underpinned by three oculomotor and cognitive systems and their associated brain structures. First, immediate saccade planning prior to refixations involves attentional selection of candidate locations to revisit. This process is likely supported by the dorsal attentional network. Second, visual working memory, involved in maintaining task-related information, is likely supported by the visual cortex. Third, higher-order relevance of scene locations, which depends on general knowledge and understanding of scene meaning, is likely supported by the hippocampal memory system. Working together, these structures bring about viewing behavior that balances exploring previously unvisited areas of a scene with exploiting visited areas through refixations.

Saccade execution increases the preview effect with faces: An EEG and eye-tracking coregistration study

Under naturalistic viewing conditions, humans conduct about three to four saccadic eye movements per second. These dynamics imply that in real life, humans rarely see something completely new; there is usually a preview of the upcoming foveal input from extrafoveal regions of the visual field. In line with results from the field of reading research, we have shown with EEG and eye-tracking coregistration that an extrafoveal preview also affects postsaccadic visual object processing and facilitates discrimination. Here, we ask whether this preview effect in the fixation-locked N170, and in manual responses to the postsaccadic target face (tilt discrimination), requires saccade execution. Participants performed a gaze-contingent experiment in which extrafoveal face images could change their orientation during a saccade directed to them. In a control block, participants maintained stable gaze throughout the experiment and the extrafoveal face reappeared foveally after a simulated saccade latency. Compared with this no-saccade condition, the neural and the behavioral preview effects were much larger in the saccade condition. We also found shorter first fixation durations after an invalid preview, which is in contrast to reading studies. We interpret the increased preview effect under saccade execution as the result of the additional sensorimotor processes that come with gaze behavior compared with visual perception under stable fixation. In addition, our findings call into question whether EEG studies with fixed gaze capture key properties and dynamics of active, natural vision.

Is covert attention necessary for programming accurate saccades? Evidence from saccade-locked event-related potentials

For decades, researchers have assumed that shifts of covert attention mandatorily occur prior to eye movements to improve perceptual processing of objects before they are fixated. However, recent research suggests that the N2pc component-a neural measure of covert attentional allocation-does not always precede eye movements. The current study investigated whether the N2pc component mandatorily precedes eye movements and assessed its role in the accuracy of gaze control. In three experiments, participants searched for a letter of a specific color (e.g., red) and directed gaze to it as a response. Electroencephalograms and eye movements were coregistered to determine whether neural markers of covert attention preceded the initial shift of gaze. The results showed that the presaccadic N2pc only occurred under limited conditions: when there were many potential target locations and distractors. Crucially, there was no evidence that the presence or magnitude of the presaccadic N2pc was associated with improved eye movement accuracy in any of the experiments. Interestingly, ERP decoding analyses were able to classify the target location well before the eyes started to move, which likely reflects a presaccadic cognitive process that is distinct from the attentional process measured by the N2pc. Ultimately, we conclude that the covert attentional mechanism indexed by the N2pc is not necessary for precise gaze control.

Differentiation in theta and gamma activation in weight-shifting learning between people with parkinson's disease of different anxiety severities

Anxiety and postural control deficits may be related in people with Parkinson's disease (PwPD). However, the association between anxiety levels and weight-shifting control remains ambiguous. This study investigated whether 1) weight-shifting control differed between PwPD with and without anxiety, and 2) the learning effect of weight-shifting differed between the two populations. Additionally, we evaluated cortical activities to investigate neural mechanisms underlying weight-shifting control. Twenty-eight PwPD (14 anxiety, 14 nonanxiety) participated in a 5-day weight-shifting study by coupling the bearing weight of their more-affected leg to a sinusoidal target at 0.25 Hz. We tested the weight-shifting control on day 1 (pretest), day 3 (posttest), and day 5 (retention test) with a learning session on day 3. The error and jerk of weight-shifting trajectory and the theta and gamma powers of electroencephalography in prefrontal, frontal, sensorimotor and parietal-occipital areas were measured. At the pretest, the anxiety group showed larger error and smaller jerk of weight-shifting with greater prefrontal theta, frontal gamma, and sensorimotor gamma powers than the nonanxiety group. Anxiety intensity was correlated positively with weight-shifting error and theta power but negatively with weight-shifting jerk. Reduced weight-shifting error with increased theta power after weight-shifting learning was observed in the nonanxiety group. However, the anxiety group showed decreased gamma power after weight-shifting learning without behavior change. Our findings suggest differential weight-shifting control and associated cortical activation between PwPD with and without anxiety. In addition, anxiety would deteriorate weight-shifting control and hinder weight-shifting learning benefits in PwPD, leading to less weight-shifting accuracy and correction.

Technical challenges in REM sleep microstructure classification: A study of patients with REM sleep behaviour disorder

While commonly treated as a uniform state in practice, rapid eye movement sleep contains two distinct microstructures-phasic (presence of rapid eye movement) and tonic (no rapid eye movement). This study aims to identify technical challenges during rapid eye movement sleep microstructure visual classification in patients with rapid eye movement sleep behaviour disorder, and to propose solutions to enhance reliability between scorers. Fifty-seven sleep recordings were randomly allocated into three subsequent batches (n = 10, 13 and 34) for scoring. To reduce single-centre bias, we recruited three raters/scorers, with each trained from a different institution. Two raters independently scored each 30-s rapid eye movement sleep into 10 × fSEM3-s phasic/tonic microstructures based on the AASM guidelines. The third rater acted as an "arbitrator" to resolve opposite opinions persisting during the revision between batches. Besides interrater differences in artefact rejection rate, interrater variance frequently occurred due to transitioning between microstructures and moderate-to-severe muscular/electrode artefact interference. To enhance interrater agreement, a rapid eye movement scoring schematic graph was developed, incorporating proxy electrode use, filters and cut-offs for microstructure transitioning. To assess potential effectiveness of the schematic graph proposed, raters were instructed to systematically apply it in scoring for the third batch. Of the 34 recordings, 27 reached a Cohen's kappa score above 0.8 (i.e. almost perfect agreement between raters), significantly improved from the prior batches (p = 0.0003, Kruskal-Wallis test). Our study illustrated potential solutions and guidance for challenges that may be encountered during rapid eye movement sleep microstructure classification.

DERCo: A Dataset for Human Behaviour in Reading Comprehension Using EEG

This paper introduces the DERCo (Dublin EEG-based Reading Experiment Corpus), a language resource combining electroencephalography (EEG) and next-word prediction data obtained from participants reading narrative texts. The dataset comprises behavioral data collected from 500 participants recruited through the Amazon Mechanical Turk online crowd-sourcing platform, along with EEG recordings from 22 healthy adult native English speakers. The online experiment was designed to examine the context-based word prediction by a large sample of participants, while the EEG-based experiment was developed to extend the validation of behavioral next-word predictability. Online participants were instructed to predict upcoming words and complete entire stories. Cloze probabilities were then calculated for each word so that this predictability measure could be used to support various analyses pertaining to semantic context effects in the EEG recordings. EEG-based analyses revealed significant differences between high and low predictable words, demonstrating one important type of potential analysis that necessitates close integration of these two datasets. This material is a valuable resource for researchers in neurolinguistics due to the word-level EEG recordings in context.

The impact of social comparison on self-deception: An event-related potentials study

Self-deception refers to an individual holding inflated beliefs about their abilities, and it plays a crucial role in human behavior and decision-making. The present study employed event-related potentials (ERPs) technique to explore the neural responses to the impacts of social comparison direction and comparison gap on self-deceptive behavior. They were instructed to predict their performance in the forward-looking paradigm. Behavioral responses and neural reactions during the decision-making process were documented. The behavioral results indicated that, in contrast to the downward comparison condition, participants engaged in upward comparison exhibited more occurrences of self-deception. However, within the context of upward comparison, participants demonstrated a higher frequency of self-deception in the large gap condition compared with the small gap condition. The ERP results showed that induced self-deception under conditions with a large comparative gap between participants and their paired counterparts stimulated larger P300 and smaller N400 amplitude than under conditions with a small gap. However, when participants were in the upward comparison situation, the late positive potential (LPP) amplitude induced by self-deception behavior in the condition of a large comparison gap between participants and paired opponents was larger than that in the condition of a small comparison gap. These results indicated that individuals in the large gap group feel strong unfairness and negative emotions. More importantly, the self-deception induced by the large gap group in the upward comparison situation used fewer cognitive resources than the small gap condition, whereas the individuals in the downward comparison situation did not show the difference in cognitive resources.

Sequential and domain-specific processing mechanisms for dual cognitive-emotional conflict

This study aimed to investigate how the cognitive control system resolves conflicts when cognitive and emotional conflicts occur simultaneously, and how it performs. To achieve this, a factorial task-crossing design was employed, combining the spatial Simon task and the face-word emotional interference task, allowing cognitive and emotional conflicts to occur concurrently within a single trial. The results revealed that the Simon cognitive conflict was only associated with N2 and early SP, while it did not affect the amplitude of N450 and late SP. Conversely, the face-word emotional conflict affected the amplitude of N450 and late SP, but had no impact on N2 and early SP. These findings demonstrate the adaptive sequencing organization and domain specificity in cognitive-emotional dual conflict processing, which reflects the precise and flexible orchestration and strategic adjustments of the cognitive control system. The results contribute to a better understanding of the dynamic and temporal processes involved in the cognitive control of multiple conflicts.

Direct lexical control of eye movements in Chinese reading: Evidence from the co-registration of EEG and eye tracking

The direct-lexical-control hypothesis stipulates that some aspect of a word's processing determines the duration of the fixation on that word and/or the next. Although the direct lexical control is incorporated into most current models of eye-movement control in reading, the precise implementation varies and the assumptions of the hypothesis may not be feasible given that lexical processing must occur rapidly enough to influence fixation durations. Conclusive empirical evidence supporting this hypothesis is therefore lacking. In this article, we report the results of an eye-tracking experiment using the boundary paradigm in which native speakers of Chinese read sentences in which target words were either high- or low-frequency and preceded by a valid or invalid preview. Eye movements were co-registered with electroencephalography, allowing standard analyses of eye-movement measures, divergence point analyses of fixation-duration distributions, and fixated-related potentials on the target words. These analyses collectively provide strong behavioral and neural evidence of early lexical processing and thus strong support for the direct-lexical-control hypothesis. We discuss the implications of the findings for our understanding of how the hypothesis might be implemented, the neural systems that support skilled reading, and the nature of eye-movement control in the reading of Chinese versus alphabetic scripts.

Decoding Remapped Spatial Information in the Peri-Saccadic Period

It has been suggested that, prior to a saccade, visual neurons predictively respond to stimuli that will fall in their receptive fields after completion of the saccade. This saccadic remapping process is thought to compensate for the shift of the visual world across the retina caused by eye movements. To map the timing of this predictive process in the brain, we recorded neural activity using electroencephalography during a saccade task. Human participants (male and female) made saccades between two fixation points while covertly attending to oriented gratings briefly presented at various locations on the screen. Data recorded during trials in which participants maintained fixation were used to train classifiers on stimuli in different positions. Subsequently, data collected during saccade trials were used to test for the presence of remapped stimulus information at the post-saccadic retinotopic location in the peri-saccadic period, providing unique insight into when remapped information becomes available. We found that the stimulus could be decoded at the remapped location ∼180 ms post-stimulus onset, but only when the stimulus was presented 100-200 ms before saccade onset. Within this range, we found that the timing of remapping was dictated by stimulus onset rather than saccade onset. We conclude that presenting the stimulus immediately before the saccade allows for optimal integration of the corollary discharge signal with the incoming peripheral visual information, resulting in a remapping of activation to the relevant post-saccadic retinotopic neurons.

Suppression and omission effects in auditory predictive processing-Two of the same?

Recent theories describe perception as an inferential process based on internal predictive models that are adjusted by prediction violations (prediction error). Two different modulations of the auditory N1 event-related brain potential component are often discussed as an expression of auditory predictive processing. The sound-related N1 component is attenuated for self-generated sounds compared to the N1 elicited by externally generated sounds (N1 suppression). An omission-related component in the N1 time-range is elicited when the self-generated sounds are occasionally omitted (omission N1). Both phenomena were explained by action-related forward modelling, which takes place when the sensory input is predictable: prediction error signals are reduced when predicted sensory input is presented (N1 suppression) and elicited when predicted sensory input is omitted (omission N1). This common theoretical account is appealing but has not yet been directly tested. We manipulated the predictability of a sound in a self-generation paradigm in which, in two conditions, either 80% or 50% of the button presses did generate a sound, inducing a strong or a weak expectation for the occurrence of the sound. Consistent with the forward modelling account, an omission N1 was observed in the 80% but not in the 50% condition. However, N1 suppression was highly similar in both conditions. Thus, our results demonstrate a clear effect of predictability for the omission N1 but not for the N1 suppression. These results imply that the two phenomena rely (at least in part) on different mechanisms and challenge prediction related accounts of N1 suppression.

Do Miniature Eye Movements Affect Neurofeedback Training Performance? A Combined EEG-Eye Tracking Study

EEG-based neurofeedback is a prominent method to modulate one's own brain activity in a desired direction. However, the EEG signal can be disturbed by artifacts, e.g., eye movements, which can consequently confound the neurofeedback performance. Involuntary miniature eye movements can be hardly detected by conventional EEG correction methods such as recording the electro-oculogram (EOG) and subtracting EOG activity from the EEG signal. However, such miniature eye movements can influence EEG activity, especially in the Gamma frequency range, enormously. In the present study, we investigated whether power in different EEG frequencies can be effectively modulated by self-control of brain signals during neurofeedback training and/or whether changes in EEG power are provoked by miniature eye movements during the training. To this end, 24 participants performed one session of SMR and one session of Gamma neurofeedback training. Additionally, in each training session sham feedback was performed. An eye tracker was used to detect miniature eye movements (< 1°) during neurofeedback training. About two thirds of the participants were able to increase their SMR power over the course of NF training, while one third was able to increase Gamma power. Generally, miniature eye movements induced a strong Gamma power increase. The number of eye movements also increased numerically over the course of the NF training. However, we did not find a significant relationship with the NF training performance. This is a first indication that miniature saccades do not affect NF training performance, but should not be neglected during NF training. Our results have to be confirmed in future studies.

Why behaviour matters: Studying inter-brain coordination during child-caregiver interaction

Modern technology allows for simultaneous neuroimaging from interacting caregiver-child dyads. Whereas most analyses that examine the coordination between brain regions within an individual brain do so by measuring changes relative to observed events, studies that examine coordination between two interacting brains generally do this by measuring average intra-brain coordination across entire blocks or experimental conditions. In other words, they do not examine changes in inter-brain coordination relative to individual behavioural events. Here, we discuss the limitations of this approach. First, we present data suggesting that fine-grained temporal interdependencies in behaviour can leave residual artifact in neuroimaging data. We show how artifact can manifest as both power and (through that) phase synchrony effects in EEG and affect wavelet transform coherence in fNIRS analyses. Second, we discuss different possible mechanistic explanations of how inter-brain coordination is established and maintained. We argue that non-event-locked approaches struggle to differentiate between them. Instead, we contend that approaches which examine how interpersonal dynamics change around behavioural events have better potential for addressing possible artifactual confounds and for teasing apart the overlapping mechanisms that drive changes in inter-brain coordination.

Electrophysiological evidence of lexical processing impacted by foreign language reading anxiety

Extensive studies have been conducted on the impact of foreign language reading anxiety on reading, primarily focusing on pedagogy and behavior but lacking electrophysiological evidence. The current study aimed to investigate the influence of foreign language reading anxiety on reading and its underlying mechanisms. The results revealed a negative correlation between foreign language reading anxiety and foreign language reading performance, irrespective of the native language. Adults with low levels of foreign language reading anxiety (LFLRA) demonstrated a significant difference in early lexical component N170 amplitude between foreign and native languages. However, this effect was not observed in adults with high levels of foreign language reading anxiety (HFLRA). In terms of N170 latency, HFLRA showed a longer N170 for the foreign language compared to the native language. Furthermore, the N170 effects were predominantly localized over the left occipitotemporal electrodes. Regarding N400 latency, a significant difference was found in LFLRA individuals between foreign and native language processing, while HFLRA individuals did not exhibit this difference. These findings suggest that HFLRA individuals experience inefficient lexical processing (such as orthography or semantics) during reading in foreign language.

Multi-Algorithm Artifact Correction (MAAC) procedure part one: Algorithm and example

The Multi-Algorithm Artifact Correction (MAAC) procedure is presented for electroencephalographic (EEG) data, as made freely available in the open-source EP Toolkit (Dien, 2010). First the major EEG artifact correction methods (regression, spatial filters, principal components analysis, and independent components analysis) are reviewed. Contrary to the dominant approach of picking one method that is thought to be most effective, this review concludes that none are globally superior, but rather each has strengths and weaknesses. Then each of the major artifact types are reviewed (Blink, Corneo-Retinal Dipole, Saccadic Spike Potential, and Movement). For each one, it is proposed that one of the major correction methods is best matched to address it, resulting in the MAAC procedure. The MAAC itself is then presented, as implemented in the EP Toolkit, in order to provide a sense of the user experience. The primary goal of this present paper is to make the conceptual argument for the MAAC approach.

The impact of electrode selection for ocular correction on the reward positivity and late positive potential components in adolescents

Electroencephalography (EEG) data processing to derive event-related potentials (ERPs) follows a standard set of procedures to maximize signal-to-noise ratio. This often includes ocular correction, which corrects for artifacts introduced by eye movements, typically measured by electrooculogram (EOG) using facial electrodes near the eyes. Yet, attaching electrodes to the face may be uncomfortable for some populations, best to avoid in some situations, and contribute to data loss. Eye movements can also be measured using electrodes in a standard 10-20 EEG cap. An examination of the impact of electrode selection on ERPs is needed to inform best practices. The present study examined data quality when using different electrodes to measure eye movements for ocular correction (i.e., facial electrodes, cap electrodes, and no ocular correction) for two well-established and widely studied ERP components (i.e., reward positivity, RewP; and late positive potential, LPP) elicited in adolescents (N = 34). Results revealed comparable split-half reliability and standardized measurement error (SME) between facial and cap electrode approaches, with lower SME for the RewP with facial or cap electrodes compared to no ocular correction. Few significant differences in mean amplitude of ERPs were observed, but the LPP to positive images differed when using facial compared to cap electrodes. Findings provide preliminary evidence of the ability to collect high-quality ERP data without facial electrodes. However, when using cap electrodes for EOG measurement and ocular correction, it is recommended to use consistent procedures across the sample or statistically examine the impact of ocular correction procedures on results.

Neurocognitive responses to spatial design behaviors and tools among interior architecture students: a pilot study

The impact of emotions on human behavior is substantial, and the ability to recognize people's feelings has a wide range of practical applications including education. Here, the methods and tools of education are being calibrated according to the data gained over electroencephalogram (EEG) signals. The issue of which design tools would be ideal in the future of interior architecture education, is an uncertain field. It is important to measure the students' emotional states while using manual and digital design tools to determine the different impacts. Brain-computer interfaces have made it possible to monitor emotional states in a way that is both convenient and economical. In the research of emotion recognition, EEG signals have been employed, and the resulting literature explains basic emotions as well as complicated scenarios that are created from the combination of numerous basic emotions. The objective of this study is to investigate the emotional states and degrees of attachment experienced by interior architecture students while engaging in their design processes. This includes examining the use of 2D or 3D tools, whether manual or digital, and identifying any changes in design tool usage and behaviors that may be influenced by different teaching techniques. Accordingly, the hierarchical clustering which is a technique used in data analysis to group objects into a hierarchical structure of clusters based on their similarities has been conducted.

The influence of self-esteem on interpersonal and competence evaluations: electrophysiological evidence from an ERP study

Using event-related potentials, this study examined how self-esteem affects neural responses to competence (interpersonal) feedback when the need for relatedness (competence) is thwarted or met. Participants with low and high self-esteem acted as advisors who selected one of two options for a putative advisee. Subsequently, they passively observed the advisee, accepted, or rejected their advice (i.e. interpersonal feedback) and received correct or incorrect outcomes (i.e. competence feedback). When interpersonal feedback was followed by competence feedback, high self-esteem participants showed a smaller P3 following incorrect than correct outcomes, irrespective of whether the advice had been accepted or rejected. However, low self-esteem participants showed this P3 effect only when the advice was rejected, and the P3 difference disappeared when the advice was accepted. When competence feedback was followed by interpersonal feedback, both low self-esteem and high self-esteem individuals showed a larger P2 for rejection than for acceptance and a larger late potential component for incorrect than correct outcomes. These findings suggest that when interpersonal feedback is followed by competence feedback, low self-esteem and high self-esteem individuals have a desire for self-positivity. When competence feedback is followed by interpersonal feedback, they may have motives for self-change. Our findings shed light on the motivational mechanisms for self-esteem and feedback.

Synchronous functional magnetic resonance eye imaging, video ophthalmoscopy, and eye surface imaging reveal the human brain and eye pulsation mechanisms

The eye possesses a paravascular solute transport pathway that is driven by physiological pulsations, resembling the brain glymphatic pathway. We developed synchronous multimodal imaging tools aimed at measuring the driving pulsations of the human eye, using an eye-tracking functional eye camera (FEC) compatible with magnetic resonance imaging (MRI) for measuring eye surface pulsations. Special optics enabled integration of the FEC with MRI-compatible video ophthalmoscopy (MRcVO) for simultaneous retinal imaging along with functional eye MRI imaging (fMREye) of the BOLD (blood oxygen level dependent) contrast. Upon optimizing the fMREye parameters, we measured the power of the physiological (vasomotor, respiratory, and cardiac) eye and brain pulsations by fast Fourier transform (FFT) power analysis. The human eye pulsated in all three physiological pulse bands, most prominently in the respiratory band. The FFT power means of physiological pulsation for two adjacent slices was significantly higher than in one-slice scans (RESP1 vs. RESP2; df = 5, p = 0.045). FEC and MRcVO confirmed the respiratory pulsations at the eye surface and retina. We conclude that in addition to the known cardiovascular pulsation, the human eye also has respiratory and vasomotor pulsation mechanisms, which are now amenable to study using non-invasive multimodal imaging of eye fluidics.

Predicting the subjective intensity of imagined experiences from electrophysiological measures of oscillatory brain activity

Most people can conjure images and sounds that they experience in their minds. There are, however, marked individual differences. Some people report that they cannot generate imagined sensory experiences at all (aphantasics) and others report that they have unusually intense imagined experiences (hyper-phantasics). These individual differences have been linked to activity in sensory brain regions, driven by feedback. We would therefore expect imagined experiences to be associated with specific frequencies of oscillatory brain activity, as these can be a hallmark of neural interactions within and across regions of the brain. Replicating a number of other studies, relative to a Resting-State we find that the act of engaging in auditory or in visual imagery is linked to reductions in the power of oscillatory brain activity across a broad range of frequencies, with prominent peaks in the alpha band (8-12 Hz). This oscillatory activity, however, did not predict individual differences in the subjective intensity of imagined experiences. For audio imagery, these were rather predicted by reductions within the theta (6-9 Hz) and gamma (33-38 Hz) bands, and by increases in beta (15-17 Hz) band activity. For visual imagery these were predicted by reductions in lower (14-16 Hz) and upper (29-32 Hz) beta band activity, and by an increase in mid-beta band (24-26 Hz) activity. Our data suggest that there is sufficient ground truth in the subjective reports people use to describe the intensity of their imagined sensory experiences to allow these to be linked to the power of distinct rhythms of brain activity. In future, we hope to combine this approach with better measures of the subjective intensity of imagined sensory experiences to provide a clearer picture of individual differences in the subjective intensity of imagined experiences, and of why these eventuate.

Voluntary Respiration Control: Signature Analysis by EEG

The perception of voluntary respiratory consciousness is quite important in some situations, such as respiratory assistance and respiratory rehabilitation training, and the key signatures about voluntary respiration control may lie in the neural signals from brain manifested as electroencephalography (EEG). The present work aims to explore whether there exists correlation between voluntary respiration and scalp EEG. Evoke voluntary respiration of different intensities, while collecting EEG and respiration signal synchronously. Data from 11 participants were analyzed. Spectrum characteristics at low-frequency band were studied. Computation of EEG-respiration phase lock value (PLV) and EEG sample entropy were conducted as well. When breathing voluntarily, the 0-2 Hz band EEG power is significantly enhanced in frontal and right-parietal area. The distance between main peaks belonging to the two signals in 0-2 Hz spectrum graph tends to get smaller, while EEG-respiration PLV increases in frontal area. Besides, the sample entropy of EEG shows a trend of decreasing during voluntary respiration in both areas. There's a strong correlation between voluntary respiration and scalp EEG. Significance: The discoveries will provide guidelines for developing a voluntary respiratory consciousness identifying method and make it possible to monitor people's intention of respiration by noninvasive BCI.

Epigenetic modification of the oxytocin receptor gene is associated with child-parent neural synchrony during competition

Interpersonal neural synchrony (INS) occurs when neural electrical activity temporally aligns between individuals during social interactions. It has been used as a metric for interpersonal closeness, often during naturalistic child-parent interactions. This study evaluated whether other biological correlates of social processing predicted the prevalence of INS during child-parent interactions, and whether their observed cooperativity modulated this association. Child-parent dyads (n = 27) performed a visuospatial tower-building task in cooperative and competitive conditions. Neural activity was recorded using mobile electroencephalogram (EEG) headsets, and experimenters coded video-recordings post-hoc for behavioral attunement. DNA methylation of the oxytocin receptor gene (OXTRm) was measured, an epigenetic modification associated with reduced oxytocin activity and socioemotional functioning. Greater INS during competition was associated with lower child OXTRm, while greater behavioral attunement during competition and cooperation was associated with higher parent OXTRm. These differential relationships suggest that interpersonal dynamics as measured by INS may be similarly reflected by other biological markers of social functioning, irrespective of observed behavior. Children's self-perceived communication skill also showed opposite associations with parent and child OXTRm, suggesting complex relationships between children's and their parents' social functioning. Our findings have implications for ongoing developmental research, supporting the utility of biological metrics in characterizing interpersonal relationships.

The influence of altruistic personality, interpersonal distance and social observation on prosocial behavior: An event-related potential (ERP) study

The psychological mechanisms that high and low altruists exhibit in different contexts remain unknown. This study examined the underlying mechanisms of the effect of altruistic personality, social observation, and interpersonal distance on prosocial behavior using event-related potentials (ERPs). Participants with high and low altruism were asked to make prosocial or non-prosocial choices toward different interpersonal distances (friends, acquaintances, or strangers) under the (non)observer condition. The electrophysiological responses to the choice stimuli were simultaneously recorded. The behavioral results demonstrated that high altruists had more prosocial choices, and these choices were unaffected by interpersonal distance and social observation. However, low altruists made more prosocial choices toward friends and acquaintances under the observer than nonobserver conditions, whereas their prosocial choices toward strangers showed no difference. The ERP results demonstrated that low altruists showed more negative N2 when the choice stimuli were toward strangers and acquaintances or under the nonobserver condition. Furthermore, low altruists showed larger P3 under the observer than nonobserver conditions when the choice stimuli were toward friends and acquaintances, while this difference was absent when the choice stimuli were toward strangers. However, for high altruists, no effect of interpersonal distance and social observation was observed in N2 and P3. These results suggest that the prosocial behavior of low altruists is mainly driven by reputational incentives, whereas high altruists are primarily motivated by concern about the well-being of others. Our findings provide insights into the prosocial behavior of high and low altruists in different contexts and support the empathy-altruism hypothesis.

Tracking gaze position from EEG: Exploring the possibility of an EEG-based virtual eye-tracker

INTRODUCTION

Ocular artifact has long been viewed as an impediment to the interpretation of electroencephalogram (EEG) signals in basic and applied research. Today, the use of blind source separation (BSS) methods, including independent component analysis (ICA) and second-order blind identification (SOBI), is considered an essential step in improving the quality of neural signals. Recently, we introduced a method consisting of SOBI and a discriminant and similarity (DANS)-based identification method, capable of identifying and extracting eye movement-related components. These recovered components can be localized within ocular structures with a high goodness of fit (>95%). This raised the possibility that such EEG-derived SOBI components may be used to build predictive models for tracking gaze position.

METHODS

As proof of this new concept, we designed an EEG-based virtual eye-tracker (EEG-VET) for tracking eye movement from EEG alone. The EEG-VET is composed of a SOBI algorithm for separating EEG signals into different components, a DANS algorithm for automatically identifying ocular components, and a linear model to transfer ocular components into gaze positions.

RESULTS

The prototype of EEG-VET achieved an accuracy of 0.920° and precision of 1.510° of a visual angle in the best participant, whereas an average accuracy of 1.008° ± 0.357° and a precision of 2.348° ± 0.580° of a visual angle across all participants (N = 18).

CONCLUSION

This work offers a novel approach that readily co-registers eye movement and neural signals from a single-EEG recording, thus increasing the ease of studying neural mechanisms underlying natural cognition in the context of free eye movement.

Crossmodal visual predictions elicit spatially specific early visual cortex activity but later than real visual stimuli

Previous studies have indicated that crossmodal visual predictions are instrumental in controlling early visual cortex activity. The exact time course and spatial precision of such crossmodal top-down influences on the visual cortex have been unknown. In the present study, participants were exposed to audiovisual combinations comprising one of two sounds and a Gabor patch either in the top left or in the bottom right visual field. Event-related potentials (ERPs) were recorded to these frequent crossmodal combinations (standards) as well as to trials in which the visual stimulus was omitted (omissions) or the visual and auditory stimuli were recombined (deviants). Standards and deviants elicited an ERP between 50 and 100 ms of opposite polarity known as the C1 effect commonly associated with retinotopic processing in early visual cortex. By contrast, a C1 effect was not observed in omission trials. Spatially specific omission and mismatch effects (deviants minus standards) started only later with a latency of 230 ms and 170 ms, respectively. These results suggest that crossmodal visual predictions control visual cortex activity in a spatially specific manner. However, visual predictions do not modulate visual cortex activity with the same timing as visual stimulation activates these areas but rather seem to involve distinct neural mechanisms. This article is part of the theme issue 'Decision and control processes in multisensory perception'.

Influence of voluntary action and outcome valence on the sense of agency

Recent studies have revealed that people might experience a lessened sense of agency for negative consequences by claiming that they were obeying orders. However, little is known about the cognitive neural mechanism behind the reduced sense of agency when individuals are forced to inflict physical harm on others. This study adopted temporal estimation tasks to investigate the internal mechanism of voluntary action on the sense of agency and the moderating effect of outcome valence as measured by event-related potentials (ERPs). In the temporal estimation task, participants were asked to make trade-offs of monetary gains for themselves against painful electric stimuli experienced by strangers, subjectively estimated the perceptual temporal interval between keypress actions (i.e., free or coercive actions) and consequent outcomes (i.e., positive or negative tones) and rated the feeling of control. The results showed that perceived temporal interval was shorter for positive tones compared with negative tones in the coercive condition, and induced more negative N1 and N300 amplitudes, which indicated that the implicit sense of agency was higher. However, the explicit sense of agency was stronger in the free condition than in the coercive condition, which was not influenced by outcome valence. We discuss the implications of utilizing positive feedback and free choice as significant strategies for those experiencing the abnormal sense of agency.

The effects of ritual and self-control resources depletion on deceptive behavior: Evidence from behavioral and ERPs studies

Although researchers have indicated that individuals with depleted self-control resources have lower self-control behavior and exhibit more deceptive behaviors, recent psychological studies have shown that ritual can improve self-control and increase the likelihood that the individual makes prosocial decisions. However, little is known about whether ritual can regulate an individual's engagement in deceptive behavior when their self-control resources are depleted. This study adopted the spot-the-difference task to investigate the influences of ritual and self-control resources depletion on simple self-control behavior and deceptive behavior (Experiment 1 and Experiment 2); furthermore, relevant neural processes were explored using event-related potentials (Experiment 3). Experiment 1 showed that individuals with depleted self-control resources had lower self-control behavior and individuals performing a ritual had higher self-control behavior. Experiment 2 showed that individuals with depleted self-control resources exhibited more deceptive behaviors and individuals performing a ritual exhibited fewer deceptive behaviors; furthermore, ritual reduced deceptive behaviors in individuals with depleted self-control resources. Experiment 3 found that individuals with depleted self-control resources had larger P2 amplitudes after performing a ritual. Moreover, individuals with depleted self-control resources had larger LPP amplitudes over parietal sites after performing a ritual. These findings suggested that performing ritual may be an effective measure of inhibiting individuals with depleted self-control resources from engaging in deceptive behavior. Our findings verify the ego-depletion model and provide a new perspective for reducing deceptive behaviors in individuals with depleted self-control resources. We provide evidence that rituals could modulate deceptive behaviors in individuals with depleted self-control resources. This reveals that performing rituals may be an effective measure for inhibiting individuals with depleted self-control resources from engaging in deceptive behavior.

Gaze onsets during naturalistic infant-caregiver interaction associate with 'sender' but not 'receiver' neural responses, and do not lead to changes in inter-brain synchrony

Temporal coordination during infant-caregiver social interaction is thought to be crucial for supporting early language acquisition and cognitive development. Despite a growing prevalence of theories suggesting that increased inter-brain synchrony associates with many key aspects of social interactions such as mutual gaze, little is known about how this arises during development. Here, we investigated the role of mutual gaze onsets as a potential driver of inter-brain synchrony. We extracted dual EEG activity around naturally occurring gaze onsets during infant-caregiver social interactions in N = 55 dyads (mean age 12 months). We differentiated between two types of gaze onset, depending on each partners' role. 'Sender' gaze onsets were defined at a time when either the adult or the infant made a gaze shift towards their partner at a time when their partner was either already looking at them (mutual) or not looking at them (non-mutual). 'Receiver' gaze onsets were defined at a time when their partner made a gaze shift towards them at a time when either the adult or the infant was already looking at their partner (mutual) or not (non-mutual). Contrary to our hypothesis we found that, during a naturalistic interaction, both mutual and non-mutual gaze onsets were associated with changes in the sender, but not the receiver's brain activity and were not associated with increases in inter-brain synchrony above baseline. Further, we found that mutual, compared to non-mutual gaze onsets were not associated with increased inter brain synchrony. Overall, our results suggest that the effects of mutual gaze are strongest at the intra-brain level, in the 'sender' but not the 'receiver' of the mutual gaze.

Neural bases of predictions during natural reading of known statements: An EEG and eye movements co-registration study

Predictions of incoming words performed during reading have an impact on how the reader moves their eyes and on the electrical brain potentials. Eye tracking (ET) experiments show that less predictable words are fixated for longer periods of times. Electroencephalography (EEG) experiments show that these words elicit a more negative potential around 400ms (N400) after the word onset when reading one word at a time (foveated reading). Nevertheless, there was no N400 potential during the foveated reading of previously known sentences (memory-encoded), which suggests that the prediction of words from memory-encoded sentences is based on different mechanisms than predictions performed on common sentences. Here, we performed an ET-EEG co-registration experiment where participants read common and memory-encoded sentences. Our results show that the N400 potential disappear when the reader recognises the sentence. Furthermore, time-frequency analyses show a larger alpha lateralisation and a beta power increase for memory-encoded sentences. This suggests a more distributed attention and an active maintenance of the cognitive set, in concordance to the predictive coding framework.

Parallel word reading revealed by fixation-related brain potentials

During reading, the brain is confronted with many relevant objects at once. But does lexical processing occur for multiple words simultaneously? Cognitive science has yet to answer this prominent question. Recently it has been argued that the issue warrants supplementing the field's traditional toolbox (response times, eye-tracking) with neuroscientific techniques (EEG, fMRI). Indeed, according to the OB1-reader model, upcoming words need not impact oculomotor behavior per se, but parallel processing of these words must nonetheless be reflected in neural activity. Here we combined eye-tracking with EEG, time-locking the neural window of interest to the fixation on target words in sentence reading. During these fixations, we manipulated the identity of the subsequent word so that it posed either a syntactically legal or illegal continuation of the sentence. In line with previous research, oculomotor measures were unaffected. Yet, syntax impacted brain potentials as early as 100 ms after the target fixation onset. Given the EEG literature on syntax processing, the presently observed timings suggest parallel word reading. We reckon that parallel word processing typifies reading, and that OB1-reader offers a good platform for theorizing about the reading brain.

Social distance modulates the influence of social observation on pro-environmental behavior: An event-related potential (ERP) study

Pro-environmental behavior benefits the environment but is individually costly. Therefore, understanding the neural processes of pro-environmental behavior can advance our knowledge of its implicit cost-benefit calculations and mechanisms. Previous studies have focused separately on the effects of social distance and social observation on explicit pro-environmental behavior responses, yet the underlying neurophysiological mechanism remains unknown. Using the event-related potentials (ERPs), we investigated the neural responses to the effects of social distance and social observation on pro-environmental behavior. Participants were instructed to decide between self-interest and pro-environmental choices towards different social distance targets (family members, acquaintances, or strangers) under observable and non-observable conditions. The behavioral results demonstrated that the rate of pro-environmental choices towards acquaintances and strangers was higher under the observable condition than under the non-observable condition. Nonetheless, the rate of pro-environmental choices was higher-unaffected by social observation-towards family members than towards acquaintances and strangers. The ERP results showed that the P2 and P3 amplitudes were smaller under the observable than under the non-observable condition when the potential bearers of environmental decisions were acquaintances and strangers. However, this difference did not emerge when the potential bearers of environmental decisions were family members. The ERP findings of smaller P2 and P3 amplitudes suggest that social observation may reduce the deliberate calculation of personal costs, thereby promoting individuals' pro-environmental behaviors towards acquaintances and strangers.

Episodic memory formation in unrestricted viewing

The brain systems of episodic memory and oculomotor control are tightly linked, suggesting a crucial role of eye movements in memory. But little is known about the neural mechanisms of memory formation across eye movements in unrestricted viewing behavior. Here, we leverage simultaneous eye tracking and EEG recording to examine episodic memory formation in free viewing. Participants memorized multi-element events while their EEG and eye movements were concurrently recorded. Each event comprised elements from three categories (face, object, place), with two exemplars from each category, in different locations on the screen. A subsequent associative memory test assessed participants' memory for the between-category associations that specified each event. We used a deconvolution approach to overcome the problem of overlapping EEG responses to sequential saccades in free viewing. Brain activity was time-locked to the fixation onsets, and we examined EEG power in the theta and alpha frequency bands, the putative oscillatory correlates of episodic encoding mechanisms. Three modulations of fixation-related EEG predicted high subsequent memory performance: (1) theta increase at fixations after between-category gaze transitions, (2) theta and alpha increase at fixations after within-element gaze transitions, (3) alpha decrease at fixations after between-exemplar gaze transitions. Thus, event encoding with unrestricted viewing behavior was characterized by three neural mechanisms, manifested in fixation-locked theta and alpha EEG activity that rapidly turned on and off during the unfolding eye movement sequences. These three distinct neural mechanisms may be the essential building blocks that subserve the buildup of coherent episodic memories during unrestricted viewing behavior.

Removing ocular artifacts from magnetoencephalographic data on naturalistic reading of continuous texts

Naturalistic reading paradigms and stimuli consisting of long continuous texts are essential for characterizing the cortical basis of reading. Due to the highly dynamic nature of the reading process, electrophysiological brain imaging methods with high spatial and temporal resolution, such as magnetoencephalography (MEG), are ideal for tracking them. However, as electrophysiological recordings are sensitive to electromagnetic artifacts, data recorded during naturalistic reading is confounded by ocular artifacts. In this study, we evaluate two different pipelines for removing ocular artifacts from MEG data collected during continuous, naturalistic reading, with the focus on saccades and blinks. Both pipeline alternatives are based on blind source separation methods but differ fundamentally in their approach. The first alternative is a multi-part process, in which saccades are first extracted by applying Second-Order Blind Identification (SOBI) and, subsequently, FastICA is used to extract blinks. The other alternative uses a single powerful method, Adaptive Mixture ICA (AMICA), to remove all artifact types at once. The pipelines were tested, and their effects compared on MEG data recorded from 13 subjects in a naturalistic reading task where the subjects read texts with the length of multiple pages. Both pipelines performed well, extracting the artifacts in a single component per artifact type in most subjects. Signal power was reduced across the whole cortex in all studied frequency bands from 1 to 90 Hz, but especially in the frontal cortex and temporal pole. The results were largely similar for the two pipelines, with the exception that SOBI-FastICA reduced signal in the right frontal cortex in all studied frequency bands more than AMICA. However, there was considerable interindividual variation in the effects of the pipelines. As a holistic conclusion, we choose to recommend AMICA for removing artifacts from MEG data on naturalistic reading but note that the SOBI-FastICA pipeline has also various favorable characteristics.

HArtMuT-modeling eye and muscle contributors in neuroelectric imaging

Objective.Magneto- and electroencephalography (M/EEG) measurements record a mix of signals from the brain, eyes, and muscles. These signals can be disentangled for artifact cleaning e.g. using spatial filtering techniques. However, correctly localizing and identifying these components relies on head models that so far only take brain sources into account.Approach.We thus developed the Head Artifact Model using Tripoles (HArtMuT). This volume conduction head model extends to the neck and includes brain sources as well as sources representing eyes and muscles that can be modeled as single dipoles, symmetrical dipoles, and tripoles. We compared a HArtMuT four-layer boundary element model (BEM) with the EEGLAB standard head model on their localization accuracy and residual variance (RV) using a HArtMuT finite element model (FEM) as ground truth. We also evaluated the RV on real-world data of mobile participants, comparing different HArtMuT BEM types with the EEGLAB standard head model.Main results.We found that HArtMuT improves localization for all sources, especially non-brain, and localization error and RV of non-brain sources were in the same range as those of brain sources. The best results were achieved by using cortical dipoles, muscular tripoles, and ocular symmetric dipoles, but dipolar sources alone can already lead to convincing results.Significance.We conclude that HArtMuT is well suited for modeling eye and muscle contributions to the M/EEG signal. It can be used to localize sources and to identify brain, eye, and muscle components. HArtMuT is freely available and can be integrated into standard software.

WildLab: A naturalistic free viewing experiment reveals previously unknown electroencephalography signatures of face processing

Neural mechanisms of face perception are predominantly studied in well-controlled experimental settings that involve random stimulus sequences and fixed eye positions. Although powerful, the employed paradigms are far from what constitutes natural vision. Here, we demonstrate the feasibility of ecologically more valid experimental paradigms using natural viewing behaviour, by combining a free viewing paradigm on natural scenes, free of photographer bias, with advanced data processing techniques that correct for overlap effects and co-varying non-linear dependencies of multiple eye movement parameters. We validate this approach by replicating classic N170 effects in neural responses, triggered by fixation onsets (fixation event-related potentials [fERPs]). Importantly, besides finding a strong correlation between both experiments, our more natural stimulus paradigm yielded smaller variability between subjects than the classic setup. Moving beyond classic temporal and spatial effect locations, our experiment furthermore revealed previously unknown signatures of face processing: This includes category-specific modulation of the event-related potential (ERP)'s amplitude even before fixation onset, as well as adaptation effects across subsequent fixations depending on their history.

The Readiness Potential Correlates with Action-Linked Modulation of Visual Accuracy

Visual accuracy is consistently shown to be modulated around the time of the action execution. The neural underpinning of this motor-induced modulation of visual perception is still unclear. Here, we investigate with EEG whether it is related to the readiness potential, an event-related potential (ERP) linked to motor preparation. Across 18 human participants, the magnitude of visual modulation following a voluntary button press was found to correlate with the readiness potential amplitude measured during visual discrimination. Participants' amplitude of the readiness potential in a purely motor-task was also found to correlate with the extent of the motor-induced modulation of visual perception in the visuomotor task. These results provide strong evidence that perceptual changes close to action execution are associated with motor preparation processes and that this mechanism is independent of task contingencies. Further, our findings suggest that the readiness potential provides a fingerprint of individual visuomotor interaction.

Intention-based predictive information modulates auditory deviance processing

The human brain is highly responsive to (deviant) sounds violating an auditory regularity. Respective brain responses are usually investigated in situations when the sounds were produced by the experimenter. Acknowledging that humans also actively produce sounds, the present event-related potential study tested for differences in the brain responses to deviants that were produced by the listeners by pressing one of two buttons. In one condition, deviants were unpredictable with respect to the button-sound association. In another condition, deviants were predictable with high validity yielding correctly predicted deviants and incorrectly predicted (mispredicted) deviants. Temporal principal component analysis revealed deviant-specific N1 enhancement, mismatch negativity (MMN) and P3a. N1 enhancements were highly similar for each deviant type, indicating that the underlying neural mechanism is not affected by intention-based expectation about the self-produced forthcoming sound. The MMN was abolished for predictable deviants, suggesting that the intention-based prediction for a deviant can overwrite the prediction derived from the auditory regularity (predicting a standard). The P3a was present for each deviant type but was largest for mispredicted deviants. It is argued that the processes underlying P3a not only evaluate the deviant with respect to the fact that it violates an auditory regularity but also with respect to the intended sensorial effect of an action. Overall, our results specify current theories of auditory predictive processing, as they reveal that intention-based predictions exert different effects on different deviance-specific brain responses.