Unfold: an integrated toolbox for overlap correction, non-linear modeling, and regression-based EEG analysis

Neural responses to global and local visual information processing provide neural signatures of ADHD symptoms

Individuals with ADHD are thought to exhibit a reduced "global bias" in perceptual processing. This bias, found in typically developed individuals, characterizes the tendency to prioritize global over local information processing. However, the relationship between specific ADHD symptoms and global or local processing remains unclear. This study addresses this gap by employing an ensemble perception task with a large sample (N = 465). EEG recordings allowed for the isolation of neural responses to individual and global stimuli using linear regression modeling. The adult ADHD self-report scale was used to assess ADHD symptoms. The results showed a significant association between ensemble perception and early responses to global stimuli. Furthermore, inattention symptoms were associated with early responses to global stimuli, suggesting a reduced global prioritization in individuals with higher inattention scores. Moreover, inattention symptom was associated with later responses to local stimuli, as shown by attenuated neural responses to local stimuli in individuals with more severe symptoms. These findings provide insights that ADHD includes deficits in both global and local processing, challenging earlier theories that focused solely on global processing impairments.

The impact of emotional feedback in learning easy and difficult tasks - an ERP study

Learning from the emotional reaction of others is crucial in our everyday lives. We assumed that additional emotional information could be especially beneficial, when a task is difficult and the limits of working memory capacity are reached. For this reason, we examined whether a potential benefit of emotional feedback during reinforcement learning is dependent on working memory load. In addition to learning performance, we analysed the neural mechanisms of reinforcement learning by examining two event-related potentials (ERPs): feedback-related negativity (FRN) and P3b. Participants were divided into two difficulty groups (with n = 21 in the difficult and n = 22 in the easy group), performing a learning task with emotional or non-emotional feedback. Task difficulty was manipulated by varying the number of stimulus-response associations. Participants' showed learning in all conditions. Emotional feedback led to increased accuracy and decreased reaction times in both groups. However, this benefit occurred earlier in the easy condition. The detection of unexpected events, as reflected in the peak-to-peak FRN, as well as working memory updating, as reflected in the P3b, were enhanced after emotional in contrast to non-emotional feedback for both groups. In contrast, task difficulty had no effect on the detection of unexpected events but led to a P3b that was more evenly distributed over the scalp, which could indicate that additional frontal resources were recruited to perform the difficult task. Our results suggest that working memory load and emotional information independently influence feedback processing without interacting.

Humans actively reconfigure neural task states

The ability to switch between tasks is a core component of adaptive cognition, yet a mechanistic understanding of this capacity has remained elusive. In particular, there are longstanding debates over the extent to which task-switching is primarily influenced by interference from previous tasks or active preparation for upcoming tasks. We advance this debate by modeling the evolution of task representations in human electroencephalographic recordings as linear dynamical systems over a high-dimensional latent space. Using control theoretic analyses of dynamical systems fit to both brains and task-optimized neural networks, we reveal neural signatures of active preparation that reconcile the role of interference and control in task switching. These findings inform a core debate in cognitive control and offer a promising new paradigm for human neuroimaging analysis.

Comprehensive Analysis of Event-Related Potentials of Response Inhibition: The Role of Negative Urgency and Compulsivity

Behavioral and neural correlates of response inhibition are assumed to relate to impulsivity and compulsivity, but findings are inconsistent, possibly due to prior research studying these dimensions in isolation. Negative urgency, the tendency to act impulsive under negative affect, and compulsivity relate to various mental disorders and are assumed to reflect deficits in inhibitory control. However, few studies have examined how response inhibition relates to negative urgency, compulsivity, or their interaction. To address this gap, we conducted a comprehensive analysis of the behavioral and neural correlates of response inhibition and their associations with negative urgency and compulsivity. We examined 233 participants who performed a stop-signal task while electroencephalography was recorded. The analysis involved single-trial regression and latency analyses to explore the relationships with self-reported negative urgency and compulsivity. Stop-signal reaction times (SSRTs) and negative urgency were associated with an attenuated P3 effect contrasting successful stop versus go trials. Crucially, longer SSRT was associated with reduced P1 amplitudes (on successful and failed stops) and a later onset and peak of the P3. Interestingly, the opposite pattern was observed for higher negative urgency with higher P1 amplitudes and an earlier P3 onset and peak in successful stop trials. Associations with compulsivity were not observed. Considering early sensorimotor processes and latency effects are important to capture differences between negative urgency and SSRT. Higher stop-signal-related P1 amplitudes and a faster action cancellation process may compensate reduced P3-related activity in high negative urgency.

Orthographic character complexity modulates dynamic neural activity in skilled handwriting

Handwriting is an outstanding case of a highly complex and efficient fine motor skill. However, little is known about its neural underpinnings during continuous handwriting production. In the present study, we examined the effects of orthographic character complexity (i.e. the stroke number of a Chinese character) on both neural and behavioural activities during an EEG-based naturalistic fluent sentence-handwriting task from 102 adult Chinese native speakers. For each written character, the interval between finishing the preceding character and its onset (inter-character interval) as well as the amplitudes of the onset-synchronized event-related potential (ERP) in pre- and post-onset time windows was defined as dependent variables. The effects of character complexity and other confounding factors were analysed with linear mixed models. Character complexity increased the inter-character interval and significantly affected ERP amplitudes in both pre- and post-onset time windows. The ERP pattern in the pre-event time window exhibited a dipole-like activation in the left motor cortex, and its amplitude increased with character complexity in line with the documented relationship between the lateralized readiness potential and motor complexity. This study demonstrates the feasibility of studying neurocognitive processes in complex naturalistic motor tasks and extends our knowledge about the dynamic pattern of handwriting-related neural activities.

VARX Granger analysis: Models for neuroscience, physiology, sociology and econometrics

Complex systems, such as in brains, markets, and societies, exhibit internal dynamics influenced by external factors. Disentangling delayed external effects from internal dynamics within these systems is often difficult. We propose using a Vector Autoregressive model with eXogenous input (VARX) to capture delayed interactions between internal and external variables. Whereas this model aligns with Granger's statistical formalism for testing "causal relations", the connection between the two is not widely understood. Here, we bridge this gap by providing fundamental equations, user-friendly code, and demonstrations using simulated and real-world data from neuroscience, physiology, sociology, and economics. Our examples illustrate how the model avoids spurious correlation by factoring out external influences from internal dynamics, leading to more parsimonious explanations of these systems. For instance, in neural recordings we find that prolonged response of the brain can be explained as a short exogenous effect, followed by prolonged internal recurrent activity. In recordings of human physiology, we find that the model recovers established effects such as eye movements affecting pupil size and a bidirectional interaction of respiration and heart rate. We also provide methods for enhancing model efficiency, such as L2 regularization for limited data and basis functions to cope with extended delays. Additionally, we analyze model performance under various scenarios where model assumptions are violated. MATLAB, Python, and R code are provided for easy adoption: https://github.com/lcparra/varx.

Fixation-related potentials during mobile map assisted navigation in the real world: The effect of landmark visualization style

An often-proposed enhancement for mobile maps to aid assisted navigation is the presentation of landmark information, yet understanding of the manner in which they should be displayed is limited. In this study, we investigated whether the visualization of landmarks as 3D map symbols with either an abstract or realistic style influenced the subsequent processing of those landmarks during route navigation. We utilized a real-world mobile electroencephalography approach to this question by combining several tools developed to overcome the challenges typically encountered in real-world neuroscience research. We coregistered eye-movement and EEG recordings from 45 participants as they navigated through a real-world environment using a mobile map. Analyses of fixation event-related potentials revealed that the amplitude of the parietal P200 component was enhanced when participants fixated landmarks in the real world that were visualized on the mobile map in a realistic style, and that frontal P200 latencies were prolonged for landmarks depicted in either a realistic or abstract style compared with features of the environment that were not presented on the map, but only for the male participants. In contrast, we did not observe any significant effects of landmark visualization style on visual P1-N1 peaks or the parietal late positive component. Overall, the findings indicate that the cognitive matching process between landmarks seen in the environment and those previously seen on a map is facilitated by more realistic map display, while low-level perceptual processing of landmarks and recall of associated information are unaffected by map visualization style.

(Micro)saccade-related potentials during face recognition: A study combining EEG, eye-tracking, and deconvolution modeling

Under natural viewing conditions, complex stimuli such as human faces are typically looked at several times in succession, implying that their recognition may unfold across multiple eye fixations. Although electrophysiological (EEG) experiments on face recognition typically prohibit eye movements, participants still execute frequent (micro)saccades on the face, each of which generates its own visuocortical response. This finding raises the question of whether the fixation-related potentials (FRPs) evoked by these tiny gaze shifts also contain psychologically valuable information about face processing. Here, we investigated this question by corecording EEG and eye movements in an experiment with emotional faces (happy, angry, neutral). Deconvolution modeling was used to separate the stimulus ERPs to face onset from the FRPs generated by subsequent microsaccades-induced refixations on the face. As expected, stimulus ERPs exhibited typical emotion effects, with a larger early posterior negativity (EPN) for happy/angry compared with neutral faces. Eye tracking confirmed that participants made small saccades in 98% of the trials, which were often aimed at the left eye of the stimulus face. However, while each saccade produced a strong response over visual areas, this response was unaffected by the face's emotional expression, both for the first and for subsequent (micro)saccades. This finding suggests that the face's affective content is rapidly evaluated after stimulus onset, leading to only a short-lived sensory enhancement by arousing stimuli that does not repeat itself during immediate refixations. Methodologically, our work demonstrates how eye tracking and deconvolution modeling can be used to extract several brain responses from each EEG trial, providing insights into neural processing at different latencies after stimulus onset.

Neural mechanism underlying preview effects and masked priming effects in visual word processing

Two classic experimental paradigms - masked repetition priming and the boundary paradigm - have played a pivotal role in understanding the process of visual word recognition. Traditionally, these paradigms have been employed by different communities of researchers, with their own long-standing research traditions. Nevertheless, a review of the literature suggests that the brain-electric correlates of word processing established with both paradigms may show interesting similarities, in particular with regard to the location, timing, and direction of N1 and N250 effects. However, as of yet, no direct comparison has been undertaken between the two paradigms. In the current study, we used combined eye-tracking/EEG to perform such a within-subject comparison using the same materials (single Chinese characters) as stimuli. To facilitate direct comparisons, we used a simplified version of the boundary paradigm - the single word boundary paradigm. Our results show the typical early repetition effects of N1 and N250 for both paradigms. However, repetition effects in N250 (i.e., a reduced negativity following identical-word primes/previews as compared to different-word primes/previews) were larger with the single word boundary paradigm than with masked priming. For N1 effects, repetition effects were similar across the two paradigms, showing a larger N1 after repetitions as compared to alternations. Therefore, the results indicate that at the neural level, a briefly presented and masked foveal prime produces qualitatively similar facilitatory effects on visual word recognition as a parafoveal preview before a single saccade, although such effects appear to be stronger in the latter case.

Fixation-related potentials during a virtual navigation task: The influence of image statistics on early cortical processing

Historically, electrophysiological correlates of scene processing have been studied with experiments using static stimuli presented for discrete timescales where participants maintain a fixed eye position. Gaps remain in generalizing these findings to real-world conditions where eye movements are made to select new visual information and where the environment remains stable but changes with our position and orientation in space, driving dynamic visual stimulation. Co-recording of eye movements and electroencephalography (EEG) is an approach to leverage fixations as time-locking events in the EEG recording under free-viewing conditions to create fixation-related potentials (FRPs), providing a neural snapshot in which to study visual processing under naturalistic conditions. The current experiment aimed to explore the influence of low-level image statistics-specifically, luminance and a metric of spatial frequency (slope of the amplitude spectrum)-on the early visual components evoked from fixation onsets in a free-viewing visual search and navigation task using a virtual environment. This research combines FRPs with an optimized approach to remove ocular artifacts and deconvolution modeling to correct for overlapping neural activity inherent in any free-viewing paradigm. The results suggest that early visual components-namely, the lambda response and N1-of the FRPs are sensitive to luminance and spatial frequency around fixation, separate from modulation due to underlying differences in eye-movement characteristics. Together, our results demonstrate the utility of studying the influence of image statistics on FRPs using a deconvolution modeling approach to control for overlapping neural activity and oculomotor covariates.

Refixation behavior in naturalistic viewing: Methods, mechanisms, and neural correlates

When freely viewing a scene, the eyes often return to previously visited locations. By tracking eye movements and coregistering eye movements and EEG, such refixations are shown to have multiple roles: repairing insufficient encoding from precursor fixations, supporting ongoing viewing by resampling relevant locations prioritized by precursor fixations, and aiding the construction of memory representations. All these functions of refixation behavior are understood to be underpinned by three oculomotor and cognitive systems and their associated brain structures. First, immediate saccade planning prior to refixations involves attentional selection of candidate locations to revisit. This process is likely supported by the dorsal attentional network. Second, visual working memory, involved in maintaining task-related information, is likely supported by the visual cortex. Third, higher-order relevance of scene locations, which depends on general knowledge and understanding of scene meaning, is likely supported by the hippocampal memory system. Working together, these structures bring about viewing behavior that balances exploring previously unvisited areas of a scene with exploiting visited areas through refixations.

Neural correlates for word-frequency effect in Chinese natural reading

Word frequency effect has always been of interest for reading research because of its critical role in exploring mental processing underlying reading behaviors. Access to word frequency information has long been considered an indicator of the beginning of lexical processing and the most sensitive marker for studying when the brain begins to extract semantic information Sereno & Rayner, Brain and Cognition, 42, 78-81, (2000), Trends in Cognitive Sciences, 7, 489-493, (2003). While the word frequency effect has been extensively studied in numerous eye-tracking and traditional EEG research using the RSVP paradigm, there is a lack of corresponding evidence in studies of natural reading. To find the neural correlates of the word frequency effect, we conducted a study of Chinese natural reading using EEG and eye-tracking coregistration to examine the time course of lexical processing. Our results reliably showed that the word frequency effect first appeared in the N200 time window and the bilateral occipitotemporal regions. Additionally, the word frequency effect was reflected in the N400 time window, spreading from the occipital region to the central parietal and frontal regions. Our current study provides the first neural correlates for word-frequency effect in natural Chinese reading so far, shedding new light on understanding lexical processing in natural reading and could serve as an important basis for further reading study when considering neural correlates in a realistic manner.

Combining EEG and eye-tracking in virtual reality: Obtaining fixation-onset event-related potentials and event-related spectral perturbations

Extensive research conducted in controlled laboratory settings has prompted an inquiry into how results can be generalized to real-world situations influenced by the subjects' actions. Virtual reality lends itself ideally to investigating complex situations but requires accurate classification of eye movements, especially when combining it with time-sensitive data such as EEG. We recorded eye-tracking data in virtual reality and classified it into gazes and saccades using a velocity-based classification algorithm, and we cut the continuous data into smaller segments to deal with varying noise levels, as introduced in the REMoDNav algorithm. Furthermore, we corrected for participants' translational movement in virtual reality. Various measures, including visual inspection, event durations, and the velocity and dispersion distributions before and after gaze onset, indicate that we can accurately classify the continuous, free-exploration data. Combining the classified eye-tracking with the EEG data, we generated fixation-onset event-related potentials (ERPs) and event-related spectral perturbations (ERSPs), providing further evidence for the quality of the eye-movement classification and timing of the onset of events. Finally, investigating the correlation between single trials and the average ERP and ERSP identified that fixation-onset ERSPs are less time sensitive, require fewer repetitions of the same behavior, and are potentially better suited to study EEG signatures in naturalistic settings. We modified, designed, and tested an algorithm that allows the combination of EEG and eye-tracking data recorded in virtual reality.

Neural markers of error processing relate to task performance, but not to substance-related risks and problems and externalizing problems in adolescence and emerging adulthood

Detecting errors and adapting behavior accordingly constitutes an integral aspect of cognition. Previous studies have linked neural correlates of error processing (e.g., error-related negativity (ERN) and error-related positivity (Pe)) to task performance and broader behavioral constructs, but few studies examined how these associations manifest in adolescence. In this study, we examined neural error processing markers and their behavioral associations in an adolescent/emerging adult sample (N = 143, Mage = 18.0 years, range 11-25 years), employing a stop-signal task. Linear regressions were conducted using bootstrap resampling to explore associations between ERN/Pe peak amplitudes and latencies, stop accuracy, stop-signal reaction time (SSRT), and post-error slowing, as well as self-reported substance-related risks and problems and externalizing problems. After adjusting for age and sex, smaller frontocentral Pe amplitude and later Pe latency were associated with longer SSRT, and later Pe latency was associated with lower stop accuracy. This might indicate that the Pe, which is thought to reflect conscious error processing, reflects task performance on a response inhibition task better than the ERN, which reflects subconscious error processing. After correcting for multiple testing, there were no associations between ERN/Pe parameters and substance-related or externalizing problems, and no age interactions for these associations were detected.

Common neural choice signals can emerge artefactually amid multiple distinct value signals

Previous work has identified characteristic neural signatures of value-based decision-making, including neural dynamics that closely resemble the ramping evidence accumulation process believed to underpin choice. Here we test whether these signatures of the choice process can be temporally dissociated from additional, choice-'independent' value signals. Indeed, EEG activity during value-based choice revealed distinct spatiotemporal clusters, with a stimulus-locked cluster reflecting affective reactions to choice sets and a response-locked cluster reflecting choice difficulty. Surprisingly, 'neither' of these clusters met the criteria for an evidence accumulation signal. Instead, we found that stimulus-locked activity can 'mimic' an evidence accumulation process when aligned to the response. Re-analysing four previous studies, including three perceptual decision-making studies, we show that response-locked signatures of evidence accumulation disappear when stimulus-locked and response-locked activity are modelled jointly. Collectively, our findings show that neural signatures of value can reflect choice-independent processes and look deceptively like evidence accumulation.

Neurophysiological approaches to exploring emotional responses to cosmetics: a systematic review of the literature

Introduction

This systematic review explores the use of neurophysiological measurements to study emotional responses to cosmetic products. The aim is to evaluate existing literature on these measurements in cosmetics, identify the main findings, highlight methodological challenges, and propose new guidelines for future research.

Method

A systematic search focusing on neurophysiological measures to determine emotions induced by different cosmetic products was carried out in accordance with PRISMA guidelines.

Results

A total of 33 articles identified with the EBSCO database met the inclusion criteria. In all, 10 different measurement tools were used in these articles to assess the emotional effects of cosmetic products.

Discussion

This review emphasizes the complexity of interactions between cosmetics and emotional responses. It underscores the importance of future research with a more holistic approach that couples several physiological measurements. Among them, electrophysiological brain activity shows potential for enhancing understanding of emotional responses related to cosmetic products. Frontal asymmetry, particularly in the alpha frequency band, was often use and frequently linked to positive emotional states, although conflicting evidence exists. Additionally, cardiac activity, specifically the LF/HF ratio, emerges as a promising marker for differentiating between different cosmetic products. However, methodological heterogeneity, present challenges for replicability, generalizability, and complicate data interpretation.

A Lightweight Multi-Mental Disorders Detection Method Using Entropy-Based Matrix from Single-Channel EEG Signals

Background: Mental health issues are increasingly prominent worldwide, posing significant threats to patients and deeply affecting their families and social relationships. Traditional diagnostic methods are subjective and delayed, indicating the need for an objective and effective early diagnosis method. Methods: To this end, this paper proposes a lightweight detection method for multi-mental disorders with fewer data sources, aiming to improve diagnostic procedures and enable early patient detection. First, the proposed method takes Electroencephalography (EEG) signals as sources, acquires brain rhythms through Discrete Wavelet Decomposition (DWT), and extracts their approximate entropy, fuzzy entropy, permutation entropy, and sample entropy to establish the entropy-based matrix. Then, six kinds of conventional machine learning classifiers, including Support Vector Machine (SVM), k-Nearest Neighbors (kNN), Naive Bayes (NB), Generalized Additive Model (GAM), Linear Discriminant Analysis (LDA), and Decision Tree (DT), are adopted for the entropy-based matrix to achieve the detection task. Their performances are assessed by accuracy, sensitivity, specificity, and F1-score. Concerning these experiments, three public datasets of schizophrenia, epilepsy, and depression are utilized for method validation. Results: The analysis of the results from these datasets identifies the representative single-channel signals (schizophrenia: O1, epilepsy: F3, depression: O2), satisfying classification accuracies (88.10%, 75.47%, and 89.92%, respectively) with minimal input. Conclusions: Such performances are impressive when considering fewer data sources as a concern, which also improves the interpretability of the entropy features in EEG, providing a reliable detection approach for multi-mental disorders and advancing insights into their underlying mechanisms and pathological states.

Saccade size predicts onset time of object processing during visual search of an open world virtual environment

OBJECTIVE

To date the vast majority of research in the visual neurosciences have been forced to adopt a highly constrained perspective of the vision system in which stimuli are processed in an open-loop reactive fashion (i.e., abrupt stimulus presentation followed by an evoked neural response). While such constraints enable high construct validity for neuroscientific investigation, the primary outcomes have been a reductionistic approach to isolate the component processes of visual perception. In electrophysiology, of the many neural processes studied under this rubric, the most well-known is, arguably, the P300 evoked response. There is, however, relatively little known about the real-world corollary of this component in free-viewing paradigms where visual stimuli are connected to neural function in a closed-loop. While growing evidence suggests that neural activity analogous to the P300 does occur in such paradigms, it is an open question when this response occurs and what behavioral or environmental factors could be used to isolate this component.

APPROACH

The current work uses convolutional networks to decode neural signals during a free-viewing visual search task in a closed-loop paradigm within an open-world virtual environment. From the decoded activity we construct fixation-locked response profiles that enable estimations of the variable latency of any P300 analogue around the moment of fixation. We then use these estimates to investigate which factors best reduce variable latency and, thus, predict the onset time of the response. We consider measurable, search-related factors encompassing top-down (i.e., goal driven) and bottom-up (i.e., stimulus driven) processes, such as fixation duration and salience. We also consider saccade size as an intermediate factor reflecting the integration of these two systems.

MAIN RESULTS

The results show that of these factors only saccade size reliably determines the onset time of P300 analogous activity for this task. Specifically, we find that for large saccades the variability in response onset is small enough to enable analysis using traditional ensemble averaging methods.

SIGNIFICANCE

The results show that P300 analogous activity does occur during closed-loop, free-viewing visual search while highlighting distinct differences between the open-loop version of this response and its real-world analogue. The results also further establish saccades, and saccade size, as a key factor in real-world visual processing.

Pupil dilation reflects covert familiar face recognition under interocular suppression

In prosopagnosia, brain lesions impair overt face recognition, but not face detection, and may coexist with residual covert recognition of familiar faces. Previous studies that simulated covert recognition in healthy individuals have impaired face detection as well as recognition, thus not fully mirroring the deficits in prosopagnosia. We evaluated a model of covert recognition based on continuous flash suppression (CFS). Familiar and unfamiliar faces and houses were masked while participants performed two discrimination tasks. With increased suppression, face/house discrimination remained largely intact, but face familiarity discrimination deteriorated. Covert recognition was present across all masking levels, evinced by higher pupil dilation to familiar than unfamiliar faces. Pupil dilation was uncorrelated with overt performance across subjects. Thus, CFS can impede overt face recognition without disrupting covert recognition and face detection, mirroring critical features of prosopagnosia. CFS could be used to uncover shared neural mechanisms of covert recognition in prosopagnosic patients and neurotypicals.

Reward positivity affects temporal interval production in a continuous timing task

The neural circuits of reward processing and interval timing (including the perception and production of temporal intervals) are functionally intertwined, suggesting that it might be possible for momentary reward processing to influence subsequent timing behavior. Previous animal and human studies have mainly focused on the effect of reward on interval perception, whereas its impact on interval production is less clear. In this study, we examined whether feedback, as an example of performance-contingent reward, biases interval production. We recorded EEG from 20 participants while they engaged in a continuous drumming task with different realistic tempos (1728 trials per participant). Participants received color-coded feedback after each beat about whether they were correct (on time) or incorrect (early or late). Regression-based EEG analysis was used to unmix the rapid occurrence of a feedback response called the reward positivity (RewP), which is traditionally observed in more slow-paced tasks. Using linear mixed modeling, we found that RewP amplitude predicted timing behavior for the upcoming beat. This performance-biasing effect of the RewP was interpreted as reflecting the impact of fluctuations in reward-related anterior cingulate cortex activity on timing, and the necessity of continuous paradigms to make such observations was highlighted.

A systematic review of mobile brain/body imaging studies using the P300 event-related potentials to investigate cognition beyond the laboratory

The P300 ERP component, related to the onset of task-relevant or infrequent stimuli, has been widely used in the Mobile Brain/Body Imaging (MoBI) literature. This systematic review evaluates the quality and breadth of P300 MoBI studies, revealing a maturing field with well-designed research yet grappling with standardization and global representation challenges. While affirming the reliability of measuring P300 ERP components in mobile settings, the review identifies significant hurdles in standardizing data cleaning and processing techniques, impacting comparability and reproducibility. Geographical disparities emerge, with studies predominantly in the Global North and a dearth of research from the Global South, emphasizing the need for broader inclusivity to counter the WEIRD bias in psychology. Collaborative projects and mobile EEG systems showcase the feasibility of reaching diverse populations, which is essential to advance precision psychiatry and to integrate varied data streams. Methodologically, a trend toward ecological validity is noted, shifting from lab-based to real-world settings with portable EEG system advancements. Future hardware developments are expected to balance signal quality and sensor intrusiveness, enriching data collection in everyday contexts. Innovative methodologies reflect a move toward more natural experimental settings, prompting critical questions about the applicability of traditional ERP markers, such as the P300 outside structured paradigms. The review concludes by highlighting the crucial role of integrating mobile technologies, physiological sensors, and machine learning to advance cognitive neuroscience. It advocates for an operational definition of ecological validity to bridge the gap between controlled experiments and the complexity of embodied cognitive experiences, enhancing both theoretical understanding and practical application in study design.

Beyond peaks and troughs: Multiplexed performance monitoring signals in the EEG

With the discovery of event-related potentials elicited by errors more than 30 years ago, a new avenue of research on performance monitoring, cognitive control, and decision making emerged. Since then, the field has developed and expanded fulminantly. After a brief overview on the EEG correlates of performance monitoring, this article reviews recent advancements based on single-trial analyses using independent component analysis, multiple regression, and multivariate pattern classification. Given the close interconnection between performance monitoring and reinforcement learning, computational modeling and model-based EEG analyses have made a particularly strong impact. The reviewed findings demonstrate that error- and feedback-related EEG dynamics represent variables reflecting how performance-monitoring signals are weighted and transformed into an adaptation signal that guides future decisions and actions. The model-based single-trial analysis approach goes far beyond conventional peak-and-trough analyses of event-related potentials and enables testing mechanistic theories of performance monitoring, cognitive control, and decision making.

The late positive event-related potential component is time locked to the decision in recognition memory tasks

Two event-related potential (ERP) components are commonly observed in recognition memory tasks: the Frontal Negativity (FN400) and the Late Positive Component (LPC). These components are widely interpreted as neural correlates of familiarity and recollection, respectively. However, the interpretation of LPC effects is complicated by inconsistent results regarding the timing of ERP amplitude differences. There are also mixed findings regarding how LPC amplitudes covary with decision confidence. Critically, LPC effects have almost always been measured using fixed time windows relative to memory probe stimulus onset, yet it has not been determined whether LPC effects are time locked to the stimulus or the recognition memory decision. To investigate this, we analysed a large (n = 132) existing dataset recorded during recognition memory tasks with old/new decisions followed by post-decisional confidence ratings. We used ERP deconvolution to disentangle contributions to LPC effects (defined as differences between hits and correct rejections) that were time locked to either the stimulus or the vocal old/new response. We identified a left-lateralised parietal LPC effect that was time locked to the vocal response rather than probe stimulus onset. We also isolated a response-locked, midline parietal ERP correlate of confidence that influenced measures of LPC amplitudes at left parietal electrodes. Our findings demonstrate that, contrary to widespread assumptions, the LPC effect is time locked to the recognition memory decision and is best measured using response-locked ERPs. By extension, differences in response time distributions across conditions of interest may lead to substantial measurement biases when analysing stimulus-locked ERPs. Our findings highlight important confounding factors that further complicate the interpretation of existing stimulus-locked LPC effects as neural correlates of recollection. We recommend that future studies adopt our analytic approach to better isolate LPC effects and their sensitivity to manipulations in recognition memory tasks.

Parafoveal and foveal N400 effects in natural reading: A timeline of semantic processing from fixation-related potentials

The depth at which parafoveal words are processed during reading is an ongoing topic of debate. Recent studies using RSVP-with-flanker paradigms have shown that implausible words within sentences elicit an N400 component while they are still in parafoveal vision, suggesting that the semantics of parafoveal words can be accessed to rapidly update the sentence representation. To study this effect in natural reading, we combined the coregistration of eye movements and EEG with the deconvolution modeling of fixation-related potentials (FRPs) to test whether semantic plausibility is processed parafoveally during Chinese sentence reading. For one target word per sentence, both its parafoveal and foveal plausibility were orthogonally manipulated using the boundary paradigm. Consistent with previous eye movement studies, we observed a delayed effect of parafoveal plausibility on fixation durations that only emerged on the foveal word. Crucially, in FRPs aligned to the pretarget fixation, a clear N400 effect emerged already based on parafoveal plausibility, with more negative voltages for implausible previews. Once participants fixated the target, we again observed an N400 effect of foveal plausibility. Interestingly, this foveal N400 was absent whenever the preview had been implausible, indicating that when a word's (im)plausibility is already processed in parafoveal vision, this information is not revised anymore upon direct fixation. Implausible words also elicited a late positive component (LPC), but exclusively when in foveal vision. Our results not only provide convergent neural and behavioral evidence for the parafoveal uptake of semantic information, but also indicate different contributions of parafoveal versus foveal information toward higher level sentence processing.

Language prediction in monolingual and bilingual speakers: an EEG study

Prediction of upcoming words is thought to be crucial for language comprehension. Here, we are asking whether bilingualism entails changes to the electrophysiological substrates of prediction. Prior findings leave it open whether monolingual and bilingual speakers predict upcoming words to the same extent and in the same manner. We address this issue with a naturalistic approach, employing an information-theoretic metric, surprisal, to predict and contrast the N400 brain potential in monolingual and bilingual speakers. We recruited 18 Iranian Azeri-Persian bilingual speakers and 22 Persian monolingual speakers. Subjects listened to a story in Persian while their electroencephalogram (EEG) was recorded. Bayesian item-level analysis was used. While in monolingual speakers N400 was sensitive to information-theoretic properties of both the current and previous words, in bilingual speakers N400 reflected the properties of the previous word only. Our findings show evidence for a processing delay in bilingual speakers which is consistent with prior research.

A Deep Learning Approach to Analyzing Continuous-Time Cognitive Processes

The dynamics of the mind are complex. Mental processes unfold continuously in time and may be sensitive to a myriad of interacting variables, especially in naturalistic settings. But statistical models used to analyze data from cognitive experiments often assume simplistic dynamics. Recent advances in deep learning have yielded startling improvements to simulations of dynamical cognitive processes, including speech comprehension, visual perception, and goal-directed behavior. But due to poor interpretability, deep learning is generally not used for scientific analysis. Here, we bridge this gap by showing that deep learning can be used, not just to imitate, but to analyze complex processes, providing flexible function approximation while preserving interpretability. To do so, we define and implement a nonlinear regression model in which the probability distribution over the response variable is parameterized by convolving the history of predictors over time using an artificial neural network, thereby allowing the shape and continuous temporal extent of effects to be inferred directly from time series data. Our approach relaxes standard simplifying assumptions (e.g., linearity, stationarity, and homoscedasticity) that are implausible for many cognitive processes and may critically affect the interpretation of data. We demonstrate substantial improvements on behavioral and neuroimaging data from the language processing domain, and we show that our model enables discovery of novel patterns in exploratory analyses, controls for diverse confounds in confirmatory analyses, and opens up research questions in cognitive (neuro)science that are otherwise hard to study.

Speech-induced suppression during natural dialogues

When engaged in a conversation, one receives auditory information from the other's speech but also from their own speech. However, this information is processed differently by an effect called Speech-Induced Suppression. Here, we studied brain representation of acoustic properties of speech in natural unscripted dialogues, using electroencephalography (EEG) and high-quality speech recordings from both participants. Using encoding techniques, we were able to reproduce a broad range of previous findings on listening to another's speech, and achieving even better performances when predicting EEG signal in this complex scenario. Furthermore, we found no response when listening to oneself, using different acoustic features (spectrogram, envelope, etc.) and frequency bands, evidencing a strong effect of SIS. The present work shows that this mechanism is present, and even stronger, during natural dialogues. Moreover, the methodology presented here opens the possibility of a deeper understanding of the related mechanisms in a wider range of contexts.

Contralateral delay activity and alpha lateralization reflect retinotopic and screen-centered reference frames in visual memory

The visual system represents objects in a lateralized manner, with contralateral cortical hemispheres responsible for left and right visual hemifields. This organization extends to visual short-term memory (VSTM), as evidenced by electrophysiological indices of VSTM maintenance: contralateral delay activity (CDA) and alpha-band lateralization. However, it remains unclear if VSTM represents object locations in gaze-centered (retinotopic) or screen-centered (spatiotopic) coordinates, especially after eye movements. In two experiments, participants encoded the colors of target objects and made a lateral saccade during the maintenance interval, thereby shifting the object's location on the retina. A non-lateralized probe stimulus was then presented at the new fixation for a change detection task. The CDA maintained lateralization towards the target's original retinotopic location, unaffected by subsequent saccades, and did not invert polarity even when a saccade brought that location into the opposite hemifield. We also found conventional alpha lateralization towards the target's location before a saccade. After a saccade, however, alpha was lateralized towards the screen center regardless of the target's original location, even in a control condition without any memory requirements. This suggests that post-saccadic alpha-band lateralization reflects attentional processes unrelated to memory, while pre- and post-saccade CDA reflect VSTM maintenance in a retinotopic reference frame.

lmeEEG: Mass linear mixed-effects modeling of EEG data with crossed random effects

BACKGROUND

Mixed-effects models are the current standard for the analysis of behavioral studies in psycholinguistics and related fields, given their ability to simultaneously model crossed random effects for subjects and items. However, they are hardly applied in neuroimaging and psychophysiology, where the use of mass univariate analyses in combination with permutation testing would be too computationally demanding to be practicable with mixed models.

NEW METHOD

Here, we propose and validate an analytical strategy that enables the use of linear mixed models (LMM) with crossed random intercepts in mass univariate analyses of EEG data (lmeEEG). It avoids the unfeasible computational costs that would arise from massive permutation testing with LMM using a simple solution: removing random-effects contributions from EEG data and performing mass univariate linear analysis and permutations on the obtained marginal EEG.

RESULTS

lmeEEG showed excellent performance properties in terms of power and false positive rate.

COMPARISON WITH EXISTING METHODS

lmeEEG overcomes the computational costs of standard available approaches (our method was indeed more than 300 times faster).

CONCLUSIONS

lmeEEG allows researchers to use mixed models with EEG mass univariate analyses. Thanks to the possibility offered by the method described here, we anticipate that LMM will become increasingly important in neuroscience. Data and codes are available at osf.io/kw87a. The codes and a tutorial are also available at github.com/antovis86/lmeEEG.

Transcranial focused ultrasound to human rIFG improves response inhibition through modulation of the P300 onset latency

Response inhibition in humans is important to avoid undesirable behavioral action consequences. Neuroimaging and lesion studies point to a locus of inhibitory control in the right inferior frontal gyrus (rIFG). Electrophysiology studies have implicated a downstream event-related potential from rIFG, the fronto-central P300, as a putative neural marker of the success and timing of inhibition over behavioral responses. However, it remains to be established whether rIFG effectively drives inhibition and which aspect of P300 activity uniquely indexes inhibitory control-ERP timing or amplitude. Here, we dissect the connection between rIFG and P300 for inhibition by using transcranial-focused ultrasound (tFUS) to target rIFG of human subjects while they performed a Stop-Signal task. By applying tFUS simultaneously with different task events, we found behavioral inhibition was improved, but only when applied to rIFG simultaneously with a 'stop' signal. Improved inhibition through tFUS to rIFG was indexed by faster stopping times that aligned with significantly shorter N200/P300 onset latencies. In contrast, P300 amplitude was modulated during tFUS across all groups without a paired change in behavior. Using tFUS, we provide evidence for a causal connection between anatomy, behavior, and electrophysiology underlying response inhibition.

The neural correlates of continuous feedback processing

Feedback processing is commonly studied by analyzing the brain's response to discrete rather than continuous events. Such studies have led to the hypothesis that rapid phasic midbrain dopaminergic activity tracks reward prediction errors (RPEs), the effects of which are measurable at the scalp via electroencephalography (EEG). Although studies using continuous feedback are sparse, recent animal work suggests that moment-to-moment changes in reward are tracked by slowly ramping midbrain dopaminergic activity. Some have argued that these ramping signals index state values rather than RPEs. Our goal here was to develop an EEG measure of continuous feedback processing in humans, then test whether its behavior could be accounted for by the RPE hypothesis. Participants completed a stimulus-response learning task in which a continuous reward cue gradually increased or decreased over time. A regression-based unmixing approach revealed EEG activity with a topography and time course consistent with the stimulus-preceding negativity (SPN), a scalp potential previously linked to reward anticipation and tonic dopamine release. Importantly, this reward-related activity depended on outcome expectancy: as predicted by the RPE hypothesis, activity for expected reward cues was reduced compared to unexpected reward cues. These results demonstrate the possibility of using human scalp-recorded potentials to track continuous feedback processing, and test candidate hypotheses of this activity.

Potentiation of cholinergic and corticofugal inputs to the lateral amygdala in threat learning

The amygdala, cholinergic basal forebrain, and higher-order auditory cortex (HO-AC) regulate brain-wide plasticity underlying auditory threat learning. Here, we perform multi-regional extracellular recordings and optical measurements of acetylcholine (ACh) release to characterize the development of discriminative plasticity within and between these brain regions as mice acquire and recall auditory threat memories. Spiking responses are potentiated for sounds paired with shock (CS+) in the lateral amygdala (LA) and optogenetically identified corticoamygdalar projection neurons, although not in neighboring HO-AC units. Spike- or optogenetically triggered local field potentials reveal enhanced corticofugal-but not corticopetal-functional coupling between HO-AC and LA during threat memory recall that is correlated with pupil-indexed memory strength. We also note robust sound-evoked ACh release that rapidly potentiates for the CS+ in LA but habituates across sessions in HO-AC. These findings highlight a distributed and cooperative plasticity in LA inputs as mice learn to reappraise neutral stimuli as possible threats.

Quantifying decision-making in dynamic, continuously evolving environments

During perceptual decision-making tasks, centroparietal electroencephalographic (EEG) potentials report an evidence accumulation-to-bound process that is time locked to trial onset. However, decisions in real-world environments are rarely confined to discrete trials; they instead unfold continuously, with accumulation of time-varying evidence being recency-weighted towards its immediate past. The neural mechanisms supporting recency-weighted continuous decision-making remain unclear. Here, we use a novel continuous task design to study how the centroparietal positivity (CPP) adapts to different environments that place different constraints on evidence accumulation. We show that adaptations in evidence weighting to these different environments are reflected in changes in the CPP. The CPP becomes more sensitive to fluctuations in sensory evidence when large shifts in evidence are less frequent, and the potential is primarily sensitive to fluctuations in decision-relevant (not decision-irrelevant) sensory input. A complementary triphasic component over occipito-parietal cortex encodes the sum of recently accumulated sensory evidence, and its magnitude covaries with parameters describing how different individuals integrate sensory evidence over time. A computational model based on leaky evidence accumulation suggests that these findings can be accounted for by a shift in decision threshold between different environments, which is also reflected in the magnitude of pre-decision EEG activity. Our findings reveal how adaptations in EEG responses reflect flexibility in evidence accumulation to the statistics of dynamic sensory environments.

Nonadditive integration of visual information in ensemble processing

Statistically summarizing information from a stimulus array into an ensemble representation (e.g., the mean) improves the efficiency of visual processing. However, little is known about how the brain computes the ensemble statistics. Here, we propose that ensemble processing is realized by nonadditive integration, rather than linear averaging, of individual items. We used a linear regression model approach to extract EEG responses to three levels of information: the individual items, their local interactions, and their global interaction. The local and global interactions, representing nonadditive integration of individual items, elicited rapid and independent neural responses. Critically, only the neural representation of the global interaction predicted the precision of the ensemble perception at the behavioral level. Furthermore, spreading attention over the global pattern to enhance ensemble processing directly promoted rapid neural representation of the global interaction. Taken together, these findings advocate a global, nonadditive mechanism of ensemble processing in the brain.

Post-stimulus encoding of decision confidence in EEG: toward a brain-computer interface for decision making

Objective.When making decisions, humans can evaluate how likely they are to be correct. If this subjective confidence could be reliably decoded from brain activity, it would be possible to build a brain-computer interface (BCI) that improves decision performance by automatically providing more information to the user if needed based on their confidence. But this possibility depends on whether confidence can be decoded right after stimulus presentation and before the response so that a corrective action can be taken in time. Although prior work has shown that decision confidence is represented in brain signals, it is unclear if the representation is stimulus-locked or response-locked, and whether stimulus-locked pre-response decoding is sufficiently accurate for enabling such a BCI.Approach.We investigate the neural correlates of confidence by collecting high-density electroencephalography (EEG) during a perceptual decision task with realistic stimuli. Importantly, we design our task to include a post-stimulus gap that prevents the confounding of stimulus-locked activity by response-locked activity and vice versa, and then compare with a task without this gap.Main results.We perform event-related potential and source-localization analyses. Our analyses suggest that the neural correlates of confidence are stimulus-locked, and that an absence of a post-stimulus gap could cause these correlates to incorrectly appear as response-locked. By preventing response-locked activity from confounding stimulus-locked activity, we then show that confidence can be reliably decoded from single-trial stimulus-locked pre-response EEG alone. We also identify a high-performance classification algorithm by comparing a battery of algorithms. Lastly, we design a simulated BCI framework to show that the EEG classification is accurate enough to build a BCI and that the decoded confidence could be used to improve decision making performance particularly when the task difficulty and cost of errors are high.Significance.Our results show feasibility of non-invasive EEG-based BCIs to improve human decision making.

P3-like signatures of temporal predictions: a computational EEG study

Many cognitive processes, ranging from perception to action, depend on the ability to predict the timing of forthcoming events. Yet, how the brain uses predictive models in the temporal domain is still an unsolved question. In previous work, we began to explore the neural correlates of temporal predictions by using a computational approach in which an ideal Bayesian observer learned the temporal probabilities of target onsets in a simple reaction time task. Because the task was specifically designed to disambiguate updating of predictive models and surprise, changes in temporal probabilities were explicitly cued. However, in the real world, we are usually incidentally exposed to changes in the statistics of the environment. Here, we thus aimed to further investigate the electroencephalographic (EEG) correlates of Bayesian belief updating and surprise associated with incidental learning of temporal probabilities. In line with our previous EEG study, results showed distinct P3-like modulations for updating and surprise. While surprise was indexed by an early fronto-central P3-like modulation, updating was associated with a later and more posterior P3 modulation. Moreover, updating was associated with a P2-like potential at centro-parietal electrodes, likely capturing integration processes between prior beliefs and likelihood of the observed event. These findings support previous evidence of trial-by-trial variability of P3 amplitudes as an index of dissociable inferential processes. Coupled with our previous findings, the present study strongly bolsters the view of the P3 as a key brain signature of temporal Bayesian inference. Data and scripts are shared on OSF: osf.io/sdy8j/.

Bilateral increase in MEG planar gradients prior to saccade onset

Every time we move our eyes, the retinal locations of objects change. To distinguish the changes caused by eye movements from actual external motion of the objects, the visual system is thought to anticipate the consequences of eye movements (saccades). Single neuron recordings have indeed demonstrated changes in receptive fields before saccade onset. Although some EEG studies with human participants have also demonstrated a pre-saccadic increased potential over the hemisphere that will process a stimulus after a saccade, results have been mixed. Here, we used magnetoencephalography to investigate the timing and lateralization of visually evoked planar gradients before saccade onset. We modelled the gradients from trials with both a saccade and a stimulus as the linear combination of the gradients from two conditions with either only a saccade or only a stimulus. We reasoned that any residual gradients in the condition with both a saccade and a stimulus must be uniquely linked to visually-evoked neural activity before a saccade. We observed a widespread increase in residual planar gradients. Interestingly, this increase was bilateral, showing activity both contralateral and ipsilateral to the stimulus, i.e. over the hemisphere that would process the stimulus after saccade offset. This pattern of results is consistent with predictive pre-saccadic changes involving both the current and the future receptive fields involved in processing an attended object, well before the start of the eye movement. The active, sensorimotor coupling of vision and the oculomotor system may underlie the seamless subjective experience of stable and continuous perception.

Anterior cingulate and medial prefrontal cortex oscillations underlie learning alterations in trait anxiety in humans

Anxiety has been linked to altered belief formation and uncertainty estimation, impacting learning. Identifying the neural processes underlying these changes is important for understanding brain pathology. Here, we show that oscillatory activity in the medial prefrontal, anterior cingulate and orbitofrontal cortex (mPFC, ACC, OFC) explains anxiety-related learning alterations. In a magnetoencephalography experiment, two groups of human participants pre-screened with high and low trait anxiety (HTA, LTA: 39) performed a probabilistic reward-based learning task. HTA undermined learning through an overestimation of volatility, leading to faster belief updating, more stochastic decisions and pronounced lose-shift tendencies. On a neural level, we observed increased gamma activity in the ACC, dmPFC, and OFC during encoding of precision-weighted prediction errors in HTA, accompanied by suppressed ACC alpha/beta activity. Our findings support the association between altered learning and belief updating in anxiety and changes in gamma and alpha/beta activity in the ACC, dmPFC, and OFC.

Parallel word reading revealed by fixation-related brain potentials

During reading, the brain is confronted with many relevant objects at once. But does lexical processing occur for multiple words simultaneously? Cognitive science has yet to answer this prominent question. Recently it has been argued that the issue warrants supplementing the field's traditional toolbox (response times, eye-tracking) with neuroscientific techniques (EEG, fMRI). Indeed, according to the OB1-reader model, upcoming words need not impact oculomotor behavior per se, but parallel processing of these words must nonetheless be reflected in neural activity. Here we combined eye-tracking with EEG, time-locking the neural window of interest to the fixation on target words in sentence reading. During these fixations, we manipulated the identity of the subsequent word so that it posed either a syntactically legal or illegal continuation of the sentence. In line with previous research, oculomotor measures were unaffected. Yet, syntax impacted brain potentials as early as 100 ms after the target fixation onset. Given the EEG literature on syntax processing, the presently observed timings suggest parallel word reading. We reckon that parallel word processing typifies reading, and that OB1-reader offers a good platform for theorizing about the reading brain.

Using spontaneous eye blink-related brain activity to investigate cognitive load during mobile map-assisted navigation

The continuous assessment of pedestrians' cognitive load during a naturalistic mobile map-assisted navigation task is challenging because of limited experimental control over stimulus presentation, human-map-interactions, and other participant responses. To overcome this challenge, the present study takes advantage of navigators' spontaneous eye blinks during navigation to serve as event markers in continuously recorded electroencephalography (EEG) data to assess cognitive load in a mobile map-assisted navigation task. We examined if and how displaying different numbers of landmarks (3 vs. 5 vs. 7) on mobile maps along a given route would influence navigators' cognitive load during navigation in virtual urban environments. Cognitive load was assessed by the peak amplitudes of the blink-related fronto-central N2 and parieto-occipital P3. Our results show increased parieto-occipital P3 amplitude indicating higher cognitive load in the 7-landmark condition, compared to showing 3 or 5 landmarks. Our prior research already demonstrated that participants acquire more spatial knowledge in the 5- and 7-landmark conditions compared to the 3-landmark condition. Together with the current study, we find that showing 5 landmarks, compared to 3 or 7 landmarks, improved spatial learning without overtaxing cognitive load during navigation in different urban environments. Our findings also indicate a possible cognitive load spillover effect during map-assisted wayfinding whereby cognitive load during map viewing might have affected cognitive load during goal-directed locomotion in the environment or vice versa. Our research demonstrates that users' cognitive load and spatial learning should be considered together when designing the display of future navigation aids and that navigators' eye blinks can serve as useful event makers to parse continuous human brain dynamics reflecting cognitive load in naturalistic settings.

Potentiated cholinergic and corticofugal inputs support reorganized sensory processing in the basolateral amygdala during auditory threat acquisition and retrieval

Reappraising neutral stimuli as environmental threats reflects rapid and discriminative changes in sensory processing within the basolateral amygdala (BLA). To understand how BLA inputs are also reorganized during discriminative threat learning, we performed multi-regional measurements of acetylcholine (ACh) release, single unit spiking, and functional coupling in the mouse BLA and higher-order auditory cortex (HO-AC). During threat memory recall, sounds paired with shock (CS+) elicited relatively higher firing rates in BLA units and optogenetically targeted corticoamygdalar (CAmy) units, though not in neighboring HO-AC units. Functional coupling was potentiated for descending CAmy projections prior to and during CS+ threat memory recall but ascending amygdalocortical coupling was unchanged. During threat acquisition, sound-evoked ACh release was selectively enhanced for the CS+ in BLA but not HO-AC. These findings suggest that phasic cholinergic inputs facilitate discriminative plasticity in the BLA during threat acquisition that is subsequently reinforced through potentiated auditory corticofugal inputs during memory recall.

Episodic memory formation in unrestricted viewing

The brain systems of episodic memory and oculomotor control are tightly linked, suggesting a crucial role of eye movements in memory. But little is known about the neural mechanisms of memory formation across eye movements in unrestricted viewing behavior. Here, we leverage simultaneous eye tracking and EEG recording to examine episodic memory formation in free viewing. Participants memorized multi-element events while their EEG and eye movements were concurrently recorded. Each event comprised elements from three categories (face, object, place), with two exemplars from each category, in different locations on the screen. A subsequent associative memory test assessed participants' memory for the between-category associations that specified each event. We used a deconvolution approach to overcome the problem of overlapping EEG responses to sequential saccades in free viewing. Brain activity was time-locked to the fixation onsets, and we examined EEG power in the theta and alpha frequency bands, the putative oscillatory correlates of episodic encoding mechanisms. Three modulations of fixation-related EEG predicted high subsequent memory performance: (1) theta increase at fixations after between-category gaze transitions, (2) theta and alpha increase at fixations after within-element gaze transitions, (3) alpha decrease at fixations after between-exemplar gaze transitions. Thus, event encoding with unrestricted viewing behavior was characterized by three neural mechanisms, manifested in fixation-locked theta and alpha EEG activity that rapidly turned on and off during the unfolding eye movement sequences. These three distinct neural mechanisms may be the essential building blocks that subserve the buildup of coherent episodic memories during unrestricted viewing behavior.

The ZuCo benchmark on cross-subject reading task classification with EEG and eye-tracking data

We present a new machine learning benchmark for reading task classification with the goal of advancing EEG and eye-tracking research at the intersection between computational language processing and cognitive neuroscience. The benchmark task consists of a cross-subject classification to distinguish between two reading paradigms: normal reading and task-specific reading. The data for the benchmark is based on the Zurich Cognitive Language Processing Corpus (ZuCo 2.0), which provides simultaneous eye-tracking and EEG signals from natural reading of English sentences. The training dataset is publicly available, and we present a newly recorded hidden testset. We provide multiple solid baseline methods for this task and discuss future improvements. We release our code and provide an easy-to-use interface to evaluate new approaches with an accompanying public leaderboard: www.zuco-benchmark.com.

WildLab: A naturalistic free viewing experiment reveals previously unknown electroencephalography signatures of face processing

Neural mechanisms of face perception are predominantly studied in well-controlled experimental settings that involve random stimulus sequences and fixed eye positions. Although powerful, the employed paradigms are far from what constitutes natural vision. Here, we demonstrate the feasibility of ecologically more valid experimental paradigms using natural viewing behaviour, by combining a free viewing paradigm on natural scenes, free of photographer bias, with advanced data processing techniques that correct for overlap effects and co-varying non-linear dependencies of multiple eye movement parameters. We validate this approach by replicating classic N170 effects in neural responses, triggered by fixation onsets (fixation event-related potentials [fERPs]). Importantly, besides finding a strong correlation between both experiments, our more natural stimulus paradigm yielded smaller variability between subjects than the classic setup. Moving beyond classic temporal and spatial effect locations, our experiment furthermore revealed previously unknown signatures of face processing: This includes category-specific modulation of the event-related potential (ERP)'s amplitude even before fixation onset, as well as adaptation effects across subsequent fixations depending on their history.

Temporal scaling of human scalp-recorded potentials

Much of human behavior is governed by common processes that unfold over varying timescales. Standard event-related potential analysis assumes fixed-duration responses relative to experimental events. However, recent single-unit recordings in animals have revealed neural activity scales to span different durations during behaviors demanding flexible timing. Here, we employed a general linear modeling approach using a combination of fixed-duration and variable-duration regressors to unmix fixed-time and scaled-time components in human magneto-/electroencephalography (M/EEG) data. We use this to reveal consistent temporal scaling of human scalp-recorded potentials across four independent electroencephalogram (EEG) datasets, including interval perception, production, prediction, and value-based decision making. Between-trial variation in the temporally scaled response predicts between-trial variation in subject reaction times, demonstrating the relevance of this temporally scaled signal for temporal variation in behavior. Our results provide a general approach for studying flexibly timed behavior in the human brain.

Brain Source Correlates of Speech Perception and Reading Processes in Children With and Without Reading Difficulties

Neural correlates in reading and speech processing have been addressed extensively in the literature. While reading skills and speech perception have been shown to be associated with each other, their relationship remains debatable. In this study, we investigated reading skills, speech perception, reading, and their correlates with brain source activity in auditory and visual modalities. We used high-density event-related potentials (ERPs), fixation-related potentials (FRPs), and the source reconstruction method. The analysis was conducted on 12–13-year-old schoolchildren who had different reading levels. Brain ERP source indices were computed from frequently repeated Finnish speech stimuli presented in an auditory oddball paradigm. Brain FRP source indices were also computed for words within sentences presented in a reading task. The results showed significant correlations between speech ERP sources and reading scores at the P100 (P1) time range in the left hemisphere and the N250 time range in both hemispheres, and a weaker correlation for visual word processing N170 FRP source(s) in the posterior occipital areas, in the vicinity of the visual word form areas (VWFA). Furthermore, significant brain-to-brain correlations were found between the two modalities, where the speech brain sources of the P1 and N250 responses correlated with the reading N170 response. The results suggest that speech processes are linked to reading fluency and that brain activations to speech are linked to visual brain processes of reading. These results indicate that a relationship between language and reading systems is present even after several years of exposure to print.

Neural Sources of Prediction Errors Detect Unrealistic VR Interactions

Objective Neural interfaces hold significant promise to implicitly track user experience. Their application in VR/AR simulations is especially favorable as it allows user assessment without breaking the immersive experience. In VR, designing immersion is one key challenge. Subjective questionnaires are the established metrics to assess the effectiveness of immersive VR simulations. However, administering such questionnaires requires breaking the immersive experience they are supposed to assess. Approach We present a complimentary metric based on a ERPs. For the metric to be robust, the neural signal employed must be reliable. Hence, it is beneficial to target the neural signal's cortical origin directly, efficiently separating signal from noise. To test this new complementary metric, we designed a reach-to-tap paradigm in VR to probe EEG and movement adaptation to visuo-haptic glitches. Our working hypothesis was, that these glitches, or violations of the predicted action outcome, may indicate a disrupted user experience. Main Results Using prediction error negativity features, we classified VR glitches with ~77\% accuracy. We localized the EEG sources driving the classification and found midline cingulate EEG sources and a distributed network of parieto-occipital EEG sources to enable the classification success. Significance Prediction error signatures from these sources reflect violations of user's predictions during interaction with AR/VR, promising a robust and targeted marker for adaptive user interfaces.

Naturalistic viewing conditions can increase task engagement and aesthetic preference but have only minimal impact on EEG quality

Free gaze and moving images are typically avoided in EEG experiments due to the expected generation of artifacts and noise. Yet for a growing number of research questions, loosening these rigorous restrictions would be beneficial. Among these is research on visual aesthetic experiences, which often involve open-ended exploration of highly variable stimuli. Here we systematically compare the effect of conservative vs. more liberal experimental settings on various measures of behavior, brain activity and physiology in an aesthetic rating task. Our primary aim was to assess EEG signal quality. 43 participants either maintained fixation or were allowed to gaze freely, and viewed either static images or dynamic (video) stimuli consisting of dance performances or nature scenes. A passive auditory background task (auditory steady-state response; ASSR) was added as a proxy measure for overall EEG recording quality. We recorded EEG, ECG and eyetracking data, and participants rated their aesthetic preference and state of boredom on each trial. Whereas both behavioral ratings and gaze behavior were affected by task and stimulus manipulations, EEG SNR was barely affected and generally robust across all conditions, despite only minimal preprocessing and no trial rejection. In particular, we show that using video stimuli does not necessarily result in lower EEG quality and can, on the contrary, significantly reduce eye movements while increasing both the participants' aesthetic response and general task engagement. We see these as encouraging results indicating that - at least in the lab - more liberal experimental conditions can be adopted without significant loss of signal quality.

Behavioural and neural indices of perceptual decision-making in autistic children during visual motion tasks

Many studies report atypical responses to sensory information in autistic individuals, yet it is not clear which stages of processing are affected, with little consideration given to decision-making processes. We combined diffusion modelling with high-density EEG to identify which processing stages differ between 50 autistic and 50 typically developing children aged 6-14 years during two visual motion tasks. Our pre-registered hypotheses were that autistic children would show task-dependent differences in sensory evidence accumulation, alongside a more cautious decision-making style and longer non-decision time across tasks. We tested these hypotheses using hierarchical Bayesian diffusion models with a rigorous blind modelling approach, finding no conclusive evidence for our hypotheses. Using a data-driven method, we identified a response-locked centro-parietal component previously linked to the decision-making process. The build-up in this component did not consistently relate to evidence accumulation in autistic children. This suggests that the relationship between the EEG measure and diffusion-modelling is not straightforward in autistic children. Compared to a related study of children with dyslexia, motion processing differences appear less pronounced in autistic children. Exploratory analyses also suggest weak evidence that ADHD symptoms moderate perceptual decision-making in autistic children.

Nonlinear Mixed Effect Model Used in a Simulation of the Impact of Climate Change on Height Growth of Cyclobalanopsis glauca

Localized climate is sensitive to terrain, underlying surface material, building distribution, green coverage and CO2 emissions. The Regional Climate Model (RegCM) was used to make a statistical detailed analysis of the climate change data in a specific study area to obtain fine-scale distribution of climatic elements data over time. The effects of climate change factors on height growth trends of a climate-sensitive tree species (Cyclobalanopsis glauca) were simulated based on historical climate base line data (1961–2010) and future climate change (2010–2100) predictions. Cyclobalanopsis glauca growth trends were simulated and analyzed by using a nonlinear mixed effect model (NLME). The results showed that under the RCP8.5 emissions scenario, the growth promotion effect on the height growth of Cyclobalanopsis glauca will be obvious. Under RCP4.5 and RCP2.6 emissions scenarios, although the inhibition intensity is not exactly the same, height growth will still be inhibited to a certain extent, which may lead to the gradual extinction of this species, affecting the composition of dominant tree species in the study area. The results indirectly reflect the impact of climate change on tree species diversity in the future.