Global Data-Driven Analysis of Brain Connectivity During Emotion Regulation by Electroencephalography Neurofeedback

Multimodal and hemispheric graph-theoretical brain network predictors of learning efficacy for frontal alpha asymmetry neurofeedback

EEG neurofeedback using frontal alpha asymmetry (FAA) has been widely used for emotion regulation, but its effectiveness is controversial. Studies indicated that individual differences in neurofeedback training can be traced to neuroanatomical and neurofunctional features. However, they only focused on regional brain structure or function and overlooked possible neural correlates of the brain network. Besides, no neuroimaging predictors for FAA neurofeedback protocol have been reported so far. We designed a single-blind pseudo-controlled FAA neurofeedback experiment and collected multimodal neuroimaging data from healthy participants before training. We assessed the learning performance for evoked EEG modulations during training (L1) and at rest (L2), and investigated performance-related predictors based on a combined analysis of multimodal brain networks and graph-theoretical features. The main findings of this study are described below. First, both real and sham groups could increase their FAA during training, but only the real group showed a significant increase in FAA at rest. Second, the predictors during training blocks and at rests were different: L1 was correlated with the graph-theoretical metrics (clustering coefficient and local efficiency) of the right hemispheric gray matter and functional networks, while L2 was correlated with the graph-theoretical metrics (local and global efficiency) of the whole-brain and left the hemispheric functional network. Therefore, the individual differences in FAA neurofeedback learning could be explained by individual variations in structural/functional architecture, and the correlated graph-theoretical metrics of learning performance indices showed different laterality of hemispheric networks. These results provided insight into the neural correlates of inter-individual differences in neurofeedback learning.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11571-023-09939-x.

Non-linear processing and reinforcement learning to predict rTMS treatment response in depression

BACKGROUND

Forecasting the efficacy of repetitive transcranial magnetic stimulation (rTMS) therapy can lead to substantial time and cost savings by preventing futile treatments. To achieve this objective, we've formulated a machine learning approach aimed at categorizing patients with major depressive disorder (MDD) into two groups: individuals who respond (R) positively to rTMS treatment and those who do not respond (NR).

METHODS

Preceding the commencement of treatment, we obtained resting-state EEG data from 106 patients diagnosed with MDD, employing 32 electrodes for data collection. These patients then underwent a 7-week course of rTMS therapy, and 54 of them exhibited positive responses to the treatment. Employing Independent Component Analysis (ICA) on the EEG data, we successfully pinpointed relevant brain sources that could potentially serve as markers of neural activity within the dorsolateral prefrontal cortex (DLPFC). These identified sources were further scrutinized to estimate the sources of activity within the sensor domain. Then, we integrated supplementary physiological data and implemented specific criteria to yield more realistic estimations when compared to conventional EEG analysis. In the end, we selected components corresponding to the DLPFC region within the sensor domain. Features were derived from the time-series data of these relevant independent components. To identify the most significant features, we used Reinforcement Learning (RL). In categorizing patients into two groups - R and NR to rTMS treatment - we utilized three distinct classification algorithms including K-Nearest Neighbor (KNN), Support Vector Machine (SVM), and Multilayer Perceptron (MLP). We assessed the performance of these classifiers through a ten-fold cross-validation method. Additionally, we conducted a statistical test to evaluate the discriminative capacity of these features between responders and non-responders, opening the door for further exploration in this field.

RESULTS

We identified EEG features that can anticipate the response to rTMS treatment. The most robust discriminators included EEG beta power, the sum of bispectrum diagonal elements in the delta and beta frequency bands. When these features were combined into a single vector, the classification of responders and non-responders achieved impressive performance, with an accuracy of 95.28 %, specificity at 94.23 %, sensitivity reaching 96.29 %, and precision standing at 94.54 %, all achieved using SVM.

CONCLUSIONS

The results of this study suggest that the proposed approach, utilizing power, non-linear, and bispectral features extracted from relevant independent component time-series, has the capability to forecast the treatment outcome of rTMS for MDD patients based solely on a single pre-treatment EEG recording session. The achieved findings demonstrate the superior performance of our method compared to previous techniques.

Enhancing the accuracy of electroencephalogram-based emotion recognition through Long Short-Term Memory recurrent deep neural networks

Introduction

Emotions play a critical role in human communication, exerting a significant influence on brain function and behavior. One effective method of observing and analyzing these emotions is through electroencephalography (EEG) signals. Although numerous studies have been dedicated to emotion recognition (ER) using EEG signals, achieving improved accuracy in recognition remains a challenging task. To address this challenge, this paper presents a deep-learning approach for ER using EEG signals.

Background

ER is a dynamic field of research with diverse practical applications in healthcare, human-computer interaction, and affective computing. In ER studies, EEG signals are frequently employed as they offer a non-invasive and cost-effective means of measuring brain activity. Nevertheless, accurately identifying emotions from EEG signals poses a significant challenge due to the intricate and non-linear nature of these signals.

Methods

The present study proposes a novel approach for ER that encompasses multiple stages, including feature extraction, feature selection (FS) employing clustering, and classification using Dual-LSTM. To conduct the experiments, the DEAP dataset was employed, wherein a clustering technique was applied to Hurst's view and statistical features during the FS phase. Ultimately, Dual-LSTM was employed for accurate ER.

Results

The proposed method achieved a remarkable accuracy of 97.5% in accurately classifying emotions across four categories: arousal, valence, liking/disliking, dominance, and familiarity. This high level of accuracy serves as strong evidence for the effectiveness of the deep-learning approach to emotion recognition (ER) utilizing EEG signals.

Conclusion

The deep-learning approach proposed in this paper has shown promising results in emotion recognition using EEG signals. This method can be useful in various applications, such as developing more effective therapies for individuals with mood disorders or improving human-computer interaction by allowing machines to respond more intelligently to users' emotional states. However, further research is needed to validate the proposed method on larger datasets and to investigate its applicability to real-world scenarios.

Neural modulation enhancement using connectivity-based EEG neurofeedback with simultaneous fMRI for emotion regulation

Emotion regulation plays a key role in human behavior and overall well-being. Neurofeedback is a non-invasive self-brain training technique used for emotion regulation to enhance brain function and treatment of mental disorders through behavioral changes. Previous neurofeedback research often focused on using activity from a single brain region as measured by fMRI or power from one or two EEG electrodes. In a new study, we employed connectivity-based EEG neurofeedback through recalling positive autobiographical memories and simultaneous fMRI to upregulate positive emotion. In our novel approach, the feedback was determined by the coherence of EEG electrodes rather than the power of one or two electrodes. We compared the efficiency of this connectivity-based neurofeedback to traditional activity-based neurofeedback through multiple experiments. The results showed that connectivity-based neurofeedback effectively improved BOLD signal change and connectivity in key emotion regulation regions such as the amygdala, thalamus, and insula, and increased EEG frontal asymmetry, which is a biomarker for emotion regulation and treatment of mental disorders such as PTSD, anxiety, and depression and coherence among EEG channels. The psychometric evaluations conducted both before and after the neurofeedback experiments revealed that participants demonstrated improvements in enhancing positive emotions and reducing negative emotions when utilizing connectivity-based neurofeedback, as compared to traditional activity-based and sham neurofeedback approaches. These findings suggest that connectivity-based neurofeedback may be a superior method for regulating emotions and could be a useful alternative therapy for mental disorders, providing individuals with greater control over their brain and mental functions.

An improved saliency model of visual attention dependent on image content

Many visual attention models have been presented to obtain the saliency of a scene, i.e., the visually significant parts of a scene. However, some mechanisms are still not taken into account in these models, and the models do not fit the human data accurately. These mechanisms include which visual features are informative enough to be incorporated into the model, how the conspicuity of different features and scales of an image may integrate to obtain the saliency map of the image, and how the structure of an image affects the strategy of our attention system. We integrate such mechanisms in the presented model more efficiently compared to previous models. First, besides low-level features commonly employed in state-of-the-art models, we also apply medium-level features as the combination of orientations and colors based on the visual system behavior. Second, we use a variable number of center-surround difference maps instead of the fixed number used in the other models, suggesting that human visual attention operates differently for diverse images with different structures. Third, we integrate the information of different scales and different features based on their weighted sum, defining the weights according to each component's contribution, and presenting both the local and global saliency of the image. To test the model's performance in fitting human data, we compared it to other models using the CAT2000 dataset and the Area Under Curve (AUC) metric. Our results show that the model has high performance compared to the other models (AUC = 0.79 and sAUC = 0.58) and suggest that the proposed mechanisms can be applied to the existing models to improve them.

Probing fMRI brain connectivity and activity changes during emotion regulation by EEG neurofeedback

Despite the existence of several emotion regulation studies using neurofeedback, interactions among a small number of regions were evaluated, and therefore, further investigation is needed to understand the interactions of the brain regions involved in emotion regulation. We implemented electroencephalography (EEG) neurofeedback with simultaneous functional magnetic resonance imaging (fMRI) using a modified happiness-inducing task through autobiographical memories to upregulate positive emotion. Then, an explorative analysis of whole brain regions was done to understand the effect of neurofeedback on brain activity and the interaction of whole brain regions involved in emotion regulation. The participants in the control and experimental groups were asked to do emotion regulation while viewing positive images of autobiographical memories and getting sham or real (based on alpha asymmetry) EEG neurofeedback, respectively. The proposed multimodal approach quantified the effects of EEG neurofeedback in changing EEG alpha power, fMRI blood oxygenation level-dependent (BOLD) activity of prefrontal, occipital, parietal, and limbic regions (up to 1.9% increase), and functional connectivity in/between prefrontal, parietal, limbic system, and insula in the experimental group. New connectivity links were identified by comparing the brain functional connectivity between experimental conditions (Upregulation and View blocks) and also by comparing the brain connectivity of the experimental and control groups. Psychometric assessments confirmed significant changes in positive and negative mood states in the experimental group by neurofeedback. Based on the exploratory analysis of activity and connectivity among all brain regions involved in emotion regions, we found significant BOLD and functional connectivity increases due to EEG neurofeedback in the experimental group, but no learning effect was observed in the control group. The results reveal several new connections among brain regions as a result of EEG neurofeedback which can be justified according to emotion regulation models and the role of those regions in emotion regulation and recalling positive autobiographical memories.

Dynamic functional connectivity estimation for neurofeedback emotion regulation paradigm with simultaneous EEG-fMRI analysis

Joint Analysis of EEG and fMRI datasets can bring new insight into brain mechanisms. In this paper, we employed the recently introduced Correlated Coupled Tensor Matrix Factorization (CCMTF) method for analysis of the emotion regulation paradigm based on EEG frontal asymmetry neurofeedback in the alpha frequency band with simultaneous fMRI. CCMTF method assumes that the co-variations of the common dimension (temporal dimension) between EEG and fMRI are correlated and not necessarily identical. The results of the CCMTF method suggested that EEG and fMRI had similar covariations during the transition of brain activities from resting states to task (view and upregulation) states and these covariations followed an increasing trend. The fMRI shared spatial component showed activations in the limbic system, DLPFC, OFC, and VLPC regions, which were consistent with the previous studies and were linked to EEG frequency patterns in the range of 1–15 Hz with a correlation value close to 0.75. The estimated regions from the CCMTF method were then used as the candidate nodes for dynamic functional connectivity (dFC) analysis, in which the changes in connectivity from view to upregulation states were examined. The results of the dFC analysis were compared with a Normalized Mutual information (NMI) based approach in two different frequency ranges (1–15 and 15–40 Hz) as the NMI method was applied to the vectors of dFC nodes of EEG and fMRI data. The results of the two methods illustrated that the relation between EEG and fMRI datasets was mostly in the frequency range of 1–15 Hz. These relations were both in the brain activations and the dFCs between the two modalities. This paper suggests that the CCMTF method is a capable approach for extracting the shared information between EEG and fMRI data and can reveal new information about brain functions and their connectivity without solving the EEG inverse problem or analyzing different frequency bands.

Evaluating interhemispheric connectivity during midline object recognition using EEG

Functional integration between two hemispheres is crucial for perceptual binding to occur when visual stimuli are presented in the midline of the visual field. Mima and colleagues (2001) showed using EEG that midline object recognition was associated with task-related decrease in alpha band power (alpha desynchronisation) and a transient increase in interhemispheric coherence. Our objective in the current study was to replicate the results of Mima et al. and to further evaluate interhemispheric effective connectivity during midline object recognition in source space. We recruited 11 healthy adult volunteers and recorded EEG from 64 channels while they performed a midline object recognition task. Task-related power and coherence were estimated in sensor and source spaces. Further, effective connectivity was evaluated using Granger causality. While we were able to replicate the alpha desynchronisation associated with midline object recognition, we could not replicate the coherence results of Mima et al. The data-driven approach that we employed in our study localised the source of alpha desynchronisation over the left occipito-temporal region. In the alpha band, we further observed significant increase in imaginary part of coherency between bilateral occipito-temporal regions during object recognition. Finally, Granger causality analysis between the left and right occipito-temporal regions provided an insight that even though there is bidirectional interaction, the left occipito-temporal region may be crucial for integrating the information necessary for object recognition. The significance of the current study lies in using high-density EEG and applying more appropriate and robust measures of connectivity as well as statistical analysis to validate and enhance our current knowledge on the neural basis of midline object recognition.

Frontal alpha asymmetry interaction with an experimental story EEG brain-computer interface

Although interest in brain-computer interfaces (BCIs) from researchers and consumers continues to increase, many BCIs lack the complexity and imaginative properties thought to guide users toward successful brain activity modulation. We investigate the possibility of using a complex BCI by developing an experimental story environment with which users interact through cognitive thought strategies. In our system, the user's frontal alpha asymmetry (FAA) measured with electroencephalography (EEG) is linearly mapped to the color saturation of the main character in the story. We implemented a user-friendly experimental design using a comfortable EEG device and short neurofeedback (NF) training protocol. In our system, seven out of 19 participants successfully increased FAA during the course of the study, for a total of ten successful blocks out of 152. We detail our results concerning left and right prefrontal cortical activity contributions to FAA in both successful and unsuccessful story blocks. Additionally, we examine inter-subject correlations of EEG data, and self-reported questionnaire data to understand the user experience of BCI interaction. Results suggest the potential of imaginative story BCI environments for engaging users and allowing for FAA modulation. Our data suggests new research directions for BCIs investigating emotion and motivation through FAA.

Source localization and functional network analysis in emotion cognitive reappraisal with EEG-fMRI integration

Background

The neural activity and functional networks of emotion-based cognitive reappraisal have been widely investigated using electroencephalography (EEG) and functional magnetic resonance imaging (fMRI). However, single-mode neuroimaging techniques are limited in exploring the regulation process with high temporal and spatial resolution.

Objectives

We proposed a source localization method with multimodal integration of EEG and fMRI and tested it in the source-level functional network analysis of emotion cognitive reappraisal.

Methods

EEG and fMRI data were simultaneously recorded when 15 subjects were performing the emotional cognitive reappraisal task. Fused priori weighted minimum norm estimation (FWMNE) with sliding windows was proposed to trace the dynamics of EEG source activities, and the phase lag index (PLI) was used to construct the functional brain network associated with the process of downregulating negative affect using the reappraisal strategy.

Results

The functional networks were constructed with the measure of PLI, in which the important regions were indicated. In the gamma band source-level network analysis, the cuneus, the lateral orbitofrontal cortex, the superior parietal cortex, the postcentral gyrus, and the pars opercularis were identified as important regions in reappraisal with high betweenness centrality.

Conclusion

The proposed multimodal integration method for source localization identified the key cortices involved in emotion regulation, and the network analysis demonstrated the important brain regions involved in the cognitive control of reappraisal. It shows promise in the utility in the clinical setting for affective disorders.

Use of autobiographical stimuli as a mood manipulation procedure: Systematic mapping review

Background

In recent years, mood induction procedures have been developed in experimental settings that are designed to facilitate studying the impact of mood states on biological and psychological processes. The aim of the present study was to conduct a systematic mapping review with the intention of describing the state of the art in the use of different types of autobiographical stimuli for mood induction procedures.

Methods

Based on a search for publications from the period 2000–2021, conducted in four recognised databases (Scopus, Medline (PubMed), PsycINFO and Web of Science), we analysed a total of 126 published articles. Text mining techniques were used to extract the main themes related.

Results

The induction of emotions through autobiographical memories is an area under construction and of growing interest. The data mining approach yielded information about the main types of stimuli used in these procedures, highlighting those that only employ a single type of cue, as well as the preference for verbal cues over others such as musical, olfactory and visual cues. This type of procedure has been used to induce both positive and negative emotions through tasks that require access to personal memories of specific events from a cue, requiring the person to set in motion different cognitive processes. The use of the latest technologies (fMRI, EEG, etc.) is also shown, demonstrating that this is a cutting-edge field of study.

Conclusions

Despite the study of mood induction procedures still being a growing field, the present review provides a novel overview of the current state of the art in the field, which may serve as a framework for future studies on the topic.

Emotion Processing Dysfunction in Alzheimer's Disease: An Overview of Behavioral Findings, Systems Neural Correlates, and Underlying Neural Biology

We described behavioral studies to highlight emotional processing deficits in Alzheimer's disease (AD). The findings suggest prominent deficit in recognizing negative emotions, pronounced effect of positive emotion on enhancing memory, and a critical role of cognitive deficits in manifesting emotional processing dysfunction in AD. We reviewed imaging studies to highlight morphometric and functional markers of hippocampal circuit dysfunction in emotional processing deficits. Despite amygdala reactivity to emotional stimuli, hippocampal dysfunction conduces to deficits in emotional memory. Finally, the reviewed studies implicating major neurotransmitter systems in anxiety and depression in AD supported altered cholinergic and noradrenergic signaling in AD emotional disorders. Overall, the studies showed altered emotions early in the course of illness and suggest the need of multimodal imaging for further investigations. Particularly, longitudinal studies with multiple behavioral paradigms translatable between preclinical and clinical models would provide data to elucidate the time course and underlying neurobiology of emotion processing dysfunction in AD.