Challenge for Affective Brain-Computer Interfaces: Non-stationary Spatio-spectral EEG Oscillations of Emotional Responses

Tackling the possibility of extracting a brain digital fingerprint based on personal hobbies predilection

In an attempt to create a more familiar brain-machine interaction for biometric authentication applications, we investigated the efficiency of using the users' personal hobbies, interests, and memory collections. This approach creates a unique and pleasant experience that can be later utilized within an authentication protocol. This paper presents a new EEG dataset recorded while subjects watch images of popular hobbies, pictures with no point of interest and images with great personal significance. In addition, we propose several applications that can be tackled with our newly collected dataset. Namely, our study showcases 4 types of applications and we obtain state-of-the-art level results for all of them. The tackled tasks are: emotion classification, category classification, authorization process, and person identification. Our experiments show great potential for using EEG response to hobby visualization for people authentication. In our study, we show preliminary results for using predilection for personal hobbies, as measured by EEG, for identifying people. Also, we propose a novel authorization process paradigm using electroencephalograms. Code and dataset are available here.

Crucial rhythms and subnetworks for emotion processing extracted by an interpretable deep learning framework from EEG networks

Electroencephalogram (EEG) brain networks describe the driving and synchronous relationships among multiple brain regions and can be used to identify different emotional states. However, methods for extracting interpretable structural features from brain networks are still lacking. In the current study, a novel deep learning structure comprising both an attention mechanism and a domain adversarial strategy is proposed to extract discriminant and interpretable features from brain networks. Specifically, the attention mechanism enhances the contribution of crucial rhythms and subnetworks for emotion recognition, whereas the domain-adversarial module improves the generalization performance of our proposed model for cross-subject tasks. We validated the effectiveness of the proposed method for subject-independent emotion recognition tasks with the SJTU Emotion EEG Dataset (SEED) and the EEGs recorded in our laboratory. The experimental results showed that the proposed method can effectively improve the classification accuracy of different emotions compared with commonly used methods such as domain adversarial neural networks. On the basis of the extracted network features, we also revealed crucial rhythms and subnetwork structures for emotion processing, which are consistent with those found in previous studies. Our proposed method not only improves the classification performance of brain networks but also provides a novel tool for revealing emotion processing mechanisms.

Bridging Minds and Machines: The Recent Advances of Brain-Computer Interfaces in Neurological and Neurosurgical Applications

Brain-computer interfaces (BCIs), a remarkable technological advancement in neurology and neurosurgery, mark a significant leap since the inception of electroencephalography in 1924. These interfaces effectively convert central nervous system signals into commands for external devices, offering revolutionary benefits to patients with severe communication and motor impairments due to a myriad of neurological conditions like stroke, spinal cord injuries, and neurodegenerative disorders. BCIs enable these individuals to communicate and interact with their environment, using their brain signals to operate interfaces for communication and environmental control. This technology is especially crucial for those completely locked in, providing a communication lifeline where other methods fall short. The advantages of BCIs are profound, offering autonomy and an improved quality of life for patients with severe disabilities. They allow for direct interaction with various devices and prostheses, bypassing damaged or nonfunctional neural pathways. However, challenges persist, including the complexity of accurately interpreting brain signals, the need for individual calibration, and ensuring reliable, long-term use. Additionally, ethical considerations arise regarding autonomy, consent, and the potential for dependence on technology. Despite these challenges, BCIs represent a transformative development in neurotechnology, promising enhanced patient outcomes and a deeper understanding of brain-machine interfaces.

Moving toward reality: Electrocortical reactivity to naturalistic multimodal emotional videos

While previous research has investigated the effects of emotional videos on peripheral physiological measures and conscious experience, this study extends the research to include electrocortical measures, specifically the steady-state visual-evoked potential (ssVEP). A carefully curated set of 45 videos, designed to represent a wide range of emotional and neutral content, were presented with a flickering border. The videos featured a continuous single-shot perspective, natural soundtrack, and excluded elements associated with professional films, to enhance realism. The results demonstrate a consistent reduction in ssVEP amplitude during emotional videos which strongly correlates with the rated emotional intensity of the clips. This suggests that narrative audiovisual stimuli have the potential to track dynamic emotional processing in the cortex, providing new avenues for research in affective neuroscience. The findings highlight the potential of using realistic video stimuli to investigate how the human brain processes emotional events in a paradigm that increases ecological validity. Future studies can further develop this paradigm by expanding the video set, targeting specific cortical networks, and manipulating narrative predictability. Overall, this study establishes a foundation for investigating emotional perception using realistic video stimuli and has the potential to expand our understanding of real-world emotional processing in the human brain.

Brain connectivity dynamics during listening to music and potential impact on task performance

To analyze brain connectivity dynamics during listening to music and estimate the potential impact on task performance. Fifteen participants (13 males and 2 females) participated in this study based on their interest in Indian classical music. Measurements of the influence of Indian music on task performance were obtained by assessing brain activation using EEG signals. Brain connectivity analysis was performed to visualize the connections between brain regions under various experimental conditions. Visual Go/No Go Stimuli was used to evaluate visual spatial attention during operation by evaluating misses, committed errors, and reaction times. In Task 1 (listening to music only), it was reported that there was a change in the positions of the electrodes (F3, F7) located in the left frontal lobe. The energy of the relative beta component was significantly higher only at F7 during task 1 (p = 0.005). Event-related desynchronization alpha and theta synchronization were significant (p = 0.005) at all electrode sites in the bilateral frontal lobes (F3, F4, F7 and F8) while listening to music and performing tasks (task 2). When the task without music (task 3) was performed, the energy of the relative alpha component was significantly higher at the Fp2 electrode position (p = 0.005). It is noteworthy that the energy of the theta component was significantly lower at the location of the Fp2 electrode (p = 0.005). The frontal asymmetry index score measures were significantly high at F4/F3 and F8/F7 during task 1. The connectivity map of theta synchronization showed a robust association between Fp2 and F8 which was in turn connected to P4 and O2 during Task 2. Results indicated an increased omission and commission errors during Task 3.

Enhancing EEG-based cross-day mental workload classification using periodic component of power spectrum

Objective. The day-to-day variability of electroencephalogram (EEG) poses a significant challenge to decode human brain activity in EEG-based passive brain-computer interfaces (pBCIs). Conventionally, a time-consuming calibration process is required to collect data from users on a new day to ensure the performance of the machine learning-based decoding model, which hinders the application of pBCIs to monitor mental workload (MWL) states in real-world settings.Approach. This study investigated the day-to-day stability of the raw power spectral density (PSD) and their periodic and aperiodic components decomposed by the Fitting Oscillations and One-Over-F algorithm. In addition, we validated the feasibility of using periodic components to improve cross-day MWL classification performance.Main results. Compared to the raw PSD (69.9% ± 18.5%) and the aperiodic component (69.4% ± 19.2%), the periodic component had better day-to-day stability and significantly higher cross-day classification accuracy (84.2% ± 11.0%).Significance. These findings indicate that periodic components of EEG have the potential to be applied in decoding brain states for more robust pBCIs.

Exploring Spatio-Spectral Electroencephalogram Modulations of Imbuing Emotional Intent During Active Piano Playing

Imbuing emotional intent serves as a crucial modulator of music improvisation during active musical instrument playing. However, most improvisation-related neural endeavors have been gained without considering the emotional context. This study attempts to exploit reproducible spatio-spectral electroencephalogram (EEG) oscillations of emotional intent using a data-driven independent component analysis framework in an ecological multiday piano playing experiment. Through the four-day 32-ch EEG dataset of 10 professional players, we showed that EEG patterns were substantially affected by both intra- and inter-individual variability underlying the emotional intent of the dichotomized valence (positive vs. negative) and arousal (high vs. low) categories. Less than half (3-4) of the 10 participants analogously exhibited day-reproducible ( ≥ three days) spectral modulations at the right frontal beta in response to the valence contrast as well as the frontal central gamma and the superior parietal alpha to the arousal counterpart. In particular, the frontal engagement facilitates a better understanding of the frontal cortex (e.g., dorsolateral prefrontal cortex and anterior cingulate cortex) and its role in intervening emotional processes and expressing spectral signatures that are relatively resistant to natural EEG variability. Such ecologically vivid EEG findings may lead to better understanding of the development of a brain-computer music interface infrastructure capable of guiding the training, performance, and appreciation for emotional improvisatory status or actuating music interaction via emotional context.

Electroencephalogram-Driven Machine-Learning Scenario for Assessing Impulse Control Disorder Comorbidity in Parkinson's Disease Using a Low-Cost, Custom LEGO-Like Headset

Patients with Parkinson's disease (PD) may develop cognitive symptoms of impulse control disorders (ICDs) when chronically treated with dopamine agonist (DA) therapy for motor deficits. Motor and cognitive comorbidities critically increase the disability and mortality of the affected patients. This study proposes an electroencephalogram (EEG)-driven machine-learning scenario to automatically assess ICD comorbidity in PD. We employed a classic Go/NoGo task to appraise the capacity of cognitive and motoric inhibition with a low-cost, custom LEGO-like headset to record task-relevant EEG activity. Further, we optimized a support vector machine (SVM) and support vector regression (SVR) pipeline to learn discriminative EEG spectral signatures for the detection of ICD comorbidity and the estimation of ICD severity, respectively. With a dataset of 21 subjects with typical PD, 9 subjects with PD and ICD comorbidity (ICD), and 25 healthy controls (HC), the study results showed that the SVM pipeline differentiated subjects with ICD from subjects with PD with an accuracy of 66.3% and returned an around-chance accuracy of 53.3% for the classification of PD versus HC subjects without the comorbidity concern. Furthermore, the SVR pipeline yielded significantly higher severity scores for the ICD group than for the PD group and resembled the ICD vs. PD distinction according to the clinical questionnaire scores, which was barely replicated by random guessing. Without a commercial, high-precision EEG product, our demonstration may facilitate deploying a wearable computer-aided diagnosis system to assess the risk of DA-triggered cognitive comorbidity in patients with PD in their daily environment.

Impact of negative links on the structural balance of brain functional network during emotion processing

Activation of specific brain areas and synchrony between them has a major role in process of emotions. Nevertheless, impact of anti-synchrony (negative links) in this process still requires to be understood. In this study, we hypothesized that quantity and topology of negative links could influence a network stability by changing of quality of its triadic associations. Therefore, a group of healthy participants were exposed to pleasant and unpleasant images while their brain responses were recorded. Subsequently, functional connectivity networks were estimated and quantity of negative links, balanced and imbalanced triads, tendency to make negative hubs, and balance energy levels of two conditions were compared. The findings indicated that perception of pleasant stimuli was associated with higher amount of negative links with a lower tendency to make a hub in theta band; while the opposite scenario was observed in beta band. It was accompanied with smaller number of imbalanced triads and more stable network in theta band, and smaller number of balanced triads and less stable network in beta band. The findings highlighted that inter regional communications require less changes to receive new information from unpleasant stimuli, although by decrement in beta band stability prepares the network for the upcoming events.

Validation of Model-Basis Transfer Learning for a Personalized Electroencephalogram-Based Emotion-Classification Model. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2023;2023:1-4. [PMID: 38082699 DOI: 10.1109/embc40787.2023.10340188] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

The electroencephalogram (EEG)-based affective brain-computer interface (aBCI) has attracted extensive attention in multidisciplinary fields in the past decade. However, the inherent variability of emotional responses recorded in EEG signals increases the vulnerability of pre-trained machine-learning models and impedes the applicability of aBCIs with real-life settings. To overcome the shortcomings associated with the limited personal data in affective modeling, this study proposes a model-basis transfer learning (TL) approach and verifies its feasibility to construct a personalized model using less emotion-annotated data in a longitudinal eight-day dataset comprising data on 10 subjects. By performing daily reliability testing, the proposed TL approach outperformed the subject-dependent counterpart (using limited data only) by ~6% in binary valence classification after recycling a compact set of the eight most transferable models from other subjects. These empirical findings practically contribute to progress in applying TL in realistic aBCI applications.Clinical Relevance- The proposed model-basis TL approach overcomes the shortcoming of inherent variability in EEG signals, supporting realistic aBCI applications.

Comparison of event-related modulation index and traditional methods for evaluating phase-amplitude coupling using simulated brain signals

The investigation of brain oscillations and connectivity has become an important topic in the recent decade. There are several types of interactions between neuronal oscillations, and one of the most interesting among these interactions is phase-amplitude coupling (PAC). Several methods have been proposed to measure the strength of PAC, including the phase-locking value, circular-linear correlation, and modulation index. In the current study, we compared these traditional PAC methods with simulated electroencephalogram signals. Further, to assess the PAC value at each time point, we also compared two recently established methods, event-related phase-locking value and event-related circular-linear correlation, with our newly proposed event-related modulation index (ERMI). Results indicated that the ERMI has better temporal resolution and is more tolerant to noise than the other two event-related methods, suggesting the advantages of utilizing ERMI in evaluating the strength of PAC within a brain region.

Multi-source joint domain adaptation for cross-subject and cross-session emotion recognition from electroencephalography

As an important component to promote the development of affective brain–computer interfaces, the study of emotion recognition based on electroencephalography (EEG) has encountered a difficult challenge; the distribution of EEG data changes among different subjects and at different time periods. Domain adaptation methods can effectively alleviate the generalization problem of EEG emotion recognition models. However, most of them treat multiple source domains, with significantly different distributions, as one single source domain, and only adapt the cross-domain marginal distribution while ignoring the joint distribution difference between the domains. To gain the advantages of multiple source distributions, and better match the distributions of the source and target domains, this paper proposes a novel multi-source joint domain adaptation (MSJDA) network. We first map all domains to a shared feature space and then align the joint distributions of the further extracted private representations and the corresponding classification predictions for each pair of source and target domains. Extensive cross-subject and cross-session experiments on the benchmark dataset, SEED, demonstrate the effectiveness of the proposed model, where more significant classification results are obtained on the more difficult cross-subject emotion recognition task.

A Review on Signal Processing Approaches to Reduce Calibration Time in EEG-Based Brain-Computer Interface

In an electroencephalogram- (EEG-) based brain–computer interface (BCI), a subject can directly communicate with an electronic device using his EEG signals in a safe and convenient way. However, the sensitivity to noise/artifact and the non-stationarity of EEG signals result in high inter-subject/session variability. Therefore, each subject usually spends long and tedious calibration time in building a subject-specific classifier. To solve this problem, we review existing signal processing approaches, including transfer learning (TL), semi-supervised learning (SSL), and a combination of TL and SSL. Cross-subject TL can transfer amounts of labeled samples from different source subjects for the target subject. Moreover, Cross-session/task/device TL can reduce the calibration time of the subject for the target session, task, or device by importing the labeled samples from the source sessions, tasks, or devices. SSL simultaneously utilizes the labeled and unlabeled samples from the target subject. The combination of TL and SSL can take advantage of each other. For each kind of signal processing approaches, we introduce their concepts and representative methods. The experimental results show that TL, SSL, and their combination can obtain good classification performance by effectively utilizing the samples available. In the end, we draw a conclusion and point to research directions in the future.

Brain-Computer Interface: Advancement and Challenges

Brain-Computer Interface (BCI) is an advanced and multidisciplinary active research domain based on neuroscience, signal processing, biomedical sensors, hardware, etc. Since the last decades, several groundbreaking research has been conducted in this domain. Still, no comprehensive review that covers the BCI domain completely has been conducted yet. Hence, a comprehensive overview of the BCI domain is presented in this study. This study covers several applications of BCI and upholds the significance of this domain. Then, each element of BCI systems, including techniques, datasets, feature extraction methods, evaluation measurement matrices, existing BCI algorithms, and classifiers, are explained concisely. In addition, a brief overview of the technologies or hardware, mostly sensors used in BCI, is appended. Finally, the paper investigates several unsolved challenges of the BCI and explains them with possible solutions.

Objective assessment of impulse control disorder in patients with Parkinson's disease using a low-cost LEGO-like EEG headset: a feasibility study

Background

Patients with Parkinson’s disease (PD) can develop impulse control disorders (ICDs) while undergoing a pharmacological treatment for motor control dysfunctions with a dopamine agonist (DA). Conventional clinical interviews or questionnaires can be biased and may not accurately diagnose at the early stage. A wearable electroencephalogram (EEG)-sensing headset paired with an examination procedure can be a potential user-friendly method to explore ICD-related signatures that can detect its early signs and progression by reflecting brain activity.

Methods

A stereotypical Go/NoGo test that targets impulse inhibition was performed on 59 individuals, including healthy controls, patients with PD, and patients with PD diagnosed by ICDs. We conducted two Go/NoGo sessions before and after the DA-pharmacological treatment for the PD and ICD groups. A low-cost LEGO-like EEG headset was used to record concurrent EEG signals. Then, we used the event-related potential (ERP) analytical framework to explore ICD-related EEG abnormalities after DA treatment.

Results

After the DA treatment, only the ICD-diagnosed PD patients made more behavioral errors and tended to exhibit the deterioration for the NoGo N2 and P3 peak amplitudes at fronto-central electrodes in contrast to the HC and PD groups. Particularly, the extent of the diminished NoGo-N2 amplitude was prone to be modulated by the ICD scores at Fz with marginal statistical significance (r = − 0.34, p = 0.07).

Conclusions

The low-cost LEGO-like EEG headset successfully captured ERP waveforms and objectively assessed ICD in patients with PD undergoing DA treatment. This objective neuro-evidence could provide complementary information to conventional clinical scales used to diagnose ICD adverse effects.

Recurrent neural network-based acute concussion classifier using raw resting state EEG data

Concussion is a global health concern. Despite its high prevalence, a sound understanding of the mechanisms underlying this type of diffuse brain injury remains elusive. It is, however, well established that concussions cause significant functional deficits; that children and youths are disproportionately affected and have longer recovery time than adults; and that individuals suffering from a concussion are more prone to experience additional concussions, with each successive injury increasing the risk of long term neurological and mental health complications. Currently, the most significant challenge in concussion management is the lack of objective, clinically- accepted, brain-based approaches for determining whether an athlete has suffered a concussion. Here, we report on our efforts to address this challenge. Specifically, we introduce a deep learning long short-term memory (LSTM)-based recurrent neural network that is able to distinguish between non-concussed and acute post-concussed adolescent athletes using only short (i.e. 90 s long) samples of resting state EEG data as input. The athletes were neither required to perform a specific task nor expected to respond to a stimulus during data collection. The acquired EEG data were neither filtered, cleaned of artefacts, nor subjected to explicit feature extraction. The LSTM network was trained and validated using data from 27 male, adolescent athletes with sports related concussion, benchmarked against 35 non-concussed adolescent athletes. During rigorous testing, the classifier consistently identified concussions with an accuracy of > 90% and achieved an ensemble median Area Under the Receiver Operating Characteristic Curve (ROC/AUC) equal to 0.971. This is the first instance of a high-performing classifier that relies only on easy-to-acquire resting state, raw EEG data. Our concussion classifier represents a promising first step towards the development of an easy-to-use, objective, brain-based, automatic classification of concussion at an individual level.

EEGs Vary Less Between Lab and Home Locations Than They Do Between People

Given the rapid development of light weight EEG devices which we have witnessed the past decade, it is reasonable to ask to which extent neuroscience could now be taken outside the lab. In this study, we have designed an EEG paradigm well suited for deployment “in the wild.” The paradigm is tested in repeated recordings on 20 subjects, on eight different occasions (4 in the laboratory, 4 in the subject's own home). By calculating the inter subject, intra subject and inter location variance, we find that the inter location variation for this paradigm is considerably less than the inter subject variation. We believe the paradigm is representative of a large group of other relevant paradigms. This means that given the positive results in this study, we find that if a research paradigm would benefit from being performed in less controlled environments, we expect limited problems in doing so.

Independent Components of EEG Activity Correlating with Emotional State

Among brain-computer interface studies, electroencephalography (EEG)-based emotion recognition is receiving attention and some studies have performed regression analyses to recognize small-scale emotional changes; however, effective brain regions in emotion regression analyses have not been identified yet. Accordingly, this study sought to identify neural activities correlating with emotional states in the source space. We employed independent component analysis, followed by a source localization method, to obtain distinct neural activities from EEG signals. After the identification of seven independent component (IC) clusters in a k-means clustering analysis, group-level regression analyses using frequency band power of the ICs were performed based on Russell's valence-arousal model. As a result, in the regression of the valence level, an IC cluster located in the cuneus predicted both high- and low-valence states and two other IC clusters located in the left precentral gyrus and the precuneus predicted the low-valence state. In the regression of the arousal level, the IC cluster located in the cuneus predicted both high- and low-arousal states and two posterior IC clusters located in the cingulate gyrus and the precuneus predicted the high-arousal state. In this proof-of-concept study, we revealed neural activities correlating with specific emotional states across participants, despite individual differences in emotional processing.