Optimal spatial filtering of single trial EEG during imagined hand movement

Enhancing robustness of spatial filters in motor imagery based brain-computer interface via temporal learning

BACKGROUND

In motor imagery-based brain-computer interface (MI-BCI) EEG decoding, spatial filtering play a crucial role in feature extraction. Recent studies have emphasized the importance of temporal filtering for extracting discriminative features in MI tasks. While many efforts have been made to optimize feature extraction externally, stabilizing features from spatial filtering remains underexplored.

NEW METHOD

To address this problem, we propose an approach to improve the robustness of temporal features by minimizing instability in the temporal domain. Specifically, we utilize Jensen-Shannon divergence to quantify temporal instability and integrate decision variables to construct an objective function that minimizes this instability. Our method enhances the stability of variance and mean values in the extracted features, improving the identification of discriminative features and reducing the effects of instability.

RESULTS

The proposed method was applied to spatial filtering models, and tested on two publicly datasets as well as a self-collected dataset. Results demonstrate that the proposed method significantly boosts classification accuracy, confirming its effectiveness in enhancing temporal feature stability.

COMPARISON WITH EXISTING METHODS

We compared our method with spatial filtering methods, and the-state-of-the-art models. The proposed approach achieves the highest accuracy, with 92.43 % on BCI competition III IVa dataset, 84.45 % on BCI competition IV 2a dataset, and 73.18 % on self-collected dataset.

CONCLUSIONS

Enhancing the instability of temporal features contributes to improved MI-BCI performance. This not only improves classification performance but also provides a stable foundation for future advancements. The proposed method shows great potential for EEG decoding.

Exoskeleton-guided passive movement elicits standardized EEG patterns for generalizable BCIs in stroke rehabilitation

BACKGROUND

Brain-computer interfaces (BCIs) hold significant potential for post-stroke motor recovery, yet active movement-based BCIs face limitations in generalization due to inter-subject variability. This study investigates passive movement-based BCIs, driven by exoskeleton-guided rehabilitation, to address these challenges by evaluating electroencephalogram (EEG) responses and algorithmic generalization in both healthy subjects and stroke patients.

METHODS

EEG signals were recorded from 20 healthy subjects and 10 stroke patients during voluntary and passive hand movements. Time and time-frequency domain analyses were performed to examine the event-related potential (ERP), event-related desynchronization (ERD), and synchronization (ERS) patterns. The performance of two BCI algorithms, Common Spatial Patterns (CSP) and EEGNet, was evaluated in both within-subject and cross-subject decoding tasks.

RESULTS

Time-domain and time-frequency analyses revealed that passive movements elicited stronger, more consistent ERPs in healthy subjects, particularly in bilateral motor cortices (contralateral: - 7.29 ± 4.51  μV; ipsilateral: - 4.33 ± 3.69  μV). Stroke patients exhibited impaired mu/beta ERD/ERS in the affected hemisphere during voluntary movements but demonstrated EEG patterns during passive movements resembling those of healthy subjects. Machine learning evaluation highlighted EEGNet's superior performance, achieving 84.19% accuracy in classifying affected vs. unaffected movements in patients, surpassing healthy subject left-right discrimination (58.38%). Cross-subject decoding further validated passive movement efficacy, with EEGNet attaining 86.00% (healthy) and 72.63% (stroke) accuracy, outperforming traditional CSP methods.

CONCLUSIONS

These findings underscore that passive movement elicits consistent neural responses, thereby enhancing the generalizability of decoding algorithms for stroke patients. By integrating exoskeleton-evoked proprioceptive feedback, this paradigm reduces inter-subject variability and improves clinical feasibility. Future work should explore the application of exoskeletons in the combination of active and passive movement for stroke rehabilitation.

Enhancing motor imagery EEG classification with a Riemannian geometry-based spatial filtering (RSF) method

Motor imagery (MI) refers to the mental simulation of movements without physical execution, and it can be captured using electroencephalography (EEG). This area has garnered significant research interest due to its substantial potential in brain-computer interface (BCI) applications, especially for individuals with physical disabilities. However, accurate classification of MI EEG signals remains a major challenge due to their non-stationary nature, low signal-to-noise ratio, and sensitivity to both external and physiological noise. Traditional classification methods, such as common spatial pattern (CSP), often assume that the data is stationary and Gaussian, which limits their applicability in real-world scenarios where these assumptions do not hold. These challenges highlight the need for more robust methods to improve classification accuracy in MI-BCI systems. To address these issues, this study introduces a Riemannian geometry-based spatial filtering (RSF) method that projects EEG signals into a lower-dimensional subspace, maximizing the Riemannian distance between covariance matrices from different classes. By leveraging the inherent geometric properties of EEG data, RSF enhances the discriminative power of the features while maintaining robustness against noise. The performance of RSF was evaluated in combination with ten commonly used MI decoding algorithms, including CSP with linear discriminant analysis (CSP-LDA), Filter Bank CSP (FBCSP), Minimum Distance to Riemannian Mean (MDM), Tangent Space Mapping (TSM), EEGNet, ShallowConvNet (sCNN), DeepConvNet (dCNN), FBCNet, Graph-CSPNet, and LMDA-Net, using six publicly available MI-BCI datasets. The results demonstrate that RSF significantly improves classification accuracy and reduces computational time, particularly for deep learning models with high computational complexity. These findings underscore the potential of RSF as an effective spatial filtering approach for MI EEG classification, providing new insights and opportunities for the development of robust MI-BCI systems. The code for this research is available at https://github.com/PLC-TJU/RSF.

Multi-view collaborative ensemble classification for EEG signals based on 3D second-order difference plot and CSP

Objective.EEG signal analysis methods based on electrical source imaging (ESI) technique have significantly improved classification accuracy and response time. However, for the refined and informative source signals, the current studies have not fully considered their dynamic variability in feature extraction and lacked an effective integration of their dynamic variability and spatial characteristics. Additionally, the adaptability and complementarity of classifiers have not been considered comprehensively. These two aspects lead to the issue of insufficient decoding of source signals, which still limits the application of brain-computer interface (BCI). To address these challenges, this paper proposes a multi-view collaborative ensemble classification method for EEG signals based on three-dimensional second-order difference plot (3D SODP) and common spatial pattern.Approach.First, EEG signals are mapped to the source domain using the ESI technique, and then the source signals in the region of interest are obtained. Next, features from three viewpoints of the source signals are extracted, including 3D SODP features, spatial features, and the weighted fusion of both. Finally, the extracted multi-view features are integrated with subject-specific sub-classifier combination, and a voting mechanism is used to determine the final classification.Main results.The results show that the proposed method achieves classification accuracy of 81.3% and 82.6% respectively in two sessions of the OpenBMI dataset, which is nearly 5% higher than the state-of-the-art method, and maintains the analysis response time required for online BCI.Significance.This paper employs multi-view feature extraction to fully capture the characteristics of the source signals and enhances feature utilization through collaborative ensemble classification. The results demonstrate high accuracy and robust performance, providing a novel approach for online BCI.

Multi-scale convolutional transformer network for motor imagery brain-computer interface

Brain-computer interface (BCI) systems allow users to communicate with external devices by translating neural signals into real-time commands. Convolutional neural networks (CNNs) have been effectively utilized for decoding motor imagery electroencephalography (MI-EEG) signals in BCIs. However, traditional CNN-based methods face challenges such as individual variability in EEG signals and the limited receptive fields of CNNs. This study presents the Multi-Scale Convolutional Transformer (MSCFormer) model that integrates multiple CNN branches for multi-scale feature extraction and a Transformer module to capture global dependencies, followed by a fully connected layer for classification. The multi-branch multi-scale CNN structure effectively addresses individual variability in EEG signals, enhancing the model's generalization capabilities, while the Transformer encoder strengthens global feature integration and improves decoding performance. Extensive experiments on the BCI IV-2a and IV-2b datasets show that MSCFormer achieves average accuracies of 82.95% (BCI IV-2a) and 88.00% (BCI IV-2b), with kappa values of 0.7726 and 0.7599 in five-fold cross-validation, surpassing several state-of-the-art methods. These results highlight MSCFormer's robustness and accuracy, underscoring its potential in EEG-based BCI applications. The code has been released in https://github.com/snailpt/MSCFormer .

Nature documentaries vs. quiet rest: no evidence for an impact on event-related desynchronization during motor imagery and neurofeedback

Motor imagery (MI) in combination with neurofeedback (NF) has emerged as a promising approach in motor neurorehabilitation, facilitating brain activity modulation and promoting motor learning. Although MI-NF has been demonstrated to enhance motor performance and cortical plasticity, its efficacy varies considerably across individuals. Various context factors have been identified as influencing neurophysiological outcomes in motor execution and MI, however, their specific impact on event-related desynchronization (ERD), a key neurophysiological marker in NF, remains insufficiently understood. Previous research suggested that declarative interference following MI-NF may serve as a context factor hindering the progression of ERD. Yet, no significant changes in ERD within the mu and beta (8-30 Hz) frequency bands were observed across blocks in either a declarative interference or a control condition. This raises the question of whether the absence of ERD modulation could be attributed to the break task that was common to both declarative interference and control condition: watching nature documentaries immediately after MI blocks. To investigate this, we conducted a follow-up study replicating the original methodology while collecting new data. We compared NF-MI-ERD between groups with and without nature documentaries as a post-MI condition. Participants completed three sessions of kinesthetic MI-NF training involving a finger-tapping task over two consecutive days, with quiet rest as the post-MI condition (group quiet rest). 64-channel EEG data were analyzed from 17 healthy participants (8 females, 18-35 years, M and SD: 25.2 ± 4.2 years). Data were compared to a previously recorded dataset (group documentaries), in which 17 participants (10 females, 23-32 years, M and SD: 25.8 ± 2.5 years) watched nature documentaries after MI blocks. The results showed no significant main effects for blocks or group, though a session-by-group interaction was observed. Post-hoc tests, however, did not reveal significant differences in ERD development between the groups across individual blocks. These findings do not provide evidence that nature documentaries used as a post-MI condition negatively affect across-block development of NF-MI-ERD. This study highlights the importance of exploring additional context factors in MI-NF training to better understand their influence on ERD development.

LGFormer: integrating local and global representations for EEG decoding

Objective.Electroencephalography (EEG) decoding is challenging because of its temporal variability and low signal-to-noise ratio, which complicate the extraction of meaningful information from signals. Although convolutional neural networks (CNNs) effectively extract local features from EEG signals, they are constrained by restricted receptive fields. In contrast, transformers excel at capturing global dependencies through self-attention mechanisms but often require extensive training data and computational resources, which limits their efficiency on EEG datasets with limited samples.Approach.In this paper, we propose LGFormer, a hybrid network designed to efficiently learn both local and global representations for EEG decoding. LGFormer employs a deep attention module to extract global information from EEG signals, dynamically adjusting the focus of CNNs. Subsequently, LGFormer incorporates a local-enhanced transformer, combining the strengths of CNNs and transformers to achieve multiscale perception from local to global. Despite integrating multiple advanced techniques, LGFormer maintains a lightweight design and training efficiency.Main results.LGFormer achieves state-of-the-art performance within 200 training epochs across four public datasets, including motor imagery, cognitive workload, and error-related negativity decoding tasks. Additionally, we propose a novel spatial and temporal attention visualization method, revealing that LGFormer captures discriminative spatial and temporal features, enhancing model interpretability and providing insights into its decision-making process.Significance.In summary, LGFormer demonstrates superior performance while maintaining high training efficiency across different tasks, highlighting its potential as a versatile and practical model for EEG decoding.

EEG-DG: A Multi-Source Domain Generalization Framework for Motor Imagery EEG Classification

Motorimagery EEG classification plays a crucial role in non-invasive Brain-Computer Interface (BCI) research. However, the performance of classification is affected by the non-stationarity and individual variations of EEG signals. Simply pooling EEG data with different statistical distributions to train a classification model can severely degrade the generalization performance. To address this issue, the existing methods primarily focus on domain adaptation, which requires access to the test data during training. This is unrealistic and impractical in many EEG application scenarios. In this paper, we propose a novel multi-source domain generalization framework called EEG-DG, which leverages multiple source domains with different statistical distributions to build generalizable models on unseen target EEG data. We optimize both the marginal and conditional distributions to ensure the stability of the joint distribution across source domains and extend it to a multi-source domain generalization framework to achieve domain-invariant feature representation, thereby alleviating calibration efforts. Systematic experiments conducted on a simulative dataset, BCI competition IV 2a, 2b, and OpenBMI datasets, demonstrate the superiority and competitive performance of our proposed framework over other state-of-the-art methods. Specifically, EEG-DG achieves average classification accuracies of 81.79% and 87.12% on datasets IV-2a and IV-2b, respectively, and 78.37% and 76.94% for inter-session and inter-subject evaluations on dataset OpenBMI, which even outperforms some domain adaptation methods.

EEG Detection and Prediction of Freezing of Gait in Parkinson's Disease Based on Spatiotemporal Coherent Modes

OBJECTIVE

Freezing of gait (FOG) in Parkinson's disease has a complex neurological mechanism. Compared with other modalities, electroencephalogram (EEG) can reflect FOG-related brain activity of both motor and non-motor symptoms. However, EEG-based FOG prediction methods often extract time, spatial, frequency, time-frequency, or phase information separately, which fragments the coupling among these heterogeneous features and cannot completely characterize the brain dynamics when FOG occurs.

METHODS

In this study, dynamic spatiotemporal coherent modes of EEG were studied and used for FOG detection and prediction. A dynamic mode decomposition (DMD) method was first applied to extract the spatiotemporal coherent modes. Dynamic changes of the spatiotemporal modes, in both amplitude and phase of motor-related frequency bands, were evaluated with analytic common spatial patterns (ACSP) to extract the essential differences among normal, freezing, and transitional gaits.

RESULTS

The proposed method was verified in practical clinical data. Results showed that, in the detection task, the DMD-ACSP achieved an accuracy of 86.4 $\pm$ 3.6% and a sensitivity of 83.5 $\pm$ 4.3%. In the prediction task, 86.5 $\pm$ 3.2% accuracy and 86.7 $\pm$ 7.8% sensitivity were achieved.

CONCLUSION

Comparative studies showed that the DMD-ACSP method significantly improves FOG detection and prediction performance. Moreover, the DMD-ACSP reveals the spatial patterns of dynamic brain functional connectivity, which best discriminate the different gaits.

SIGNIFICANCE

The spatiotemporal coherent modes may provide a useful indication for personalized intervention and transcranial magnetic stimulation neuromodulation in medical practices.

Recognition of brain activities via graph-based long short-term memory-convolutional neural network

Introduction

Human brain activities are always difficult to recognize due to its diversity and susceptibility to disturbance. With its unique capability of measuring brain activities, magnetoencephalography (MEG), as a high temporal and spatial resolution neuroimaging technique, has been used to identify multi-task brain activities. Accurately and robustly classifying motor imagery (MI) and cognitive imagery (CI) from MEG signals is a significant challenge in the field of brain-computer interface (BCI).

Methods

In this study, a graph-based long short-term memory-convolutional neural network (GLCNet) is proposed to classify the brain activities in MI and CI tasks. It was characterized by implementing three modules of graph convolutional network (GCN), spatial convolution and long short-term memory (LSTM) to effectively extract time-frequency-spatial features simultaneously. For performance evaluation, our method was compared with six benchmark algorithms of FBCSP, FBCNet, EEGNet, DeepConvNets, Shallow ConvNet and MEGNet on two public datasets of MEG-BCI and BCI competition IV dataset 3.

Results

The results demonstrated that the proposed GLCNet outperformed other models with the average accuracies of 78.65% and 65.8% for two classification and four classification on the MEG-BCI dataset, respectively.

Discussion

It was concluded that the GLCNet enhanced the model's adaptability in handling individual variability with robust performance. This would contribute to the exploration of brain activates in neuroscience.

GREEN: A lightweight architecture using learnable wavelets and Riemannian geometry for biomarker exploration with EEG signals

Spectral analysis using wavelets is widely used for identifying biomarkers in EEG signals. Recently, Riemannian geometry has provided an effective mathematical framework for predicting biomedical outcomes from multichannel electroencephalography (EEG) recordings while showing concord with neuroscientific domain knowledge. However, these methods rely on handcrafted rules and sequential optimization. In contrast, deep learning (DL) offers end-to-end trainable models achieving state-of-the-art performance on various prediction tasks but lacks interpretability and interoperability with established neuroscience concepts. We introduce Gabor Riemann EEGNet (GREEN), a lightweight neural network that integrates wavelet transforms and Riemannian geometry for processing raw EEG data. Benchmarking on six prediction tasks across four datasets with over 5,000 participants, GREEN outperformed non-deep state-of-the-art models and performed favorably against large DL models while using orders-of-magnitude fewer parameters. Computational experiments showed that GREEN facilitates learning sparse representations without compromising performance. By integrating domain knowledge, GREEN combines a desirable complexity-performance trade-off with interpretable representations.

Classification of motor imagery EEG with ensemble RNCA model

Motor Imagery (MI) based brain-computer interface (BCI) systems are used for regaining the motor functions of neurophysiologically affected persons. But the performance of MI tasks is degraded due to the presence of redundant EEG channels. Hence, a novel ensemble regulated neighborhood component analysis (ERNCA) method provides a perfect identification of neural region that stimulate motor movements. Domains of statistical, frequency, spatial and transform-based features narrowed down the misclassification rate. The gradient boosting method selects the relevant features thereby reduces the computational complexity. Finally, Bayesian optimized ensemble classifier finetuned the classification accuracies of 97.22 % and 91.62 % for Datasets IIIa and IVa respectively. This approach is further strengthened by analyzing real-time data with the accuracy of 93.75 %. This method qualifies out of four benchmark methods with significant percent of improvement in accuracy for these three datasets. As per the spatial distribution of refined EEG channels, majority of the brain's motor functions concentrates on frontal and central cortex regions of brain.

The influence of different flavor peptides on brain perception via scalp electroencephalogram and development of a taste model

Traditional taste evaluation methods often rely on subjective assessments, introducing biases. To address this, we propose using electroencephalography (EEG) to explore the link between brain activity and taste perception. Our EEG analysis showed significant activity differences in specific brain regions, particularly at electrodes Pz, FT7, F7, and TP7, highlighting their role in taste signal processing. Consistent activity at Pz across various tastes supports the development of a mathematical model and sensory evaluation system. We used wavelet packet transform for EEG signal preprocessing, followed by feature extraction and classification with the Common Spatial Pattern (CSP) and Support Vector Machine (SVM) algorithms. Testing five taste categories-sour, sweet, bitter, salty, and umami-resulted in an overall prediction accuracy of 0.7613, with the highest accuracy of 0.8235 for "sweet" taste. Despite challenges in predicting "sour" and "salty" tastes, our study demonstrates the potential of combining wavelet packet transform, CSP, and SVM for EEG-based taste classification.

A composite improved attention convolutional network for motor imagery EEG classification

Introduction

A brain-computer interface (BCI) is an emerging technology that aims to establish a direct communication pathway between the human brain and external devices. Motor imagery electroencephalography (MI-EEG) signals are analyzed to infer users' intentions during motor imagery. These signals hold potential for applications in rehabilitation training and device control. However, the classification accuracy of MI-EEG signals remains a key challenge for the development of BCI technology.

Methods

This paper proposes a composite improved attention convolutional network (CIACNet) for MI-EEG signals classification. CIACNet utilizes a dual-branch convolutional neural network (CNN) to extract rich temporal features, an improved convolutional block attention module (CBAM) to enhance feature extraction, temporal convolutional network (TCN) to capture advanced temporal features, and multi-level feature concatenation for more comprehensive feature representation.

Results

The CIACNet model performs well on both the BCI IV-2a and BCI IV-2b datasets, achieving accuracies of 85.15 and 90.05%, respectively, with a kappa score of 0.80 on both datasets. These results indicate that the CIACNet model's classification performance exceeds that of four other comparative models.

Conclusion

Experimental results demonstrate that the proposed CIACNet model has strong classification capabilities and low time cost. Removing one or more blocks results in a decline in the overall performance of the model, indicating that each block within the model makes a significant contribution to its overall effectiveness. These results demonstrate the ability of the CIACNet model to reduce time costs and improve performance in motor imagery brain-computer interface (MI-BCI) systems, while also highlighting its practical applicability.

A cross-domain-based channel selection method for motor imagery

Selecting channels for motor imagery (MI)-based brain-computer interface (BCI) systems can not only enhance the portability of the systems, but also improve the decoding performance. Hence, we propose a cross-domain-based channel selection (CDCS) approach, which effectively minimizes the number of EEG channels used while maintaining high accuracy in MI recognition. The EEG source imaging (ESI) technique is employed to map scalp EEG into the cortical source domain. We divide the equivalent dipoles in the source domain into different regions by k-means clustering. Then, we calculate the band energy (5-40 Hz) of time series of dipoles in these regions by power spectral density (PSD), and the regions with the highest and lowest band energy are selected as the region of interests (ROIs) in the source domain. Subsequently, Pearson correlation coefficients between the dipole time series in ROIs and scalp EEG are used as the criterion for channel selection and a multi-trial-sorting-based channel selection strategy is proposed. Finally, we propose the CDCS-based MI classification framework, where common spatial pattern is applied to extract features and linear discriminant analysis is used to identify MI tasks. The CDCS method demonstrated significant improvement in decoding accuracy on two public datasets, achieving increases of 18.51% and 13.37% compared to all-channel method, and 10.74% and 3.43% compared to the three-channel method. The experimental results validated that CDCS is effective in selecting important channels.

A multi-branch, multi-scale, and multi-view CNN with lightweight temporal attention mechanism for EEG-based motor imagery decoding

Convolutional neural networks (CNNs) have been widely utilized for decoding motor imagery (MI) from electroencephalogram (EEG) signals. However, extracting discriminative spatial-temporal-spectral features from low signal-to-noise ratio EEG signals remains challenging. This paper proposes MBMSNet , a multi-branch, multi-scale, and multi-view CNN with a lightweight temporal attention mechanism for EEG-Based MI decoding. Specifically, MBMSNet first extracts multi-view representations from raw EEG signals, followed by independent branches to capture spatial, spectral, temporal-spatial, and temporal-spectral features. Each branch includes a domain-specific convolutional layer, a variance layer, and a temporal attention layer. Finally, the features derived from each branch are concatenated with weights and classified through a fully connected layer. Experiments demonstrate MBMSNet outperforms state-of-the-art models, achieving accuracies of 84.60% on BCI Competition IV 2a, 87.80% on 2b, and 74.58% on OpenBMI, showcasing its potential for robust BCI applications.

MFRC-Net: Multi-Scale Feature Residual Convolutional Neural Network for Motor Imagery Decoding

Motor imagery (MI) decoding is the basis of external device control via electroencephalogram (EEG). However, the majority of studies prioritize enhancing the accuracy of decoding methods, often overlooking the magnitude and computational resource demands of deep learning models. In this study, we propose a novel lightweight Multi-Scale Feature Residual Convolutional Neural Network (MFRC-Net). MFRC-Net primarily consists of two blocks: temporal multi-scale residual convolution blocks and cross-domain dual-stream spatial convolution blocks. The former captures dynamic changes in EEG signals across various time scales through multi-scale grouped convolution and backbone temporal convolution skip connections; the latter improves local spatial feature extraction and calibrates feature mapping through the introduction of cross-domain spatial filtering layers. Furthermore, by specifically optimizing the loss function, MFRC-Net effectively reduces sensitivity to outliers. Experiment results on the BCI Competition IV 2a dataset and the SHU dataset demonstrate that, with a parameter size of only 13 K, MFRC-Net achieves accuracy of 85.1% and 69.3%, respectively, surpassing current state-of-the-art models. The integration of temporal multi-scale residual convolution blocks and cross-domain dual-stream spatial convolution blocks in lightweight models significantly boosts performance, as evidenced by ablation studies and visualizations.

Supervised Contrastive Learning-Based Domain Generalization Network for Cross-Subject Motor Decoding

Developing an electroencephalogram (EEG)-based motor imagery and motor execution (MI/ME) decoding system that is both highly accurate and calibration-free for cross-subject applications remains challenging due to domain shift problem inherent in such scenario. Recent research has increasingly embraced transfer learning strategies, especially domain adaptation techniques. However, domain adaptation becomes impractical when the target subject data is either difficult to obtain or unknown. To address this issue, we propose a supervised contrastive learning-based domain generalization network (SCLDGN) for cross-subject MI/ME decoding. Firstly, the feature encoder is purposefully designed to learn the EEG discriminative feature representations. Secondly, the domain alignment based on deep correlation alignment constrains the representations distance across various domains to learn domain-invariant features. In addition, the class regularization block is proposed, where the supervised contrastive learning with domain-agnostic mixup is established to learn the class-relevant features and achieve class-level alignment. Finally, in the latent space, clusters of domain-agnostic representations from the same class are mapped closer together. Consequently, SCLDGN is capable of learning domain-invariant and class-relevant discriminative representations, which are essential for effective cross-subject decoding. Extensive experiments conducted on six MI/ME datasets demonstrate the effectiveness of the proposed method in comparison with other state-of-the-art approaches. Furthermore, ablation study and visualization analyses explain the generalization mechanism of the proposed method and also show neurophysiologically meaningful patterns related to MI/ME.

Decoding Depth of Meditation: Electroencephalography Insights From Expert Vipassana Practitioners

Background

Meditation practices have demonstrated numerous psychological and physiological benefits, but capturing the neural correlates of varying meditative depths remains challenging. In this study, we aimed to decode self-reported time-varying meditative depth in expert practitioners using electroencephalography (EEG).

Methods

Expert Vipassana meditators (n = 34) participated in 2 separate sessions. Participants reported their meditative depth on a personally defined 1 to 5 scale using both traditional probing and a novel spontaneous emergence method. EEG activity and effective connectivity in theta, alpha, and gamma bands were used to predict meditative depth using machine/deep learning, including a novel method that fused source activity and connectivity information.

Results

We achieved significant accuracy in decoding self-reported meditative depth across unseen sessions. The spontaneous emergence method yielded improved decoding performance compared with traditional probing and correlated more strongly with postsession outcome measures. Best performance was achieved by a novel machine learning method that fused spatial, spectral, and connectivity information. Conventional EEG channel-level methods and preselected default mode network regions fell short in capturing the complex neural dynamics associated with varying meditation depths.

Conclusions

This study demonstrates the feasibility of decoding personally defined meditative depth using EEG. The findings highlight the complex, multivariate nature of neural activity during meditation and introduce spontaneous emergence as an ecologically valid and less obtrusive experiential sampling method. These results have implications for advancing neurofeedback techniques and enhancing our understanding of meditative practices.

Adaptive deep feature representation learning for cross-subject EEG decoding

BACKGROUND

The collection of substantial amounts of electroencephalogram (EEG) data is typically time-consuming and labor-intensive, which adversely impacts the development of decoding models with strong generalizability, particularly when the available data is limited. Utilizing sufficient EEG data from other subjects to aid in modeling the target subject presents a potential solution, commonly referred to as domain adaptation. Most current domain adaptation techniques for EEG decoding primarily focus on learning shared feature representations through domain alignment strategies. Since the domain shift cannot be completely removed, target EEG samples located near the edge of clusters are also susceptible to misclassification.

METHODS

We propose a novel adaptive deep feature representation (ADFR) framework to improve the cross-subject EEG classification performance through learning transferable EEG feature representations. Specifically, we first minimize the distribution discrepancy between the source and target domains by employing maximum mean discrepancy (MMD) regularization, which aids in learning the shared feature representations. We then utilize the instance-based discriminative feature learning (IDFL) regularization to make the learned feature representations more discriminative. Finally, the entropy minimization (EM) regularization is further integrated to adjust the classifier to pass through the low-density region between clusters. The synergistic learning between above regularizations during the training process enhances EEG decoding performance across subjects.

RESULTS

The effectiveness of the ADFR framework was evaluated on two public motor imagery (MI)-based EEG datasets: BCI Competition III dataset 4a and BCI Competition IV dataset 2a. In terms of average accuracy, ADFR achieved improvements of 3.0% and 2.1%, respectively, over the state-of-the-art methods on these datasets.

CONCLUSIONS

The promising results highlight the effectiveness of the ADFR algorithm for EEG decoding and show its potential for practical applications.

Multiband Convolutional Riemannian Network With Band-Wise Riemannian Triplet Loss for Motor Imagery Classification

This paper presents a novel motor imagery classification algorithm that uses an overlapping multiscale multiband convolutional Riemannian network with band-wise Riemannian triplet loss to improve classification performance. Despite the superior performance of the Riemannian approach over the common spatial pattern filter approach, deep learning methods that generalize the Riemannian approach have received less attention. The proposed algorithm develops a state-of-the-art multiband Riemannian network that reduces the potential overfitting problem of Riemannian networks, a drawback of Riemannian networks due to their inherent large feature dimension from covariance matrix, by using fewer subbands with discriminative frequency diversity, by inserting convolutional layers before computing the subband covariance matrix, and by regularizing subband networks with Riemannian triplet loss. The proposed method is evaluated using the publicly available datasets, the BCI Competition IV dataset 2a and the OpenBMI dataset. The experimental results confirm that the proposed method improves performance, in particular achieving state-of-the-art classification accuracy among the currently studied Riemannian networks.

Graph Neural Networks on SPD Manifolds for Motor Imagery Classification: A Perspective From the Time-Frequency Analysis

The motor imagery (MI) classification has been a prominent research topic in brain-computer interfaces (BCIs) based on electroencephalography (EEG). Over the past few decades, the performance of MI-EEG classifiers has seen gradual enhancement. In this study, we amplify the geometric deep-learning-based MI-EEG classifiers from the perspective of time-frequency analysis, introducing a new architecture called Graph-CSPNet. We refer to this category of classifiers as Geometric Classifiers, highlighting their foundation in differential geometry stemming from EEG spatial covariance matrices. Graph-CSPNet utilizes novel manifold-valued graph convolutional techniques to capture the EEG features in the time-frequency domain, offering heightened flexibility in signal segmentation for capturing localized fluctuations. To evaluate the effectiveness of Graph-CSPNet, we employ five commonly used publicly available MI-EEG datasets, achieving near-optimal classification accuracies in nine out of 11 scenarios. The Python repository can be found at https://github.com/GeometricBCI/Tensor-CSPNet-and-Graph-CSPNet.

Attention Induced Dual Convolutional-Capsule Network (AIDC-CN): A deep learning framework for motor imagery classification

In recent times, Electroencephalography (EEG)-based motor imagery (MI) decoding has garnered significant attention due to its extensive applicability in healthcare, including areas such as assistive robotics and rehabilitation engineering. Nevertheless, the decoding of EEG signals presents considerable challenges owing to their inherent complexity, non-stationary characteristics, and low signal-to-noise ratio. Notably, deep learning-based classifiers have emerged as a prominent focus for addressing the EEG signal decoding process. This study introduces a novel deep learning classifier named the Attention Induced Dual Convolutional-Capsule Network (AIDC-CN) with the specific aim of accurately categorizing various motor imagination class labels. To enhance the classifier's performance, a dual feature extraction approach leveraging spectrogram and brain connectivity networks has been employed, diversifying the feature set in the classification task. The main highlights of the proposed AIDC-CN classifier includes the introduction of a dual convolution layer to handle the brain connectivity and spectrogram features, addition of a novel self-attention module (SAM) to accentuate the relevant parts of the convolved spectrogram features, introduction of a new cross-attention module (CAM) to refine the outputs obtained from the dual convolution layers and incorporation of a Gaussian Error Linear Unit (GELU) based dynamic routing algorithm to strengthen the coupling among the primary and secondary capsule layers. Performance analysis undertaken on four public data sets depict the superior performance of the proposed model with respect to the state-of-the-art techniques. The code for this model is available at https://github.com/RiteshSurChowdhury/AIDC-CN.

Partial prior transfer learning based on self-attention CNN for EEG decoding in stroke patients

The utilization of motor imagery-based brain-computer interfaces (MI-BCI) has been shown to assist stroke patients activate motor regions in the brain. In particular, the brain regions activated by unilateral upper limb multi-task are more extensive, which is more beneficial for rehabilitation, but it also increases the difficulty of decoding. In this paper, self-attention convolutional neural network based partial prior transfer learning (SACNN-PPTL) is proposed to improve the classification performance of patients' MI multi-task. The backbone network of the algorithm is SACNN, which accords with the inherent features of electroencephalogram (EEG) and contains the temporal feature module, the spatial feature module and the feature generalization module. In addition, PPTL is introduced to transfer part of the target domain while preserving the generalization of the base model while improving the specificity of the target domain. In the experiment, five backbone networks and three training modes are selected as comparison algorithms. The experimental results show that SACNN-PPTL had a classification accuracy of 55.4%±0.17 in four types of MI tasks for 22 patients, which is significantly higher than comparison algorithms (P < 0.05). SACNN-PPTL effectively improves the decoding performance of MI tasks and promotes the development of BCI-based rehabilitation for unilateral upper limb.

Steady-State Visual Evoked Potential-Based Brain-Computer Interface System for Enhanced Human Activity Monitoring and Assessment

Advances in brain-computer interfaces (BCIs) have enabled direct and functional connections between human brains and computing systems. Recent developments in artificial intelligence have also significantly improved the ability to detect brain activity patterns. In particular, using steady-state visual evoked potentials (SSVEPs) in BCIs has enabled noticeable advances in human activity monitoring and identification. However, the lack of publicly available electroencephalogram (EEG) datasets has limited the development of SSVEP-based BCI systems (SSVEP-BCIs) for human activity monitoring and assisted living. This study aims to provide an open-access multicategory EEG dataset created under the SSVEP-BCI paradigm, with participants performing forward, backward, left, and right movements to simulate directional control commands in a virtual environment developed in Unity. The purpose of these actions is to explore how the brain responds to visual stimuli of control commands. An SSVEP-BCI system is proposed to enable hands-free control of a virtual target in the virtual environment allowing participants to maneuver the virtual target using only their brain activity. This work demonstrates the feasibility of using SSVEP-BCIs in human activity monitoring and assessment. The preliminary experiment results indicate the effectiveness of the developed system with high accuracy, successfully classifying 89.88% of brainwave activity.

Session-independent subject-adaptive mental imagery BCI using selective filter-bank adaptive Riemannian features

The brain-computer interfaces (BCIs) facilitate the users to exploit information encoded in neural signals, specifically electroencephalogram (EEG), to control devices and for neural rehabilitation. Mental imagery (MI)-driven BCI predicts the user's pre-meditated mental objectives, which could be deployed as command signals. This paper presents a novel learning-based framework for classifying MI tasks using EEG-based BCI. In particular, our work focuses on the variation in inter-session data and the extraction of multi-spectral user-tailored features for robust performance. Thus, the goal is to create a calibration-free subject-adaptive learning framework for various mental imagery tasks not restricted to motor imagery alone. In this regard, critical spectral bands and the best temporal window are first selected from the EEG training trials of the subject based on the Riemannian user learning distance metric (Dscore) that checks for distinct and stable patterns. The filtered covariance matrices of the EEG trials in each spectral band are then transformed towards a reference covariance matrix using the Riemannian transfer learning, enabling the different sessions to be comparable. The evaluation of our proposed Selective Time-window and Multi-scale Filter-Bank with Adaptive Riemannian (STFB-AR) features on four public datasets, including disabled subjects, showed around 15% and 8% improvement in mean accuracy over baseline and fixed filter-bank models, respectively.

Motor-Related EEG Analysis Using a Pole Tracking Approach

This study introduces an alternative approach to electroencephalography (EEG) time-frequency analysis based on time-varying autoregressive (TV-AR) models in a cascade configuration to independently monitor key EEG spectral components. The method is evaluated for its neurophysiological interpretation and effectiveness in motor-related brain-computer interface (BCI) applications. Specifically, we assess the ability of the tracked EEG poles to discriminate between rest, movement execution (ME) and movement imagination (MI) in healthy subjects, as well as movement attempts (MA) in individuals with spinal cord injury (SCI). Our results show that pole tracking effectively captures broad changes in EEG dynamics, such as transitions between rest and movement-related states. It outperformed traditional EEG-based features, increasing detection accuracy for ME by an average of 4.1% (with individual improvements reaching as high as 15%) and MI by an average of 4.5% (up to 13.8%) compared to time-domain low-frequency EEG features. Similarly, compared to alpha/beta band power, the method improved ME detection by an average of 5.9% (up to 10.4%) and MI by an average of 4.3% (up to 10.2%), with results averaged across 15 healthy participants. In one participant with SCI, pole tracking improved MA detection by 12.9% over low-frequency EEG features and 4.8% over alpha/beta band power. However, its ability to distinguish finer movement details within specific movement types was limited. Additionally, the temporal evolution of the extracted pole tracking features revealed event-related desynchronization phenomena, typically observed during ME, MA and MI, as well as increases in frequency, which are of neurophysiological interest.

Sensorimotor brain-computer interface performance depends on signal-to-noise ratio but not connectivity of the mu rhythm in a multiverse analysis of longitudinal data

Objective.Serving as a channel for communication with locked-in patients or control of prostheses, sensorimotor brain-computer interfaces (BCIs) decode imaginary movements from the recorded activity of the user's brain. However, many individuals remain unable to control the BCI, and the underlying mechanisms are unclear. The user's BCI performance was previously shown to correlate with the resting-state signal-to-noise ratio (SNR) of the mu rhythm and the phase synchronization (PS) of the mu rhythm between sensorimotor areas. Yet, these predictors of performance were primarily evaluated in a single BCI session, while the longitudinal aspect remains rather uninvestigated. In addition, different analysis pipelines were used to estimate PS in source space, potentially hindering the reproducibility of the results.Approach.To systematically address these issues, we performed an extensive validation of the relationship between pre-stimulus SNR, PS, and session-wise BCI performance using a publicly available dataset of 62 human participants performing up to 11 sessions of BCI training. We performed the analysis in sensor space using the surface Laplacian and in source space by combining 24 processing pipelines in a multiverse analysis. This way, we could investigate how robust the observed effects were to the selection of the pipeline.Main results.Our results show that SNR had both between- and within-subject effects on BCI performance for the majority of the pipelines. In contrast, the effect of PS on BCI performance was less robust to the selection of the pipeline and became non-significant after controlling for SNR.Significance.Taken together, our results demonstrate that changes in neuronal connectivity within the sensorimotor system are not critical for learning to control a BCI, and interventions that increase the SNR of the mu rhythm might lead to improvements in the user's BCI performance.

STaRNet: A spatio-temporal and Riemannian network for high-performance motor imagery decoding

Brain-computer interfaces (BCIs), representing a transformative form of human-computer interaction, empower users to interact directly with external environments through brain signals. In response to the demands for high accuracy, robustness, and end-to-end capabilities within BCIs based on motor imagery (MI), this paper introduces STaRNet, a novel model that integrates multi-scale spatio-temporal convolutional neural networks (CNNs) with Riemannian geometry. Initially, STaRNet integrates a multi-scale spatio-temporal feature extraction module that captures both global and local features, facilitating the construction of Riemannian manifolds from these comprehensive spatio-temporal features. Subsequently, a matrix logarithm operation transforms the manifold-based features into the tangent space, followed by a dense layer for classification. Without preprocessing, STaRNet surpasses state-of-the-art (SOTA) models by achieving an average decoding accuracy of 83.29% and a kappa value of 0.777 on the BCI Competition IV 2a dataset, and 95.45% accuracy with a kappa value of 0.939 on the High Gamma Dataset. Additionally, a comparative analysis between STaRNet and several SOTA models, focusing on the most challenging subjects from both datasets, highlights exceptional robustness of STaRNet. Finally, the visualizations of learned frequency bands demonstrate that temporal convolutions have learned MI-related frequency bands, and the t-SNE analyses of features across multiple layers of STaRNet exhibit strong feature extraction capabilities. We believe that the accurate, robust, and end-to-end capabilities of the STaRNet will facilitate the advancement of BCIs.

Significance of EEG-electrode combinations while calculating filters with common spatial patterns

Objective

Common spatial pattern (CSP) is a common filter technique used for pre-processing of electroencephalography (EEG) signals for imaginary movement classification tasks. It is crucial to reduce the amount of features especially in cases where few data is available. Therefore, different approaches to reduce the amount of electrodes used for CSP calculation are tried in this research.

Methods

Freely available EEG datasets are used for the evaluation. To evaluate the approaches a simple classification pipeline consisting mainly of the CSP calculation and linear discriminant analysis for classification is used. A baseline over all electrodes is calculated and compared against the results of the approaches.

Results

The most promising approach is to use the ability of CSP to provide information about the origin of the created filter. An algorithm that extracts the important electrodes from the CSP utilizing these information is proposed.The results show that using subject specific electrode positions has a positive impact on accuracy for the classification task. Further, it is shown that good performing electrode combinations in one session are not necessarily good performing electrodes in another session of the same subject. In addition to the combinations calculated using the developed algorithm, 26 additional electrode combinations are proposed. These can be taken into account when selecting well-performing electrode combinations. In this research we could achieve an accuracy improvement of over 10%.

Conclusions

Carefully selecting the correct electrode combination can improve accuracy for classifying an imaginary movement task.

Improved motor imagery training for subject's self-modulation in EEG-based brain-computer interface

For the electroencephalogram- (EEG-) based motor imagery (MI) brain-computer interface (BCI) system, more attention has been paid to the advanced machine learning algorithms rather than the effective MI training protocols over past two decades. However, it is crucial to assist the subjects in modulating their active brains to fulfill the endogenous MI tasks during the calibration process, which will facilitate signal processing using various machine learning algorithms. Therefore, we propose a trial-feedback paradigm to improve MI training and introduce a non-feedback paradigm for comparison. Each paradigm corresponds to one session. Two paradigms are applied to the calibration runs of corresponding sessions. And their effectiveness is verified in the subsequent testing runs of respective sessions. Different from the non-feedback paradigm, the trial-feedback paradigm presents a topographic map and its qualitative evaluation in real time after each MI training trial, so the subjects can timely realize whether the current trial successfully induces the event-related desynchronization/event-related synchronization (ERD/ERS) phenomenon, and then they can adjust their brain rhythm in the next MI trial. Moreover, after each calibration run of the trial-feedback session, a feature distribution is visualized and quantified to show the subjects' abilities to distinguish different MI tasks and promote their self-modulation in the next calibration run. Additionally, if the subjects feel distracted during the training processes of the non-feedback and trial-feedback sessions, they can execute the blinking movement which will be captured by the electrooculogram (EOG) signals, and the corresponding MI training trial will be abandoned. Ten healthy participants sequentially performed the non-feedback and trial-feedback sessions on the different days. The experiment results showed that the trial-feedback session had better spatial filter visualization, more beneficiaries, higher average off-line and on-line classification accuracies than the non-feedback session, suggesting the trial-feedback paradigm's usefulness in subject's self-modulation and good ability to perform MI tasks.

Connectivity study on resting-state EEG between motor imagery BCI-literate and BCI-illiterate groups

Objective.Although motor imagery-based brain-computer interface (MI-BCI) holds significant potential, its practical application faces challenges such as BCI-illiteracy. To mitigate this issue, researchers have attempted to predict BCI-illiteracy by using the resting state, as this was found to be associated with BCI performance. As connectivity's significance in neuroscience has grown, BCI researchers have applied connectivity to it. However, the issues of connectivity have not been considered fully. First, although various connectivity metrics exist, only some have been used to predict BCI-illiteracy. This is problematic because each metric has a distinct hypothesis and perspective to estimate connectivity, resulting in different outcomes according to the metric. Second, the frequency range affects the connectivity estimation. In addition, it is still unknown whether each metric has its own optimal frequency range. Third, the way that estimating connectivity may vary depending upon the dataset has not been investigated. Meanwhile, we still do not know a great deal about how the resting state electroencephalography (EEG) network differs between BCI-literacy and -illiteracy.Approach.To address the issues above, we analyzed three large public EEG datasets using three functional connectivity and three effective connectivity metrics by employing diverse graph theory measures. Our analysis revealed that the appropriate frequency range to predict BCI-illiteracy varies depending upon the metric. The alpha range was found to be suitable for the metrics of the frequency domain, while alpha + theta were found to be appropriate for multivariate Granger causality. The difference in network efficiency between BCI-literate and -illiterate groups was constant regardless of the metrics and datasets used. Although we observed that BCI-literacy had stronger connectivity, no other significant constructional differences were found.Significance.Based upon our findings, we predicted MI-BCI performance for the entire dataset. We discovered that combining several graph features could improve the prediction's accuracy.

Channel reflection: Knowledge-driven data augmentation for EEG-based brain-computer interfaces

A brain-computer interface (BCI) enables direct communication between the human brain and external devices. Electroencephalography (EEG) based BCIs are currently the most popular for able-bodied users. To increase user-friendliness, usually a small amount of user-specific EEG data are used for calibration, which may not be enough to develop a pure data-driven decoding model. To cope with this typical calibration data shortage challenge in EEG-based BCIs, this paper proposes a parameter-free channel reflection (CR) data augmentation approach that incorporates prior knowledge on the channel distributions of different BCI paradigms in data augmentation. Experiments on eight public EEG datasets across four different BCI paradigms (motor imagery, steady-state visual evoked potential, P300, and seizure classifications) using different decoding algorithms demonstrated that: (1) CR is effective, i.e., it can noticeably improve the classification accuracy; (2) CR is robust, i.e., it consistently outperforms existing data augmentation approaches in the literature; and, (3) CR is flexible, i.e., it can be combined with other data augmentation approaches to further improve the performance. We suggest that data augmentation approaches like CR should be an essential step in EEG-based BCIs. Our code is available online.

A novel feature extraction method PSS-CSP for binary motor imagery - based brain-computer interfaces

In order to improve the performance of binary motor imagery (MI) - based brain-computer interfaces (BCIs) using electroencephalography (EEG), a novel method (PSS-CSP) is proposed, which combines spectral subtraction with common spatial pattern. Spectral subtraction is an effective denoising method which is initially adopted to process MI-based EEG signals for binary BCIs in this work. On this basis, we proposed a novel feature extraction method called power spectral subtraction-based common spatial pattern (PSS-CSP) , which calculates the differences in power spectrum between binary classes of EEG signals and uses the differences in the feature extraction process. Additionally, support vector machine (SVM) algorithm is used for signal classification. Results show the proposed method (PSS-CSP) outperforms certain existing methods, achieving a classification accuracy of 76.8% on the BCIIV dataset 2b, and 76.25% and 77.38% on the OpenBMI dataset session 1 and session 2, respectively.

Channel- and Label-Flip Data Augmentation for Motor Imagery-Based Brain-Computer Interfaces

Achieving high classification accuracy in motor-imagery-based brain-computer interfaces (BCIs) requires substantial amounts of training data. A challenge arises because of the impracticality of measuring large amounts of data from users. Data augmentation (DA) has emerged as a promising solution for this challenge. We propose a novel DA method called channel&label-flip DA that involves not only flipping channels but also flipping class labels. This method is based on the neuroscience finding that motor imageries of left- and right-hand movements are roughly symmetrical. The efficiency of the proposed method was evaluated using the OpenBMI dataset, which comprises electroencephalograms collected from 54 participants engaged in left- and right-hand motor imagery tasks. To compare the impact on classifiers, we employed three classical machine learning models utilizing filter bank common spatial pattern features, along with a deep learning-based model that uses raw signal input. As a result, the channel&label-flip DA improved the classification accuracy on average, whereas simple flipping of the channels reduced the classification accuracy compared to the case without DA.

Role of Scalp EEG Brain Connectivity in Motor Imagery Decoding for BCI Applications

Brain Connectivity (BC) features of multichannel EEG have been proposed for Motor Imagery (MI) decoding in Brain-Computer Interface applications, but the advantages of BC features vs. single-channel features are unclear. Here, we consider three BC features, i.e., Phase Locking Value (PLV), Granger Causality, and weighted Phase Lag Index, and investigate the relationship between the most central nodes in BC-based networks and the most influential EEG channels in single-channel classification based on common spatial pattern filtering. Then, we compare the accuracy of MI decoders that use BC features in source vs. sensor space. Applied to the BCI Competition VI Dataset 2a (left- vs. right-hand MI decoding), our study found that PLV in sensor space achieves the highest classification accuracy among BC features and has similar performance compared to single-channel features, while the transition from sensor to source space reduces the average accuracy of BC features. Across all BC measures, the network topology is similar in left- vs. right-hand MI tasks, and the most central nodes in BC-based networks partially overlap with the most influential channels in single-channel classification.

Neurophysiologically Meaningful Motor Imagery EEG Simulation With Applications to Data Augmentation

Motor imagery-based Brain-Computer Interfaces (MI-BCIs) have gained a lot of attention due to their potential usability in neurorehabilitation and neuroprosthetics. However, the accurate recognition of MI patterns in electroencephalography signals (EEG) is hindered by several data-related limitations, which restrict the practical utilization of these systems. Moreover, leveraging deep learning (DL) models for MI decoding is challenged by the difficulty of accessing user-specific MI-EEG data on large scales. Simulated MI-EEG signals can be useful to address these issues, providing well-defined data for the validation of decoding models and serving as a data augmentation approach to improve the training of DL models. While substantial efforts have been dedicated to implementing effective data augmentation strategies and model-based EEG signal generation, the simulation of neurophysiologically plausible EEG-like signals has not yet been exploited in the context of data augmentation. Furthermore, none of the existing approaches have integrated user-specific neurophysiological information during the data generation process. Here, we present PySimMIBCI, a framework for generating realistic MI-EEG signals by integrating neurophysiologically meaningful activity into biophysical forward models. By means of PySimMIBCI, different user capabilities to control an MI-BCI can be simulated and fatigue effects can be included in the generated EEG. Results show that our simulated data closely resemble real data. Moreover, a proposed data augmentation strategy based on our simulated user-specific data significantly outperforms other state-of-the-art augmentation approaches, enhancing DL models performance by up to 15%.

Utilizing Motor-Imagery Brain-Computer Interfaces for the Assessment of Developmental Coordination Disorder in Children

Developmental Coordination Disorder (DCD) is a neurodevelopmental disorder characterized by significant motor difficulties that affect daily life. Current assessment methods primarily focus on behavioral analysis, lacking in neuroscientific metrics for a comprehensive evaluation. This study introduced an electroencephalography-based motor imagery brain-computer interface classification system for evaluating children with DCD. A key of this system was the implementation of entropy-based data screening, which markedly enhanced classification performance. Notably, using mu band power in a support vector machine achieved an accuracy rate of 79.0%. These findings pave the way for developing a tool that could assist professionals in identifying children potentially affected by DCD.

A cross-dataset adaptive domain selection transfer learning framework for motor imagery-based brain-computer interfaces

Objective. In brain-computer interfaces (BCIs) that utilize motor imagery (MI), minimizing calibration time has become increasingly critical for real-world applications. Recently, transfer learning (TL) has been shown to effectively reduce the calibration time in MI-BCIs. However, variations in data distribution among subjects can significantly influence the performance of TL in MI-BCIs.Approach.We propose a cross-dataset adaptive domain selection transfer learning framework that integrates domain selection, data alignment, and an enhanced common spatial pattern (CSP) algorithm. Our approach uses a huge dataset of 109 subjects as the source domain. We begin by identifying non-BCI illiterate subjects from this huge dataset, then determine the source domain subjects most closely aligned with the target subjects using maximum mean discrepancy. After undergoing Euclidean alignment processing, features are extracted by multiple composite CSP. The final classification is carried out using the support vector machine.Main results.Our findings indicate that the proposed technique outperforms existing methods, achieving classification accuracies of 75.05% and 76.82% in two cross-dataset experiments, respectively.Significance.By reducing the need for extensive training data, yet maintaining high accuracy, our method optimizes the practical implementation of MI-BCIs.

An EEG & eye-tracking dataset of ALS patients & healthy people during eye-tracking-based spelling system usage

This research presents a dataset consisting of electroencephalogram and eye tracking recordings obtained from six patients with amyotrophic lateral sclerosis (ALS) in a locked-in state and one hundred seventy healthy individuals. The ALS patients exhibited varying degrees of disease progression, ranging from partial mobility and weakened speech to complete paralysis and loss of speech. Despite these physical impairments, the ALS patients retained good eye function, which allowed them to use a virtual keyboard for communication. Data from ALS patients was recorded multiple times at their homes, while data from healthy individuals was recorded once in a laboratory setting. For each data recording, the experimental design involved nine recording sessions per participant, each corresponding to a common human action or demand. This dataset can serve as a valuable benchmark for several applications, such as improving spelling systems with brain-computer interfaces, investigating motor imagination, exploring motor cortex function, monitoring motor impairment progress in patients undergoing rehabilitation, and studying the effects of ALS on cognitive and motor processes.

Preparatory movement state enhances premovement EEG representations for brain-computer interfaces

Objective. Motor-related brain-computer interface (BCI) have a broad range of applications, with the detection of premovement intentions being a prominent use case. However, the electroencephalography (EEG) features during the premovement phase are not distinctly evident and are susceptible to attentional influences. These limitations impede the enhancement of performance in motor-based BCI. The objective of this study is to establish a premovement BCI encoding paradigm that integrates the preparatory movement state and validates its feasibility in improving the detection of movement intentions.Methods. Two button tasks were designed to induce subjects into a preparation state for two movement intentions (left and right) based on visual guidance, in contrast to spontaneous premovement. The low frequency movement-related cortical potentials (MRCPs) and high frequency event-related desynchronization (ERD) EEG data of 14 subjects were recorded. Extracted features were fused and classified using task related common spatial patterns (CSP) and CSP algorithms. Differences between prepared premovement and spontaneous premovement were compared in terms of time domain, frequency domain, and classification accuracy.Results. In the time domain, MRCPs features reveal that prepared premovement induce lower amplitude and earlier latency on both contralateral and ipsilateral motor cortex compared to spontaneous premovement, with susceptibility to the dominant hand's influence. Frequency domain ERD features indicate that prepared premovement induce lower ERD values bilaterally, and the ERD recovery speed after button press is the fastest. By using the fusion approach, the classification accuracy increased from 78.92% for spontaneous premovement to 83.59% for prepared premovement (p< 0.05). Along with the 4.67% improvement in classification accuracy, the standard deviation decreased by 0.95.Significance. The research findings confirm that incorporating a preparatory state into premovement enhances neural representations related to movement. This encoding enhancement paradigm effectively improves the performance of motor-based BCI. Additionally, this concept has the potential to broaden the range of decodable movement intentions and related information in motor-related BCI.

Motor Imagery EEG Signal Classification Using Distinctive Feature Fusion with Adaptive Structural LASSO

A motor imagery brain-computer interface connects the human brain and computers via electroencephalography (EEG). However, individual differences in the frequency ranges of brain activity during motor imagery tasks pose a challenge, limiting the manual feature extraction for motor imagery classification. To extract features that match specific subjects, we proposed a novel motor imagery classification model using distinctive feature fusion with adaptive structural LASSO. Specifically, we extracted spatial domain features from overlapping and multi-scale sub-bands of EEG signals and mined discriminative features by fusing the task relevance of features with spatial information into the adaptive LASSO-based feature selection. We evaluated the proposed model on public motor imagery EEG datasets, demonstrating that the model has excellent performance. Meanwhile, ablation studies and feature selection visualization of the proposed model further verified the great potential of EEG analysis.

Semi-supervised multi-source transfer learning for cross-subject EEG motor imagery classification

Electroencephalogram (EEG) motor imagery (MI) classification refers to the use of EEG signals to identify and classify subjects' motor imagery activities; this task has received increasing attention with the development of brain-computer interfaces (BCIs). However, the collection of EEG data is usually time-consuming and labor-intensive, which makes it difficult to obtain sufficient labeled data from the new subject to train a new model. Moreover, the EEG signals of different individuals exhibit significant differences, leading to a significant drop in the performance of a model trained on the existing subjects when directly classifying EEG signals acquired from new subjects. Therefore, it is crucial to make full use of the EEG data of the existing subjects and the unlabeled EEG data of the new target subject to improve the MI classification performance achieved for the target subject. This research study proposes a semi-supervised multi-source transfer (SSMT) learning model to address the above problems; the model learns informative and domain-invariant representations to address cross-subject MI-EEG classification tasks. In particular, a dynamic transferred weighting schema is presented to obtain the final predictions by integrating the weighted features derived from multi-source domains. The average accuracies achieved on two publicly available EEG datasets reach 83.57 % and 85.09 % , respectively, validating the effectiveness of the SSMT process. The SSMT process reveals the importance of informative and domain-invariant representations in MI classification tasks, as they make full use of the domain-invariant information acquired from each subject.

Attention-based convolutional neural network with multi-modal temporal information fusion for motor imagery EEG decoding

Convolutional neural network (CNN) has been widely applied in motor imagery (MI)-based brain computer interface (BCI) to decode electroencephalography (EEG) signals. However, due to the limited perceptual field of convolutional kernel, CNN only extracts features from local region without considering long-term dependencies for EEG decoding. Apart from long-term dependencies, multi-modal temporal information is equally important for EEG decoding because it can offer a more comprehensive understanding of the temporal dynamics of neural processes. In this paper, we propose a novel deep learning network that combines CNN with self-attention mechanism to encapsulate multi-modal temporal information and global dependencies. The network first extracts multi-modal temporal information from two distinct perspectives: average and variance. A shared self-attention module is then designed to capture global dependencies along these two feature dimensions. We further design a convolutional encoder to explore the relationship between average-pooled and variance-pooled features and fuse them into more discriminative features. Moreover, a data augmentation method called signal segmentation and recombination is proposed to improve the generalization capability of the proposed network. The experimental results on the BCI Competition IV-2a (BCIC-IV-2a) and BCI Competition IV-2b (BCIC-IV-2b) datasets show that our proposed method outperforms the state-of-the-art methods and achieves 4-class average accuracy of 85.03% on the BCIC-IV-2a dataset. The proposed method implies the effectiveness of multi-modal temporal information fusion in attention-based deep learning networks and provides a new perspective for MI-EEG decoding. The code is available at https://github.com/Ma-Xinzhi/EEG-TransNet.

Alpha band modulation caused by selective attention to music enables EEG classification

Humans are able to pay selective attention to music or speech in the presence of multiple sounds. It has been reported that in the speech domain, selective attention enhances the cross-correlation between the envelope of speech and electroencephalogram (EEG) while also affecting the spatial modulation of the alpha band. However, when multiple music pieces are performed at the same time, it is unclear how selective attention affects neural entrainment and spatial modulation. In this paper, we hypothesized that the entrainment to the attended music differs from that to the unattended music and that spatial modulation in the alpha band occurs in conjunction with attention. We conducted experiments in which we presented musical excerpts to 15 participants, each listening to two excerpts simultaneously but paying attention to one of the two. The results showed that the cross-correlation function between the EEG signal and the envelope of the unattended melody had a more prominent peak than that of the attended melody, contrary to the findings for speech. In addition, the spatial modulation in the alpha band was found with a data-driven approach called the common spatial pattern method. Classification of the EEG signal with a support vector machine identified attended melodies and achieved an accuracy of 100% for 11 of the 15 participants. These results suggest that selective attention to music suppresses entrainment to the melody and that spatial modulation of the alpha band occurs in conjunction with attention. To the best of our knowledge, this is the first report to detect attended music consisting of several types of music notes only with EEG.

FetchEEG: a hybrid approach combining feature extraction and temporal-channel joint attention for EEG-based emotion classification

Objective. Electroencephalogram (EEG) analysis has always been an important tool in neural engineering, and the recognition and classification of human emotions are one of the important tasks in neural engineering. EEG data, obtained from electrodes placed on the scalp, represent a valuable resource of information for brain activity analysis and emotion recognition. Feature extraction methods have shown promising results, but recent trends have shifted toward end-to-end methods based on deep learning. However, these approaches often overlook channel representations, and their complex structures pose certain challenges to model fitting.Approach. To address these challenges, this paper proposes a hybrid approach named FetchEEG that combines feature extraction and temporal-channel joint attention. Leveraging the advantages of both traditional feature extraction and deep learning, the FetchEEG adopts a multi-head self-attention mechanism to extract representations between different time moments and channels simultaneously. The joint representations are then concatenated and classified using fully-connected layers for emotion recognition. The performance of the FetchEEG is verified by comparison experiments on a self-developed dataset and two public datasets.Main results. In both subject-dependent and subject-independent experiments, the FetchEEG demonstrates better performance and stronger generalization ability than the state-of-the-art methods on all datasets. Moreover, the performance of the FetchEEG is analyzed for different sliding window sizes and overlap rates in the feature extraction module. The sensitivity of emotion recognition is investigated for three- and five-frequency-band scenarios.Significance. FetchEEG is a novel hybrid method based on EEG for emotion classification, which combines EEG feature extraction with Transformer neural networks. It has achieved state-of-the-art performance on both self-developed datasets and multiple public datasets, with significantly higher training efficiency compared to end-to-end methods, demonstrating its effectiveness and feasibility.

Machine learning reveals differential effects of depression and anxiety on reward and punishment processing

Recent studies suggest that depression and anxiety are associated with unique aspects of EEG responses to reward and punishment, respectively; also, abnormal responses to punishment in depressed individuals are related to anxiety, the symptoms of which are comorbid with depression. In a non-clinical sample, we aimed to investigate the relationships between reward processing and anxiety, between punishment processing and anxiety, between reward processing and depression, and between punishment processing and depression. Towards this aim, we separated feedback-related brain activity into delta and theta bands to isolate activity that indexes functionally distinct processes. Based on the delta/theta frequency and feedback valence, we then used machine learning (ML) to classify individuals with high severity of depressive symptoms and individuals with high severity of anxiety symptoms versus controls. The significant difference between the depression and control groups was driven mainly by delta activity; there were no differences between reward- and punishment-theta activities. The high severity of anxiety symptoms was marginally more strongly associated with the punishment- than the reward-theta feedback processing. The findings provide new insights into the differences in the impacts of anxiety and depression on reward and punishment processing; our study shows the utility of ML in testing brain-behavior hypotheses and emphasizes the joint effect of theta-RewP/FRN and delta frequency on feedback-related brain activity.

Classification of Motor Imagery EEG signals using high resolution time-frequency representations and convolutional neural network

A Motor Imagery (MI) based Brain Computer Interface (BCI) system aims to provide neuro-rehabilitation for the motor disabled people and patients with brain injuries (e.g., stroke patients) etc. The aim of this work is to classify the left and right hand MI tasks by utilizing the occurrence of event related desynchronization and synchronization (ERD\ERS) in the Electroencephalogram (EEG) during these tasks. This study proposes to use a set of Complex Morlet Wavelets (CMW) having frequency dependent widths to generate high-resolution time-frequency representations (TFR) of the MI EEG signals present in the channels C3 and C4. A novel method for the selection of the value of number of cycles relative to the center frequency of the CMW is studied here for extracting the MI task features. The generated TFRs are given as input to a Convolutional neural network (CNN) for classifying them into left or right hand MI tasks. The proposed framework attains a classification accuracy of 82.2% on the BCI Competition IV dataset 2a, showing that the TFRs generated in this work give a higher classification accuracy than the baseline methods and other existing algorithms.

EEG rhythm separation and time-frequency analysis of fast multivariate empirical mode decomposition for motor imagery BCI

Motor imagery electroencephalogram (EEG) is widely employed in brain-computer interface (BCI) systems. As a time-frequency analysis method for nonlinear and non-stationary signals, multivariate empirical mode decomposition (MEMD) and its noise-assisted version (NA-MEMD) has been widely used in the preprocessing step of BCI systems for separating EEG rhythms corresponding to specific brain activities. However, when applied to multichannel EEG signals, MEMD or NA-MEMD often demonstrate low robustness to noise and high computational complexity. To address these issues, we have explored the advantages of our recently proposed fast multivariate empirical mode decomposition (FMEMD) and its noise-assisted version (NA-FMEMD) for analyzing motor imagery data. We emphasize that FMEMD enables a more accurate estimation of EEG frequency information and exhibits a more noise-robust decomposition performance with improved computational efficiency. Comparative analysis with MEMD on simulation data and real-world EEG validates the above assertions. The joint average frequency measure is employed to automatically select intrinsic mode functions that correspond to specific frequency bands. Thus, FMEMD-based classification architecture is proposed. Using FMEMD as a preprocessing algorithm instead of MEMD can improve the classification accuracy by 2.3% on the BCI Competition IV dataset. On the Physiobank Motor/Mental Imagery dataset and BCI Competition IV Dataset 2a, FMEMD-based architecture also attained a comparable performance to complex algorithms. The results indicate that FMEMD proficiently extracts feature information from small benchmark datasets while mitigating dimensionality constraints resulting from computational complexity. Hence, FMEMD or NA-FMEMD can be a powerful time-frequency preprocessing method for BCI.

Multilayer network-based channel selection for motor imagery brain-computer interface

Objective. The number of electrode channels in a motor imagery-based brain-computer interface (MI-BCI) system influences not only its decoding performance, but also its convenience for use in applications. Although many channel selection methods have been proposed in the literature, they are usually based on the univariate features of a single channel. This leads to a loss of the interaction between channels and the exchange of information between networks operating at different frequency bands.Approach. We integrate brain networks containing four frequency bands into a multilayer network framework and propose a multilayer network-based channel selection (MNCS) method for MI-BCI systems. A graph learning-based method is used to estimate the multilayer network from electroencephalogram (EEG) data that are filtered by multiple frequency bands. The multilayer participation coefficient of the multilayer network is then computed to select EEG channels that do not contain redundant information. Furthermore, the common spatial pattern (CSP) method is used to extract effective features. Finally, a support vector machine classifier with a linear kernel is trained to accurately identify MI tasks.Main results. We used three publicly available datasets from the BCI Competition containing data on 12 healthy subjects and one dataset containing data on 15 stroke patients to validate the effectiveness of our proposed method. The results showed that the proposed MNCS method outperforms all channels (85.8% vs. 93.1%, 84.4% vs. 89.0%, 71.7% vs. 79.4%, and 72.7% vs. 84.0%). Moreover, it achieved significantly higher decoding accuracies on MI-BCI systems than state-of-the-art methods (pairedt-tests,p< 0.05).Significance. The experimental results showed that the proposed MNCS method can select appropriate channels to improve the decoding performance as well as the convenience of the application of MI-BCI systems.