EEG datasets for motor imagery brain-computer interface

Effects of Different Preprocessing Pipelines on Motor Imagery-Based Brain-Computer Interfaces

In recent years, brain-computer interfaces (BCIs) leveraging electroencephalography (EEG) signals for the control of external devices have garnered increasing attention. The information transfer rate of BCI has been significantly improved by a lot of cutting-edge methods. The exploration of effective preprocessing in brain-computer interfaces, particularly in terms of identifying suitable preprocessing methods and determining the optimal sequence for their application, remains an area ripe for further investigation. To address this gap, this study explores a range of preprocessing techniques, including but not limited to independent component analysis, surface Laplacian, bandpass filtering, and baseline correction, examining their potential contributions and synergies in the context of BCI applications. In this extensive research, a variety of preprocessing pipelines were rigorously tested across four EEG data sets, all of which were pertinent to motor imagery-based BCIs. These tests incorporated five EEG machine learning models, working in tandem with the preprocessing methods discussed earlier. The study's results highlighted that baseline correction and bandpass filtering consistently provided the most beneficial preprocessing effects. From the perspective of online deployment, after testing and time complexity analysis, this study recommends baseline correction, bandpass filtering and surface Laplace as more suitable for online implementation. An interesting revelation of the study was the enhanced effectiveness of the surface Laplacian algorithm when used alongside algorithms that focus on spatial information. Using appropriate processing algorithms, we can even achieve results (92.91% and 88.11%) that exceed the SOTA feature extraction methods in some cases. Such findings are instrumental in offering critical insights for the selection of effective preprocessing pipelines in EEG signal decoding. This, in turn, contributes to the advancement and refinement of brain-computer interface technologies.

Enhancing motor imagery EEG classification with a Riemannian geometry-based spatial filtering (RSF) method

Motor imagery (MI) refers to the mental simulation of movements without physical execution, and it can be captured using electroencephalography (EEG). This area has garnered significant research interest due to its substantial potential in brain-computer interface (BCI) applications, especially for individuals with physical disabilities. However, accurate classification of MI EEG signals remains a major challenge due to their non-stationary nature, low signal-to-noise ratio, and sensitivity to both external and physiological noise. Traditional classification methods, such as common spatial pattern (CSP), often assume that the data is stationary and Gaussian, which limits their applicability in real-world scenarios where these assumptions do not hold. These challenges highlight the need for more robust methods to improve classification accuracy in MI-BCI systems. To address these issues, this study introduces a Riemannian geometry-based spatial filtering (RSF) method that projects EEG signals into a lower-dimensional subspace, maximizing the Riemannian distance between covariance matrices from different classes. By leveraging the inherent geometric properties of EEG data, RSF enhances the discriminative power of the features while maintaining robustness against noise. The performance of RSF was evaluated in combination with ten commonly used MI decoding algorithms, including CSP with linear discriminant analysis (CSP-LDA), Filter Bank CSP (FBCSP), Minimum Distance to Riemannian Mean (MDM), Tangent Space Mapping (TSM), EEGNet, ShallowConvNet (sCNN), DeepConvNet (dCNN), FBCNet, Graph-CSPNet, and LMDA-Net, using six publicly available MI-BCI datasets. The results demonstrate that RSF significantly improves classification accuracy and reduces computational time, particularly for deep learning models with high computational complexity. These findings underscore the potential of RSF as an effective spatial filtering approach for MI EEG classification, providing new insights and opportunities for the development of robust MI-BCI systems. The code for this research is available at https://github.com/PLC-TJU/RSF.

The Riemannian Means Field Classifier for EEG-Based BCI Data

: A substantial amount of research has demonstrated the robustness and accuracy of the Riemannian minimum distance to mean (MDM) classifier for all kinds of EEG-based brain-computer interfaces (BCIs). This classifier is simple, fully deterministic, robust to noise, computationally efficient, and prone to transfer learning. Its training is very simple, requiring just the computation of a geometric mean of a symmetric positive-definite (SPD) matrix per class. We propose an improvement of the MDM involving a number of power means of SPD matrices instead of the sole geometric mean. By the analysis of 20 public databases, 10 for the motor-imagery BCI paradigm and 10 for the P300 BCI paradigm, comprising 587 individuals in total, we show that the proposed classifier clearly outperforms the MDM, approaching the state-of-the art in terms of performance while retaining the simplicity and the deterministic behavior. In order to promote reproducible research, our code will be released as open source.

Flexible Patched Brain Transformer model for EEG decoding

Decoding the human brain using non-invasive methods is a significant challenge. This study aims to enhance electroencephalography (EEG) decoding by developing of machine learning methods. Specifically, we propose the novel, attention-based Patched Brain Transformer model to achieve this goal. The model exhibits flexibility regarding the number of EEG channels and recording duration, enabling effective pre-training across diverse datasets. We investigate the effect of data augmentation methods and pre-training on the training process. To gain insights into the training behavior, we incorporate an inspection of the architecture. We compare our model with state-of-the-art models and demonstrate superior performance using only a fraction of the parameters. The results are achieved with supervised pre-training, coupled with time shifts as data augmentation for multi-participant classification on motor imagery datasets.

Motor imagery EEG microstates are influenced by alpha power

Electroencephalogram (EEG) microstates are pivotal in understanding brain dynamics, reflecting transitions between global states. These parameters undergo selective inhibition within cortical areas, modulated by alpha oscillations. This study investigates how alpha band power influences microstate parameters across various task conditions, including resting state, actual motor execution, and imagined motor tasks. By comparing these three conditions, we aim to elucidate the distinct effects of alpha power on microstate dynamics, as each condition represents a unique pattern of brain activity. Motor imagery (MI) induces event-related desynchronization/synchronization, modulating Mu (alpha) and Beta rhythms in sensorimotor areas. However, the relationship between MI-EEG microstates and alpha power remains unclear. Our results show that alpha power was highest in resting state, followed by imagined motion, and lowest during actual motion. As alpha power increased, microstate A parameters in resting state (occurrence, coverage) decreased, while those in actual motion increased. Additionally, microstate B parameters rose with alpha power in resting state but decreased during imagined motion. Notably, alpha power correlated more strongly with microstate parameters in task states than in resting state. In addition, alpha, theta, and beta powers during task performance were negatively correlated with the duration of microstates A, B, and C, while being positively correlated with the occurrence of microstates A, B, C, and D. These findings suggest that alpha power influences microstate parameters differently depending on the brain, underscoring the significance of inter-band interactions in shaping microstate dynamics.

Effect of natural and synthetic noise data augmentation on physical action classification by brain-computer interface and deep learning

Analysis of electroencephalography (EEG) signals gathered by brain-computer interface (BCI) recently demonstrated that deep neural networks (DNNs) can be effectively used for investigation of time sequences for physical actions (PA) classification. In this study, the relatively simple DNN with fully connected network (FCN) components and convolutional neural network (CNN) components was considered to classify finger-palm-hand manipulations each from the grasp-and-lift (GAL) dataset. The main aim of this study was to imitate and investigate environmental influence by the proposed noise data augmentation (NDA) of two kinds: (i) natural NDA by inclusion of noise EEG data from neighboring regions by increasing the sampling size N and the different offset values for sample labeling and (ii) synthetic NDA by adding the generated Gaussian noise. The natural NDA by increasing N leads to the higher micro and macro area under the curve (AUC) for receiver operating curve values for the bigger N values than usage of synthetic NDA. The detrended fluctuation analysis (DFA) was applied to investigate the fluctuation properties and calculate the correspondent Hurst exponents H for the quantitative characterization of the fluctuation variability. H values for the low time window scales (< 2 s) are higher in comparison with ones for the bigger time window scales. For example, H more than 2-3 times higher for some PAs, i.e., it means that the shorter EEG fragments (< 2 s) demonstrate the scaling behavior of the higher complexity than the longer fragments. As far as these results were obtained by the relatively small DNN with the low resource requirements, this approach can be promising for porting such models to Edge Computing infrastructures on devices with the very limited computational resources.

Lower limb motor imagery EEG dataset based on the multi-paradigm and longitudinal-training of stroke patients

Motor dysfunction is one of the most significant sequelae of stroke, with lower limb impairment being a major concern for stroke patients. Motor imagery (MI) technology based on brain-computer interface (BCI) offers promising rehabilitation potential for stroke patients by activating motor-related brain areas. However, developing a robust BCI-MI system and uncovering the underlying mechanisms of neural plasticity during stroke recovery through such systems requires large-scale datasets. These datasets are particularly needed for accurate lower limb MI in stroke patients and for longitudinal data reflecting the rehabilitation process. This study addresses this gap by collecting EEG data from 27 stroke patients, covering two enhanced paradigms and three different time points. The dataset includes raw EEG signals, preprocessed data, and patient information. An initial analysis using CSP-SVM on the dataset yielded an average classification accuracy of 80.50%. We anticipate that this dataset will facilitate research into brain neuroplasticity in stroke patients, aid in the development of decoding algorithms for lower limb stroke, and contribute to the establishment of comprehensive stroke rehabilitation systems.

Geometric neural network based on phase space for BCI-EEG decoding

Objective.The integration of Deep Learning (DL) algorithms on brain signal analysis is still in its nascent stages compared to their success in fields like Computer Vision. This is particularly true for Brain-computer interface (BCI), where the brain activity is decoded to control external devices without requiring muscle control. Electroencephalography is a widely adopted choice for designing BCI systems due to its non-invasive and cost-effective nature and excellent temporal resolution. Still, it comes at the expense of limited training data, poor signal-to-noise, and a large variability across and within-subject recordings. Finally, setting up a BCI system with many electrodes takes a long time, hindering the widespread adoption of reliable DL architectures in BCIs outside research laboratories. To improve adoption, we need to improve user comfort using, for instance, reliable algorithms that operate with few electrodes.Approach.Our research aims to develop a DL algorithm that delivers effective results with a limited number of electrodes. Taking advantage of the Augmented Covariance Method and the framework of SPDNet, we propose the Phase-SPDNet architecture and analyze its performance and the interpretability of the results. The evaluation is conducted on 5-fold cross-validation, using only three electrodes positioned above the Motor Cortex. The methodology was tested on nearly 100 subjects from several open-source datasets using the Mother Of All BCI Benchmark framework.Main results.The results of our Phase-SPDNet demonstrate that the augmented approach combined with the SPDNet significantly outperforms all the current state-of-the-art DL architecture in MI decoding.Significance.This new architecture is explainable and with a low number of trainable parameters.

Assessing brain-muscle networks during motor imagery to detect covert command-following

BACKGROUND

In this study, we evaluated the potential of a network approach to electromyography and electroencephalography recordings to detect covert command-following in healthy participants. The motivation underlying this study was the development of a diagnostic tool that can be applied in common clinical settings to detect awareness in patients that are unable to convey explicit motor or verbal responses, such as patients that suffer from disorders of consciousness (DoC).

METHODS

We examined the brain and muscle response during movement and imagined movement of simple motor tasks, as well as during resting state. Brain-muscle networks were obtained using non-negative matrix factorization (NMF) of the coherence spectra for all the channel pairs. For the 15/38 participants who showed motor imagery, as indexed by common spatial filters and linear discriminant analysis, we contrasted the configuration of the networks during imagined movement and resting state at the group level, and subject-level classifiers were implemented using as features the weights of the NMF together with trial-wise power modulations and heart response to classify resting state from motor imagery.

RESULTS

Kinesthetic motor imagery produced decreases in the mu-beta band compared to resting state, and a small correlation was found between mu-beta power and the kinesthetic imagery scores of the Movement Imagery Questionnaire-Revised Second version. The full-feature classifiers successfully distinguished between motor imagery and resting state for all participants, and brain-muscle functional networks did not contribute to the overall classification. Nevertheless, heart activity and cortical power were crucial to detect when a participant was mentally rehearsing a movement.

CONCLUSIONS

Our work highlights the importance of combining EEG and peripheral measurements to detect command-following, which could be important for improving the detection of covert responses consistent with volition in unresponsive patients.

A Survey of Few-Shot Learning for Biomedical Time Series

Advancements in wearable sensor technologies and the digitization of medical records have contributed to the unprecedented ubiquity of biomedical time series data. Data-driven models have tremendous potential to assist clinical diagnosis and improve patient care by improving long-term monitoring capabilities, facilitating early disease detection and intervention, as well as promoting personalized healthcare delivery. However, accessing extensively labeled datasets to train data-hungry deep learning models encounters many barriers, such as long-tail distribution of rare diseases, cost of annotation, privacy and security concerns, data-sharing regulations, and ethical considerations. An emerging approach to overcome the scarcity of labeled data is to augment AI methods with human-like capabilities to leverage past experiences to learn new tasks with limited examples, called few-shot learning. This survey provides a comprehensive review and comparison of few-shot learning methods for biomedical time series applications. The clinical benefits and limitations of such methods are discussed in relation to traditional data-driven approaches. This paper aims to provide insights into the current landscape of few-shot learning for biomedical time series and its implications for future research and applications.

The 'Sandwich' meta-framework for architecture agnostic deep privacy-preserving transfer learning for non-invasive brainwave decoding

Objective. Machine learning has enhanced the performance of decoding signals indicating human behaviour. Electroencephalography (EEG) brainwave decoding, as an exemplar indicating neural activity and human thoughts non-invasively, has been helpful in neural activity analysis and aiding paralysed patients via brain-computer interfaces. However, training machine learning algorithms on EEG encounters two primary challenges: variability across data sets and privacy concerns using data from individuals and data centres. Our objective is to address these challenges by integrating transfer learning for data variability and federated learning for data privacy into a unified approach.Approach. We introduce the 'Sandwich' as a novel deep privacy-preserving meta-framework combining transfer learning and federated learning. The 'Sandwich' framework comprises three components: federated networks (first layers) that handle data set differences at the input level, a shared network (middle layer) learning common rules and applying transfer learning techniques, and individual classifiers (final layers) for specific brain tasks of each data set. This structure enables the central network (central server) to benefit from multiple data sets, while local branches (local servers) maintain data and label privacy.Main results. We evaluated the 'Sandwich' meta-architecture in various configurations using the BEETL motor imagery challenge, a benchmark for heterogeneous EEG data sets. Compared with baseline models likeShallow ConvNetandEEGInception, our 'Sandwich' implementations showed superior performance. The best-performing model, the Inception SanDwich with deep set alignment (Inception-SD-Deepset), exceeded baseline methods by 9%.Significance. The 'Sandwich' framework demonstrates advancements in federated deep transfer learning for diverse tasks and data sets. It outperforms conventional deep learning methods, showcasing the potential for effective use of larger, heterogeneous data sets with enhanced privacy. In addition, through its diverse implementations with various backbone architectures and transfer learning approaches, the 'Sandwich' framework shows the potential as a model-agnostic meta-framework for decoding time series data like EEG, suggesting a direction towards large-scale brainwave decoding by combining deep transfer learning with privacy-preserving federated learning.

DHCT-GAN: Improving EEG Signal Quality with a Dual-Branch Hybrid CNN-Transformer Network

Electroencephalogram (EEG) signals are important bioelectrical signals widely used in brain activity studies, cognitive mechanism research, and the diagnosis and treatment of neurological disorders. However, EEG signals are often influenced by various physiological artifacts, which can significantly affect data analysis and diagnosis. Recently, deep learning-based EEG denoising methods have exhibited unique advantages over traditional methods. Most existing methods mainly focus on identifying the characteristics of clean EEG signals to facilitate artifact removal; however, the potential to integrate cross-disciplinary knowledge, such as insights from artifact research, remains an area that requires further exploration. In this study, we developed DHCT-GAN, a new EEG denoising model, using a dual-branch hybrid network architecture. This model independently learns features from both clean EEG signals and artifact signals, then fuses this information through an adaptive gating network to generate denoised EEG signals that accurately preserve EEG signal features while effectively removing artifacts. We evaluated DHCT-GAN's performance through waveform analysis, power spectral density (PSD) analysis, and six performance metrics. The results demonstrate that DHCT-GAN significantly outperforms recent state-of-the-art networks in removing various artifacts. Furthermore, ablation experiments revealed that the hybrid model surpasses single-branch models in artifact removal, underscoring the crucial role of artifact knowledge constraints in improving denoising effectiveness.

A Novel Recognition and Classification Approach for Motor Imagery Based on Spatio-Temporal Features

Motor imagery, as a paradigm of brain-computer interface, holds vast potential in the field of medical rehabilitation. Addressing the challenges posed by the non-stationarity and low signal-to-noise ratio of EEG signals, the effective extraction of features from motor imagery signals for accurate recognition stands as a key focus in motor imagery brain-computer interface technology. This paper proposes a motor imagery EEG signal classification model that combines functional brain networks with graph convolutional networks. First, functional brain networks are constructed using different brain functional connectivity metrics, and graph theory features are calculated to deeply analyze the characteristics of brain networks under different motor tasks. Then, the constructed functional brain networks are combined with graph convolutional networks for the classification and recognition of motor imagery tasks. The analysis based on brain functional connectivity reveals that the functional connectivity strength during the both fists task is significantly higher than that of other motor imagery tasks, and the functional connectivity strength during actual movement is generally superior to that of motor imagery tasks. In experiments conducted on the Physionet public dataset, the proposed model achieved a classification accuracy of 88.39% under multi-subject conditions, significantly outperforming traditional methods. Under single-subject conditions, the model effectively addressed the issue of individual variability, achieving an average classification accuracy of 99.31%. These results indicate that the proposed model not only exhibits excellent performance in the classification of motor imagery tasks but also provides new insights into the functional connectivity characteristics of different motor tasks and their corresponding brain regions.

Supervised Contrastive Learning-Based Domain Generalization Network for Cross-Subject Motor Decoding

Developing an electroencephalogram (EEG)-based motor imagery and motor execution (MI/ME) decoding system that is both highly accurate and calibration-free for cross-subject applications remains challenging due to domain shift problem inherent in such scenario. Recent research has increasingly embraced transfer learning strategies, especially domain adaptation techniques. However, domain adaptation becomes impractical when the target subject data is either difficult to obtain or unknown. To address this issue, we propose a supervised contrastive learning-based domain generalization network (SCLDGN) for cross-subject MI/ME decoding. Firstly, the feature encoder is purposefully designed to learn the EEG discriminative feature representations. Secondly, the domain alignment based on deep correlation alignment constrains the representations distance across various domains to learn domain-invariant features. In addition, the class regularization block is proposed, where the supervised contrastive learning with domain-agnostic mixup is established to learn the class-relevant features and achieve class-level alignment. Finally, in the latent space, clusters of domain-agnostic representations from the same class are mapped closer together. Consequently, SCLDGN is capable of learning domain-invariant and class-relevant discriminative representations, which are essential for effective cross-subject decoding. Extensive experiments conducted on six MI/ME datasets demonstrate the effectiveness of the proposed method in comparison with other state-of-the-art approaches. Furthermore, ablation study and visualization analyses explain the generalization mechanism of the proposed method and also show neurophysiologically meaningful patterns related to MI/ME.

Motion Cognitive Decoding of Cross-Subject Motor Imagery Guided on Different Visual Stimulus Materials

BACKGROUND

Motor imagery (MI) plays an important role in brain-computer interfaces, especially in evoking event-related desynchronization and synchronization (ERD/S) rhythms in electroencephalogram (EEG) signals. However, the procedure for performing a MI task for a single subject is subjective, making it difficult to determine the actual situation of an individual's MI task and resulting in significant individual EEG response variations during motion cognitive decoding.

METHODS

To explore this issue, we designed three visual stimuli (arrow, human, and robot), each of which was used to present three MI tasks (left arm, right arm, and feet), and evaluated differences in brain response in terms of ERD/S rhythms. To compare subject-specific variations of different visual stimuli, a novel cross-subject MI-EEG classification method was proposed for the three visual stimuli. The proposed method employed a covariance matrix centroid alignment for preprocessing of EEG samples, followed by a model agnostic meta-learning method for cross-subject MI-EEG classification.

RESULTS AND CONCLUSION

The experimental results showed that robot stimulus materials were better than arrow or human stimulus materials, with an optimal cross-subject motion cognitive decoding accuracy of 79.04%. Moreover, the proposed method produced robust classification of cross-subject MI-EEG signal decoding, showing superior results to conventional methods on collected EEG signals.

Improving classification performance of motor imagery BCI through EEG data augmentation with conditional generative adversarial networks

In brain-computer interface (BCI), building accurate electroencephalogram (EEG) classifiers for specific mental tasks is critical for BCI performance. The classifiers are developed by machine learning (ML) and deep learning (DL) techniques, requiring a large dataset for training to build reliable and accurate models. However, collecting large enough EEG datasets is difficult due to intra-/inter-subject variabilities and experimental costs. This leads to the data scarcity problem, which causes overfitting issues to training samples, resulting in reducing generalization performance. To solve the EEG data scarcity problem and improve the performance of the EEG classifiers, we propose a novel EEG data augmentation (DA) framework using conditional generative adversarial networks (cGANs). An experimental study is implemented with two public EEG datasets, including motor imagery (MI) tasks (BCI competition IV IIa and III IVa), to validate the effectiveness of the proposed EEG DA method for the EEG classifiers. To evaluate the proposed cGAN-based DA method, we tested eight EEG classifiers for the experiment, including traditional MLs and state-of-the-art DLs with three existing EEG DA methods. Experimental results showed that most DA methods with proper DA proportion in the training dataset had higher classification performances than without DA. Moreover, applying the proposed DA method showed superior classification performance improvement than the other DA methods. This shows that the proposed method is a promising EEG DA method for enhancing the performances of the EEG classifiers in MI-based BCIs.

Attention Induced Dual Convolutional-Capsule Network (AIDC-CN): A deep learning framework for motor imagery classification

In recent times, Electroencephalography (EEG)-based motor imagery (MI) decoding has garnered significant attention due to its extensive applicability in healthcare, including areas such as assistive robotics and rehabilitation engineering. Nevertheless, the decoding of EEG signals presents considerable challenges owing to their inherent complexity, non-stationary characteristics, and low signal-to-noise ratio. Notably, deep learning-based classifiers have emerged as a prominent focus for addressing the EEG signal decoding process. This study introduces a novel deep learning classifier named the Attention Induced Dual Convolutional-Capsule Network (AIDC-CN) with the specific aim of accurately categorizing various motor imagination class labels. To enhance the classifier's performance, a dual feature extraction approach leveraging spectrogram and brain connectivity networks has been employed, diversifying the feature set in the classification task. The main highlights of the proposed AIDC-CN classifier includes the introduction of a dual convolution layer to handle the brain connectivity and spectrogram features, addition of a novel self-attention module (SAM) to accentuate the relevant parts of the convolved spectrogram features, introduction of a new cross-attention module (CAM) to refine the outputs obtained from the dual convolution layers and incorporation of a Gaussian Error Linear Unit (GELU) based dynamic routing algorithm to strengthen the coupling among the primary and secondary capsule layers. Performance analysis undertaken on four public data sets depict the superior performance of the proposed model with respect to the state-of-the-art techniques. The code for this model is available at https://github.com/RiteshSurChowdhury/AIDC-CN.

Deep Autoencoder for Real-Time Single-Channel EEG Cleaning and Its Smartphone Implementation Using TensorFlow Lite With Hardware/Software Acceleration

OBJECTIVE

To remove signal contamination in electroencephalogram (EEG) traces coming from ocular, motion, and muscular artifacts which degrade signal quality. To do this in real-time, with low computational overhead, on a mobile platform in a channel count independent manner to enable portable Brain-Computer Interface (BCI) applications.

METHODS

We propose a Deep AutoEncoder (DAE) neural network for single-channel EEG artifact removal, and implement it on a smartphone via TensorFlow Lite. Delegate based acceleration is employed to allow real-time, low computational resource operation. Artifact removal performance is quantified by comparing corrupted and ground-truth clean EEG data from public datasets for a range of artifact types. The on-phone computational resources required are also measured when processing pre-saved data.

RESULTS

DAE cleaned EEG shows high correlations with ground-truth clean EEG, with average Correlation Coefficients of 0.96, 0.85, 0.70 and 0.79 for clean EEG reconstruction, and EOG, motion, and EMG artifact removal respectively. On-smartphone tests show the model processes a 4 s EEG window within 5 ms, substantially outperforming a comparison FastICA artifact removal algorithm.

CONCLUSION

The proposed DAE model shows effectiveness in single-channel EEG artifact removal. This is the first demonstration of a low-computational resource deep learning model for mobile EEG in smartphones with hardware/software acceleration.

SIGNIFICANCE

This work enables portable BCIs which require low latency real-time artifact removal, and potentially operation with a small number of EEG channels for wearability. It makes use of the artificial intelligence accelerators found in modern smartphones to improve computational performance compared to previous artifact removal approaches.

Dual Attention Relation Network With Fine-Tuning for Few-Shot EEG Motor Imagery Classification

Recently, motor imagery (MI) electroencephalography (EEG) classification techniques using deep learning have shown improved performance over conventional techniques. However, improving the classification accuracy on unseen subjects is still challenging due to intersubject variability, scarcity of labeled unseen subject data, and low signal-to-noise ratio (SNR). In this context, we propose a novel two-way few-shot network able to efficiently learn how to learn representative features of unseen subject categories and classify them with limited MI EEG data. The pipeline includes an embedding module that learns feature representations from a set of signals, a temporal-attention module to emphasize important temporal features, an aggregation-attention module for key support signal discovery, and a relation module for final classification based on relation scores between a support set and a query signal. In addition to the unified learning of feature similarity and a few-shot classifier, our method can emphasize informative features in support data relevant to the query, which generalizes better on unseen subjects. Furthermore, we propose to fine-tune the model before testing by arbitrarily sampling a query signal from the provided support set to adapt to the distribution of the unseen subject. We evaluate our proposed method with three different embedding modules on cross-subject and cross-dataset classification tasks using brain-computer interface (BCI) competition IV 2a, 2b, and GIST datasets. Extensive experiments show that our model significantly improves over the baselines and outperforms existing few-shot approaches.

Differences in Mu rhythm when seeing grasping/motor actions in a real context versus on screens

Mu rhythm (∼8-12 Hz) in the somatosensory cortex has traditionally been linked with doing and seeing motor activities. Here, we aimed to learn how the medium (physical or screened) in which motor actions are seen could impact on that specific brain rhythm. To do so, we presented to 40 participants the very same narrative content both in a one-shot movie with no cuts and in a real theatrical performance. We recorded subjects' brain activities with electroencephalographic (EEG) procedures, and analyzed Mu rhythm present in left (C3) and right (C4) somatosensory areas in relation to the 24 motor activities included in each visual stimulus (screen vs. reality) (24 motor and grasping actions x 40 participants x 2 conditions = 1920 trials). We found lower Mu spectral power in the somatosensory area after the onset of the motor actions in real performance than on-screened content, more pronounced in the left hemisphere. In our results, the sensorimotor Mu-ERD (event-related desynchronization) was stronger during the real-world observation compared to screen observation. This could be relevant in research areas where the somatosensory cortex is important, such as online learning, virtual reality, or brain-computer interfaces.

Leveraging temporal dependency for cross-subject-MI BCIs by contrastive learning and self-attention

Brain-computer interfaces (BCIs) built based on motor imagery paradigm have found extensive utilization in motor rehabilitation and the control of assistive applications. However, traditional MI-BCI systems often exhibit suboptimal classification performance and require significant time for new users to collect subject-specific training data. This limitation diminishes the user-friendliness of BCIs and presents significant challenges in developing effective subject-independent models. In response to these challenges, we propose a novel subject-independent framework for learning temporal dependency for motor imagery BCIs by Contrastive Learning and Self-attention (CLS). In CLS model, we incorporate self-attention mechanism and supervised contrastive learning into a deep neural network to extract important information from electroencephalography (EEG) signals as features. We evaluate the CLS model using two large public datasets encompassing numerous subjects in a subject-independent experiment condition. The results demonstrate that CLS outperforms six baseline algorithms, achieving a mean classification accuracy improvement of 1.3 % and 4.71 % than the best algorithm on the Giga dataset and OpenBMI dataset, respectively. Our findings demonstrate that CLS can effectively learn invariant discriminative features from training data obtained from non-target subjects, thus showcasing its potential for building models for new users without the need for calibration.

Significance of EEG-electrode combinations while calculating filters with common spatial patterns

Objective

Common spatial pattern (CSP) is a common filter technique used for pre-processing of electroencephalography (EEG) signals for imaginary movement classification tasks. It is crucial to reduce the amount of features especially in cases where few data is available. Therefore, different approaches to reduce the amount of electrodes used for CSP calculation are tried in this research.

Methods

Freely available EEG datasets are used for the evaluation. To evaluate the approaches a simple classification pipeline consisting mainly of the CSP calculation and linear discriminant analysis for classification is used. A baseline over all electrodes is calculated and compared against the results of the approaches.

Results

The most promising approach is to use the ability of CSP to provide information about the origin of the created filter. An algorithm that extracts the important electrodes from the CSP utilizing these information is proposed.The results show that using subject specific electrode positions has a positive impact on accuracy for the classification task. Further, it is shown that good performing electrode combinations in one session are not necessarily good performing electrodes in another session of the same subject. In addition to the combinations calculated using the developed algorithm, 26 additional electrode combinations are proposed. These can be taken into account when selecting well-performing electrode combinations. In this research we could achieve an accuracy improvement of over 10%.

Conclusions

Carefully selecting the correct electrode combination can improve accuracy for classifying an imaginary movement task.

A Strong and Simple Deep Learning Baseline for BCI Motor Imagery Decoding

We propose EEG-SimpleConv, a straightforward 1D convolutional neural network for Motor Imagery decoding in BCI. Our main motivation is to propose a simple and performing baseline that achieves high classification accuracy, using only standard ingredients from the literature, to serve as a standard for comparison. The proposed architecture is composed of standard layers, including 1D convolutions, batch normalisations, ReLU activation functions and pooling functions. EEG-SimpleConv architecture is accompanied by a straightforward and tailored training routine, which is subjected to an extensive ablation study to quantify the influence of its components. We evaluate its performance on four EEG Motor Imagery datasets, including simulated online setups, and compare it to recent Deep Learning and Machine Learning approaches. EEG-SimpleConv is at least as good or far more efficient than other approaches, showing strong knowledge-transfer capabilities across subjects, at the cost of a low inference time. We believe that using standard components and ingredients can significantly help the adoption of Deep Learning approaches for BCI. We make the code of the models and the experiments accessible.

Connectivity study on resting-state EEG between motor imagery BCI-literate and BCI-illiterate groups

Objective.Although motor imagery-based brain-computer interface (MI-BCI) holds significant potential, its practical application faces challenges such as BCI-illiteracy. To mitigate this issue, researchers have attempted to predict BCI-illiteracy by using the resting state, as this was found to be associated with BCI performance. As connectivity's significance in neuroscience has grown, BCI researchers have applied connectivity to it. However, the issues of connectivity have not been considered fully. First, although various connectivity metrics exist, only some have been used to predict BCI-illiteracy. This is problematic because each metric has a distinct hypothesis and perspective to estimate connectivity, resulting in different outcomes according to the metric. Second, the frequency range affects the connectivity estimation. In addition, it is still unknown whether each metric has its own optimal frequency range. Third, the way that estimating connectivity may vary depending upon the dataset has not been investigated. Meanwhile, we still do not know a great deal about how the resting state electroencephalography (EEG) network differs between BCI-literacy and -illiteracy.Approach.To address the issues above, we analyzed three large public EEG datasets using three functional connectivity and three effective connectivity metrics by employing diverse graph theory measures. Our analysis revealed that the appropriate frequency range to predict BCI-illiteracy varies depending upon the metric. The alpha range was found to be suitable for the metrics of the frequency domain, while alpha + theta were found to be appropriate for multivariate Granger causality. The difference in network efficiency between BCI-literate and -illiterate groups was constant regardless of the metrics and datasets used. Although we observed that BCI-literacy had stronger connectivity, no other significant constructional differences were found.Significance.Based upon our findings, we predicted MI-BCI performance for the entire dataset. We discovered that combining several graph features could improve the prediction's accuracy.

Riemannian Locality Preserving Method for Transfer Learning With Applications on Brain-Computer Interface

Brain-computer interfaces (BCIs) have been widely focused and extensively studied in recent years for their huge prospect of medical rehabilitation and commercial applications. Transfer learning exploits the information in the source domain and applies in another different but related domain (target domain), and is therefore introduced into the BCIs to figure out the inter-subject variances of electroencephalography (EEG) signals. In this article, a novel transfer learning method is proposed to preserve the Riemannian locality of data structure in both the source and target domains and simultaneously realize the joint distribution adaptation of both domains to enhance the effectiveness of transfer learning. Specifically, a Riemannian graph is first defined and constructed based on the Riemannian distance to represent the Riemannian geometry information. To simultaneously align the marginal and conditional distribution of source and target domains and preserve the Riemannian locality of data structure in both domains, the Riemannian graph is embedded in the joint distribution adaptation (JDA) framework and forms the proposed Riemannian locality preserving-based transfer learning (RLPTL). To validate the effect of the proposed method, it is compared with several existing methods on two open motor imagery datasets, and both multi-source domains (MSD) and single-source domains (SSD) experiments are considered. Experimental results show that the proposed method achieves the highest accuracies in MSD and SSD experiments on three datasets and outperforms eight baseline methods, which demonstrates that the proposed method creates a feasible and efficient way to realize transfer learning.

A novel method for modeling effective connections between brain regions based on EEG signals and graph neural networks for motor imagery detection

Classified as biomedical signal processing, cerebral signal processing plays a key role in human-computer interaction (HCI) and medical diagnosis. The motor imagery (MI) problem is an important research area in this field. Accurate solutions to this problem will greatly affect real-world applications. Most of the proposed methods are based on raw signal processing techniques. Known as prior knowledge, the structural-functional information and interregional connections can improve signal processing accuracy. It is possible to correctly perceive the generated signals by considering the brain structure (i.e. anatomical units), the source of signals, and the structural-functional dependence of different brain regions (i.e. effective connection) that are the semantic generators of signals. This study employed electroencephalograph (EEG) signals based on the activity of brain regions (cortex) and effective connections between brain regions based on dynamic causal modeling to solve the MI problem. EEG signals, as well as effective connections between brain regions to improve the interpretability of MI action, were fed into the architecture of Graph Convolutional Neural Network (GCN). The proposed model allowed GCN to extract more discriminative features. The results indicated that the proposed method was successful in developing a model with a MI detection accuracy of 93.73%.

Bi-hemisphere Interaction Convolutional Neural Network for Motor Imagery Classification

Decoding EEG-based, Motor Imagery Brain-Computer Interfaces (MI-BCI) in a subject-independent manner is very challenging due to high dimensionality of the EEG signal, and high inter-subject variability. In recent years, Convolutional neural networks (CNNs) have significantly enhanced decoding accuracy. Nevertheless, the majority of these CNN designs did not explicitly incorporate the inter-hemisphere functional connections, omitting crucial spatial information. Notably, in binary MI decoding of the left-hand versus right-hand, the Event-Related Desynchronization is observed in the contralateral hemisphere. Building upon this concept and various Neuroscience research, we have designed a CNN architecture that forges a functional connection between the two hemispheres. Specifically, we applied the Channel Average Referencing to one hemisphere and compared the output with all channels of the opposite hemisphere. Then, we utilized the cosine similarity to identify the most correlated channels and combined with them the original hemisphere for spatial filtering to learn the inter-hemispheric connections. This innovative technique aligns more closely with the actual brain functionality. Our method has demonstrated superior results on the Cho2017 and OpenBMI datasets, underscoring its effectiveness.

Multi-source deep domain adaptation ensemble framework for cross-dataset motor imagery EEG transfer learning

Objective. Electroencephalography (EEG) is an important kind of bioelectric signal for measuring physiological activities of the brain, and motor imagery (MI) EEG has significant clinical application prospects. Convolutional neural network has become a mainstream algorithm for MI EEG classification, however lack of subject-specific data considerably restricts its decoding accuracy and generalization performance. To address this challenge, a novel transfer learning (TL) framework using auxiliary dataset to improve the MI EEG classification performance of target subject is proposed in this paper.Approach. We developed a multi-source deep domain adaptation ensemble framework (MSDDAEF) for cross-dataset MI EEG decoding. The proposed MSDDAEF comprises three main components: model pre-training, deep domain adaptation, and multi-source ensemble. Moreover, for each component, different designs were examined to verify the robustness of MSDDAEF.Main results. Bidirectional validation experiments were performed on two large public MI EEG datasets (openBMI and GIST). The highest average classification accuracy of MSDDAEF reaches 74.28% when openBMI serves as target dataset and GIST serves as source dataset. While the highest average classification accuracy of MSDDAEF is 69.85% when GIST serves as target dataset and openBMI serves as source dataset. In addition, the classification performance of MSDDAEF surpasses several well-established studies and state-of-the-art algorithms.Significance. The results of this study show that cross-dataset TL is feasible for left/right-hand MI EEG decoding, and further indicate that MSDDAEF is a promising solution for addressing MI EEG cross-dataset variability.

Large scale investigation of the effect of gender on mu rhythm suppression in motor imagery brain-computer interfaces

The utmost issue in Motor Imagery Brain-Computer Interfaces (MI-BCI) is the BCI poor performance known as 'BCI inefficiency'. Although past research has attempted to find a solution by investigating factors influencing users' MI-BCI performance, the issue persists. One of the factors that has been studied in relation to MI-BCI performance is gender. Research regarding the influence of gender on a user's ability to control MI-BCIs remains inconclusive, mainly due to the small sample size and unbalanced gender distribution in past studies. To address these issues and obtain reliable results, this study combined four MI-BCI datasets into one large dataset with 248 subjects and equal gender distribution. The datasets included EEG signals from healthy subjects from both gender groups who had executed a right- vs. left-hand motor imagery task following the Graz protocol. The analysis consisted of extracting the Mu Suppression Index from C3 and C4 electrodes and comparing the values between female and male participants. Unlike some of the previous findings which reported an advantage for female BCI users in modulating mu rhythm activity, our results did not show any significant difference between the Mu Suppression Index of both groups, indicating that gender may not be a predictive factor for BCI performance.

Aggregating intrinsic information to enhance BCI performance through federated learning

Insufficient data is a long-standing challenge for Brain-Computer Interface (BCI) to build a high-performance deep learning model. Though numerous research groups and institutes collect a multitude of EEG datasets for the same BCI task, sharing EEG data from multiple sites is still challenging due to the heterogeneity of devices. The significance of this challenge cannot be overstated, given the critical role of data diversity in fostering model robustness. However, existing works rarely discuss this issue, predominantly centering their attention on model training within a single dataset, often in the context of inter-subject or inter-session settings. In this work, we propose a hierarchical personalized Federated Learning EEG decoding (FLEEG) framework to surmount this challenge. This innovative framework heralds a new learning paradigm for BCI, enabling datasets with disparate data formats to collaborate in the model training process. Each client is assigned a specific dataset and trains a hierarchical personalized model to manage diverse data formats and facilitate information exchange. Meanwhile, the server coordinates the training procedure to harness knowledge gleaned from all datasets, thus elevating overall performance. The framework has been evaluated in Motor Imagery (MI) classification with nine EEG datasets collected by different devices but implementing the same MI task. Results demonstrate that the proposed framework can boost classification performance up to 8.4% by enabling knowledge sharing between multiple datasets, especially for smaller datasets. Visualization results also indicate that the proposed framework can empower the local models to put a stable focus on task-related areas, yielding better performance. To the best of our knowledge, this is the first end-to-end solution to address this important challenge.

An EEG motor imagery dataset for brain computer interface in acute stroke patients

The brain-computer interface (BCI) is a technology that involves direct communication with parts of the brain and has evolved rapidly in recent years; it has begun to be used in clinical practice, such as for patient rehabilitation. Patient electroencephalography (EEG) datasets are critical for algorithm optimization and clinical applications of BCIs but are rare at present. We collected data from 50 acute stroke patients with wireless portable saline EEG devices during the performance of two tasks: 1) imagining right-handed movements and 2) imagining left-handed movements. The dataset consists of four types of data: 1) the motor imagery instructions, 2) raw recording data, 3) pre-processed data after removing artefacts and other manipulations, and 4) patient characteristics. This is the first open dataset to address left- and right-handed motor imagery in acute stroke patients. We believe that the dataset will be very helpful for analysing brain activation and designing decoding methods that are more applicable for acute stroke patients, which will greatly facilitate research in the field of motor imagery-BCI.

Beta bursts question the ruling power for brain-computer interfaces

Objective: Current efforts to build reliable brain-computer interfaces (BCI) span multiple axes from hardware, to software, to more sophisticated experimental protocols, and personalized approaches. However, despite these abundant efforts, there is still room for significant improvement. We argue that a rather overlooked direction lies in linking BCI protocols with recent advances in fundamental neuroscience.Approach: In light of these advances, and particularly the characterization of the burst-like nature of beta frequency band activity and the diversity of beta bursts, we revisit the role of beta activity in 'left vs. right hand' motor imagery (MI) tasks. Current decoding approaches for such tasks take advantage of the fact that MI generates time-locked changes in induced power in the sensorimotor cortex and rely on band-passed power changes in single or multiple channels. Although little is known about the dynamics of beta burst activity during MI, we hypothesized that beta bursts should be modulated in a way analogous to their activity during performance of real upper limb movements.Main results and Significance: We show that classification features based on patterns of beta burst modulations yield decoding results that are equivalent to or better than typically used beta power across multiple open electroencephalography datasets, thus providing insights into the specificity of these bio-markers.

MetaBCI: An open-source platform for brain-computer interfaces

BACKGROUND

Recently, brain-computer interfaces (BCIs) have attracted worldwide attention for their great potential in clinical and real-life applications. To implement a complete BCI system, one must set up several links to translate the brain intent into computer commands. However, there is not an open-source software platform that can cover all links of the BCI chain.

METHOD

This study developed a one-stop open-source BCI software, namely MetaBCI, to facilitate the construction of a BCI system. MetaBCI is written in Python, and has the functions of stimulus presentation (Brainstim), data loading and processing (Brainda), and online information flow (Brainflow). This paper introduces the detailed information of MetaBCI and presents four typical application cases.

RESULTS

The results showed that MetaBCI was an extensible and feature-rich software platform for BCI research and application, which could effectively encode, decode, and feedback brain activities.

CONCLUSIONS

MetaBCI can greatly lower the BCI's technical threshold for BCI beginners and can save time and cost to build up a practical BCI system. The source code is available at https://github.com/TBC-TJU/MetaBCI, expecting new contributions from the BCI community.

FBLPF-ABOW: An Effective Method for Blink Artifact Removal in Single-Channel EEG Signal

OBJECTIVE

The latest development in low-cost single-channel Electroencephalography (EEG) devices is gaining widespread attention because it reduces hardware complexity. Discrete wavelet transform (DWT) has been a popular solution to eliminate the blink artifacts in EEG signals. However, the existing DWT-based methods share the same wavelet function among subjects, which ignores the individual difference. To remedy this deficiency, this article proposes a novel approach to eliminate the blink artifacts in single-channel EEG signals.

METHODS

Firstly, the forward-backward low-pass filter (FBLPF) and a fixed-length window are used to detect blink artifact intervals. Secondly, the adaptive bi-orthogonal wavelet (ABOW) is constructed based on the most representative blink signal. Thirdly, these detected signals are filtered by ABOW-DWT. The DWT's decomposition depth is automatically chosen by a similarity-based method.

RESULTS

Compared to eight state-of-the-art methods, experiments on semi-simulated and real EEG signals demonstrate the proposed method's superiority in removing the blink artifacts with less neural information loss.

SIGNIFICANCE

To filter the blink artifacts in single-channel EEG signals, the innovative idea of constructing an adaptive wavelet function based on the signal characteristics rather than using the conventional wavelet is proposed for the first time.

Explainable cross-task adaptive transfer learning for motor imagery EEG classification

Objective. In the field of motor imagery (MI) electroencephalography (EEG)-based brain-computer interfaces, deep transfer learning (TL) has proven to be an effective tool for solving the problem of limited availability in subject-specific data for the training of robust deep learning (DL) models. Although considerable progress has been made in the cross-subject/session and cross-device scenarios, the more challenging problem of cross-task deep TL remains largely unexplored.Approach. We propose a novel explainable cross-task adaptive TL method for MI EEG decoding. Firstly, similarity analysis and data alignment are performed for EEG data of motor execution (ME) and MI tasks. Afterwards, the MI EEG decoding model is obtained via pre-training with extensive ME EEG data and fine-tuning with partial MI EEG data. Finally, expected gradient-based post-hoc explainability analysis is conducted for the visualization of important temporal-spatial features.Main results. Extensive experiments are conducted on one large ME EEG High-Gamma dataset and two large MI EEG datasets (openBMI and GIST). The best average classification accuracy of our method reaches 80.00% and 72.73% for OpenBMI and GIST respectively, which outperforms several state-of-the-art algorithms. In addition, the results of the explainability analysis further validate the correlation between ME and MI EEG data and the effectiveness of ME/MI cross-task adaptation.Significance. This paper confirms that the decoding of MI EEG can be well facilitated by pre-existing ME EEG data, which largely relaxes the constraint of training samples for MI EEG decoding and is important in a practical sense.

Spatio-Temporal Explanation of 3D-EEGNet for Motor Imagery EEG Classification Using Permutation and Saliency

Recently, convolutional neural network (CNN)-based classification models have shown good performance for motor imagery (MI) brain-computer interfaces (BCI) using electroencephalogram (EEG) in end-to-end learning. Although a few explainable artificial intelligence (XAI) techniques have been developed, it is still challenging to interpret the CNN models for EEG-based BCI classification effectively. In this research, we propose 3D-EEGNet as a 3D CNN model to improve both the explainability and performance of MI EEG classification. The proposed approach exhibited better performances on two MI EEG datasets than the existing EEGNet, which uses a 2D input shape. The MI classification accuracies are improved around 1.8% and 6.1% point in average on the datasets, respectively. The permutation-based XAI method is first applied for the reliable explanation of the 3D-EEGNet. Next, to find a faster XAI method for spatio-temporal explanation, we design a novel technique based on the normalized discounted cumulative gain (NDCG) for selecting the best among a few saliency-based methods due to their higher time complexity than the permutation-based method. Among the saliency-based methods, DeepLIFT was selected because the NDCG scores indicated its results are the most similar to the permutation-based results. Finally, the fast spatio-temporal explanation using DeepLIFT provides deeper understanding for the classification results of the 3D-EEGNet and the important properties in the MI EEG experiments.

Hierarchical Transformer for Motor Imagery-Based Brain Computer Interface

In this paper, we propose a novel transformer-based classification algorithm for the brain computer interface (BCI) using a motor imagery (MI) electroencephalogram (EEG) signal. To design the MI classification algorithm, we apply an up-to-date deep learning model, the transformer, that has revolutionized the natural language processing (NLP) and successfully widened its application to many other domains such as the computer vision. Within a long MI trial spanning a few seconds, the classification algorithm should give more attention to the time periods during which the intended motor task is imagined by the subject without any artifact. To achieve this goal, we propose a hierarchical transformer architecture that consists of a high-level transformer (HLT) and a low-level transformer (LLT). We break down a long MI trial into a number of short-term intervals. The LLT extracts a feature from each short-term interval, and the HLT pays more attention to the features from more relevant short-term intervals by using the self-attention mechanism of the transformer. We have done extensive tests of the proposed scheme on four open MI datasets, and shown that the proposed hierarchical transformer excels in both the subject-dependent and subject-independent tests.

Feature learning framework based on EEG graph self-attention networks for motor imagery BCI systems

Learning distinguishable features from raw EEG signals is crucial for accurate classification of motor imagery (MI) tasks. To incorporate spatial relationships between EEG sources, we developed a feature set based on an EEG graph. In this graph, EEG channels represent the nodes, with power spectral density (PSD) features defining their properties, and the edges preserving the spatial information. We designed an EEG based graph self-attention network (EGSAN) to learn low-dimensional embedding vector for EEG graph, which can be used as distinguishable features for motor imagery task classification. We evaluated our EGSAN model on two publicly available MI EEG datasets, each containing different types of motor imagery tasks. Our experiments demonstrate that our proposed model effectively extracts distinguishable features from EEG graphs, achieving significantly higher classification accuracies than existing state-of-the-art methods.

MBGA-Net: A multi-branch graph adaptive network for individualized motor imagery EEG classification

BACKGROUND AND OBJECTIVE

The development of deep learning has led to significant improvements in the decoding accuracy of Motor Imagery (MI) EEG signal classification. However, current models are inadequate in ensuring high levels of classification accuracy for an individual. Since MI EEG data is primarily used in medical rehabilitation and intelligent control, it is crucial to ensure that each individual's EEG signal is recognized with precision.

METHODS

We propose a multi-branch graph adaptive network (MBGA-Net), which matches each individual EEG signal with a suitable time-frequency domain processing method based on spatio-temporal domain features. We then feed the signal into the relevant model branch using an adaptive technique. Through an enhanced attention mechanism and deep convolutional method with residual connectivity, each model branch more effectively harvests the features of the related format data.

RESULTS

We validate the proposed model using the BCI Competition IV dataset 2a and dataset 2b. On dataset 2a, the average accuracy and kappa values are 87.49% and 0.83, respectively. The standard deviation of individual kappa values is only 0.08. For dataset 2b, the average classification accuracies obtained by feeding the data into the three branches of MBGA-Net are 85.71%, 85.83%, and 86.99%, respectively.

CONCLUSIONS

The experimental results demonstrate that MBGA-Net could effectively perform the classification task of motor imagery EEG signals, and it exhibits strong generalization performance. The proposed adaptive matching technique enhances the classification accuracy of each individual, which is beneficial for the practical application of EEG classification.

A large EEG database with users' profile information for motor imagery brain-computer interface research

We present and share a large database containing electroencephalographic signals from 87 human participants, collected during a single day of brain-computer interface (BCI) experiments, organized into 3 datasets (A, B, and C) that were all recorded using the same protocol: right and left hand motor imagery (MI). Each session contains 240 trials (120 per class), which represents more than 20,800 trials, or approximately 70 hours of recording time. It includes the performance of the associated BCI users, detailed information about the demographics, personality profile as well as some cognitive traits and the experimental instructions and codes (executed in the open-source platform OpenViBE). Such database could prove useful for various studies, including but not limited to: (1) studying the relationships between BCI users' profiles and their BCI performances, (2) studying how EEG signals properties varies for different users' profiles and MI tasks, (3) using the large number of participants to design cross-user BCI machine learning algorithms or (4) incorporating users' profile information into the design of EEG signal classification algorithms.

Brain-computer interface: trend, challenges, and threats

Brain-computer interface (BCI), an emerging technology that facilitates communication between brain and computer, has attracted a great deal of research in recent years. Researchers provide experimental results demonstrating that BCI can restore the capabilities of physically challenged people, hence improving the quality of their lives. BCI has revolutionized and positively impacted several industries, including entertainment and gaming, automation and control, education, neuromarketing, and neuroergonomics. Notwithstanding its broad range of applications, the global trend of BCI remains lightly discussed in the literature. Understanding the trend may inform researchers and practitioners on the direction of the field, and on where they should invest their efforts more. Noting this significance, we have analyzed 25,336 metadata of BCI publications from Scopus to determine advancement of the field. The analysis shows an exponential growth of BCI publications in China from 2019 onwards, exceeding those from the United States that started to decline during the same period. Implications and reasons for this trend are discussed. Furthermore, we have extensively discussed challenges and threats limiting exploitation of BCI capabilities. A typical BCI architecture is hypothesized to address two prominent BCI threats, privacy and security, as an attempt to make the technology commercially viable to the society.

Differentiating brain states via multi-clip random fragment strategy-based interactive bidirectional recurrent neural network

EEG is widely adopted to study the brain and brain computer interface (BCI) for its non-invasiveness and low costs. Specifically EEG can be applied to differentiate brain states, which is important for better understanding the working mechanisms of the brain. Recurrent neural network (RNN)-based learning strategy has been widely utilized to differentiate brain states, because its optimization architectures improve the classification performance for differentiating brain states at the group level. However, present classification performance is still far from satisfactory. We have identified two major focal points for improvements: one is about organizing the input EEG signals, and the other is related to the design of the RNN architecture. To optimize the above-mentioned issues and achieve better brain state classification performance, we propose a novel multi-clip random fragment strategy-based interactive bidirectional recurrent neural network (McRFS-IBiRNN) model in this work. This model has two advantages over previous methods. First, the McRFS component is designed to re-organize the input EEG signals to make them more suitable for the RNN architecture. Second, the IBiRNN component is an innovative design to model the RNN layers with interaction connections to enhance the fusion of bidirectional features. By adopting the proposed model, promising brain states classification performances are obtained. For example, 96.97% and 99.34% of individual and group level four-category classification accuracies are successfully obtained on the EEG motor/imagery dataset, respectively. A 99.01% accuracy can be observed for four-category classification tasks with new subjects not seen before, which demonstrates the generalization of our proposed method. Compared with existing methods, our model outperforms them with superior results. Overall, the proposed McRFS-IBiRNN model demonstrates great superiority in differentiating brain states on EEG signals.

Paving the Way for Motor Imagery-Based Tele-Rehabilitation through a Fully Wearable BCI System

The present study introduces a brain-computer interface designed and prototyped to be wearable and usable in daily life. Eight dry electroencephalographic sensors were adopted to acquire the brain activity associated with motor imagery. Multimodal feedback in extended reality was exploited to improve the online detection of neurological phenomena. Twenty-seven healthy subjects used the proposed system in five sessions to investigate the effects of feedback on motor imagery. The sample was divided into two equal-sized groups: a "neurofeedback" group, which performed motor imagery while receiving feedback, and a "control" group, which performed motor imagery with no feedback. Questionnaires were administered to participants aiming to investigate the usability of the proposed system and an individual's ability to imagine movements. The highest mean classification accuracy across the subjects of the control group was about 62% with 3% associated type A uncertainty, and it was 69% with 3% uncertainty for the neurofeedback group. Moreover, the results in some cases were significantly higher for the neurofeedback group. The perceived usability by all participants was high. Overall, the study aimed at highlighting the advantages and the pitfalls of using a wearable brain-computer interface with dry sensors. Notably, this technology can be adopted for safe and economically viable tele-rehabilitation.

Review of public motor imagery and execution datasets in brain-computer interfaces

The demand for public datasets has increased as data-driven methodologies have been introduced in the field of brain-computer interfaces (BCIs). Indeed, many BCI datasets are available in various platforms or repositories on the web, and the studies that have employed these datasets appear to be increasing. Motor imagery is one of the significant control paradigms in the BCI field, and many datasets related to motor tasks are open to the public already. However, to the best of our knowledge, these studies have yet to investigate and evaluate the datasets, although data quality is essential for reliable results and the design of subject- or system-independent BCIs. In this study, we conducted a thorough investigation of motor imagery/execution EEG datasets recorded from healthy participants published over the past 13 years. The 25 datasets were collected from six repositories and subjected to a meta-analysis. In particular, we reviewed the specifications of the recording settings and experimental design, and evaluated the data quality measured by classification accuracy from standard algorithms such as Common Spatial Pattern (CSP) and Linear Discriminant Analysis (LDA) for comparison and compatibility across the datasets. As a result, we found that various stimulation types, such as text, figure, or arrow, were used to instruct subjects what to imagine and the length of each trial also differed, ranging from 2.5 to 29 s with a mean of 9.8 s. Typically, each trial consisted of multiple sections: pre-rest (2.38 s), imagination ready (1.64 s), imagination (4.26 s, ranging from 1 to 10 s), the post-rest (3.38 s). In a meta-analysis of the total of 861 sessions from all datasets, the mean classification accuracy of the two-class (left-hand vs. right-hand motor imagery) problem was 66.53%, and the population of the BCI poor performers, those who are unable to reach proficiency in using a BCI system, was 36.27% according to the estimated accuracy distribution. Further, we analyzed the CSP features and found that each dataset forms a cluster, and some datasets overlap in the feature space, indicating a greater similarity among them. Finally, we checked the minimal essential information (continuous signals, event type/latency, and channel information) that should be included in the datasets for convenient use, and found that only 71% of the datasets met those criteria. Our attempts to evaluate and compare the public datasets are timely, and these results will contribute to understanding the dataset's quality and recording settings as well as the use of using public datasets for future work on BCIs.

KCS-FCnet: Kernel Cross-Spectral Functional Connectivity Network for EEG-Based Motor Imagery Classification

This paper uses EEG data to introduce an approach for classifying right and left-hand classes in Motor Imagery (MI) tasks. The Kernel Cross-Spectral Functional Connectivity Network (KCS-FCnet) method addresses these limitations by providing richer spatial-temporal-spectral feature maps, a simpler architecture, and a more interpretable approach for EEG-driven MI discrimination. In particular, KCS-FCnet uses a single 1D-convolutional-based neural network to extract temporal-frequency features from raw EEG data and a cross-spectral Gaussian kernel connectivity layer to model channel functional relationships. As a result, the functional connectivity feature map reduces the number of parameters, improving interpretability by extracting meaningful patterns related to MI tasks. These patterns can be adapted to the subject’s unique characteristics. The validation results prove that introducing KCS-FCnet shallow architecture is a promising approach for EEG-based MI classification with the potential for real-world use in brain–computer interface systems.

Comparison of subject-independent and subject-specific EEG-based BCI using LDA and SVM classifiers

Motor imagery brain-computer interface (MI-BCI) is one of the most used paradigms in EEG-based brain-computer interface (BCI). The current state-of-the-art in BCI involves tuning classifiers to subject-specific training data, acquired over several sessions, in order to perform calibration prior to actual use of the so-called subject-specific BCI system (SS-BCI). Herein, the goal is to provide a ready-to-use system requiring minimal effort for setup. Thus, our challenge was to design a subject-independent BCI (SI-BCI) to be used by any new user without the constraint of individual calibration. Outcomes from other studies with the same purpose were used to undertake comparisons and validate our findings. For the EEG signal processing, we used a combination of the delta (0.5-4 Hz), alpha (8-13 Hz), and beta+gamma (13-40 Hz) bands at a stage prior to feature extraction. Next, we extracted features from the 27-channel EEG using common spatial pattern (CSP) and performed binary classification (MI of right- and left-hand) with linear discriminant analysis (LDA) and support vector machine (SVM) classifiers. These analyses were done for both the SS-BCI and SI-BCI models. We employed "leave-one-subject-out" (LOSO) arrangement and 10-fold cross-validation to evaluate our SI-BCI and SS-BCI systems, respectively. Compared with other two studies, our work was the only one that showed higher accuracy for the LDA classifier in SI-BCI as compared to SS-BCI. On the other hand, LDA accuracy was lower than accuracy achieved with SVM in both conditions (SI-BCI and SS-BCI). Our SS-BCI accuracy reached 76.85% using LDA and 94.20% using SVM and for SI-BCI we got 80.30% with LDA and 83.23% with SVM. We conclude that SI-BCI may be a feasible and relevant option, which can be used in scenarios where subjects are not able to submit themselves to long training sessions or to fast evaluation of the so called "BCI illiteracy." Comparatively, our strategy proved to be more efficient, giving us the best result for SI-BCI when faced against the classification performances of other three studies, even considering the caveat that different datasets were used in the comparison of the four studies.

Riemannian geometry-based metrics to measure and reinforce user performance changes during brain-computer interface user training

Despite growing interest and research into brain-computer interfaces (BCI), their usage remains limited outside of research laboratories. One reason for this is BCI inefficiency, the phenomenon where a significant number of potential users are unable to produce machine-discernible brain signal patterns to control the devices. To reduce the prevalence of BCI inefficiency, some have advocated for novel user-training protocols that enable users to more effectively modulate their neural activity. Important considerations for the design of these protocols are the assessment measures that are used for evaluating user performance and for providing feedback that guides skill acquisition. Herein, we present three trial-wise adaptations (running, sliding window and weighted average) of Riemannian geometry-based user-performance metrics (classDistinct reflecting the degree of class separability and classStability reflecting the level of within-class consistency) to enable feedback to the user following each individual trial. We evaluated these metrics, along with conventional classifier feedback, using simulated and previously recorded sensorimotor rhythm-BCI data to assess their correlation with and discrimination of broader trends in user performance. Analysis revealed that the sliding window and weighted average variants of our proposed trial-wise Riemannian geometry-based metrics more accurately reflected performance changes during BCI sessions compared to conventional classifier output. The results indicate the metrics are a viable method for evaluating and tracking user performance changes during BCI-user training and, therefore, further investigation into how these metrics may be presented to users during training is warranted.

Mutual Information-Driven Subject-Invariant and Class-Relevant Deep Representation Learning in BCI

In recent years, deep learning-based feature representation methods have shown a promising impact on electroencephalography (EEG)-based brain-computer interface (BCI). Nonetheless, owing to high intra- and inter-subject variabilities, many studies on decoding EEG were designed in a subject-specific manner by using calibration samples, with no concern of its practical use, hampered by time-consuming steps and a large data requirement. To this end, recent studies adopted a transfer learning strategy, especially domain adaptation techniques. Among those, we have witnessed the potential of adversarial learning-based transfer learning in BCIs. In the meantime, it is known that adversarial learning-based domain adaptation methods are prone to negative transfer that disrupts learning generalized feature representations, applicable to diverse domains, for example, subjects or sessions in BCIs. In this article, we propose a novel framework that learns class-relevant and subject-invariant feature representations in an information-theoretic manner, without using adversarial learning. To be specific, we devise two operational components in a deep network that explicitly estimate mutual information between feature representations: 1) to decompose features in an intermediate layer into class-relevant and class-irrelevant ones and 2) to enrich class-discriminative feature representation. On two large EEG datasets, we validated the effectiveness of our proposed framework by comparing with several comparative methods in performance. Furthermore, we conducted rigorous analyses by performing an ablation study in regard to the components in our network, explaining our model's decision on input EEG signals via layer-wise relevance propagation, and visualizing the distribution of learned features via t-SNE.

M3CV: A multi-subject, multi-session, and multi-task database for EEG-based biometrics challenge

EEG signals exhibit commonality and variability across subjects, sessions, and tasks. But most existing EEG studies focus on mean group effects (commonality) by averaging signals over trials and subjects. The substantial intra- and inter-subject variability of EEG have often been overlooked. The recently significant technological advances in machine learning, especially deep learning, have brought technological innovations to EEG signal application in many aspects, but there are still great challenges in cross-session, cross-task, and cross-subject EEG decoding. In this work, an EEG-based biometric competition based on a large-scale M³CV (A Multi-subject, Multi-session, and Multi-task Database for investigation of EEG Commonality and Variability) database was launched to better characterize and harness the intra- and inter-subject variability and promote the development of machine learning algorithm in this field. In the M³CV database, EEG signals were recorded from 106 subjects, of which 95 subjects repeated two sessions of the experiments on different days. The whole experiment consisted of 6 paradigms, including resting-state, transient-state sensory, steady-state sensory, cognitive oddball, motor execution, and steady-state sensory with selective attention with 14 types of EEG signals, 120000 epochs. Two learning tasks (identification and verification), performance metrics, and baseline methods were introduced in the competition. In general, the proposed M³CV dataset and the EEG-based biometric competition aim to provide the opportunity to develop advanced machine learning algorithms for achieving an in-depth understanding of the commonality and variability of EEG signals across subjects, sessions, and tasks.