Optimal spatial filtering of single trial EEG during imagined hand movement

Spatial Visual Imagery (SVI)-Based Electroencephalograph Discrimination for Natural CAD Manipulation

With the increasing demand for natural interactions, people have realized that an intuitive Computer-Aided Design (CAD) interaction mode can reduce the complexity of CAD operation and improve the design experience. Although interaction modes like gaze and gesture are compatible with some complex CAD manipulations, they still require people to express their design intentions physically. The brain contains design intentions implicitly and controls the corresponding body parts that execute the task. Therefore, building an end-to-end channel between the brain and computer as an auxiliary mode for CAD manipulation will allow people to send design intentions mentally and make their interaction more intuitive. This work focuses on the 1-D translation scene and studies a spatial visual imagery (SVI) paradigm to provide theoretical support for building an electroencephalograph (EEG)-based brain-computer interface (BCI) for CAD manipulation. Based on the analysis of three spatial EEG features related to SVI (e.g., common spatial patterns, cross-correlation, and coherence), a multi-feature fusion-based discrimination model was built for SVI. The average accuracy of the intent discrimination of 10 subjects was 86%, and the highest accuracy was 93%. The method proposed was verified to be feasible for discriminating the intentions of CAD object translation with good classification performance. This work further proves the potential of BCI in natural CAD manipulation.

Brain-computer interface paradigms and neural coding

Brain signal patterns generated in the central nervous system of brain-computer interface (BCI) users are closely related to BCI paradigms and neural coding. In BCI systems, BCI paradigms and neural coding are critical elements for BCI research. However, so far there have been few references that clearly and systematically elaborated on the definition and design principles of the BCI paradigm as well as the definition and modeling principles of BCI neural coding. Therefore, these contents are expounded and the existing main BCI paradigms and neural coding are introduced in the review. Finally, the challenges and future research directions of BCI paradigm and neural coding were discussed, including user-centered design and evaluation for BCI paradigms and neural coding, revolutionizing the traditional BCI paradigms, breaking through the existing techniques for collecting brain signals and combining BCI technology with advanced AI technology to improve brain signal decoding performance. It is expected that the review will inspire innovative research and development of the BCI paradigm and neural coding.

Revealing brain connectivity: graph embeddings for EEG representation learning and comparative analysis of structural and functional connectivity

This study employs deep learning techniques to present a compelling approach for modeling brain connectivity in EEG motor imagery classification through graph embedding. The compelling aspect of this study lies in its combination of graph embedding, deep learning, and different brain connectivity types, which not only enhances classification accuracy but also enriches the understanding of brain function. The approach yields high accuracy, providing valuable insights into brain connections and has potential applications in understanding neurological conditions. The proposed models consist of two distinct graph-based convolutional neural networks, each leveraging different types of brain connectivities to enhance classification performance and gain a deeper understanding of brain connections. The first model, Adjacency-based Convolutional Neural Network Model (Adj-CNNM), utilizes a graph representation based on structural brain connectivity to embed spatial information, distinguishing it from prior spatial filtering approaches dependent on subjects and tasks. Extensive tests on a benchmark dataset-IV-2a demonstrate that an accuracy of 72.77% is achieved by the Adj-CNNM, surpassing baseline and state-of-the-art methods. The second model, Phase Locking Value Convolutional Neural Network Model (PLV-CNNM), incorporates functional connectivity to overcome structural connectivity limitations and identifies connections between distinct brain regions. The PLV-CNNM achieves an overall accuracy of 75.10% across the 1-51 Hz frequency range. In the preferred 8-30 Hz frequency band, known for motor imagery data classification (including α, μ, and β waves), individual accuracies of 91.9%, 90.2%, and 85.8% are attained for α, μ, and β, respectively. Moreover, the model performs admirably with 84.3% accuracy when considering the entire 8-30 Hz band. Notably, the PLV-CNNM reveals robust connections between different brain regions during motor imagery tasks, including the frontal and central cortex and the central and parietal cortex. These findings provide valuable insights into brain connectivity patterns, enriching the comprehension of brain function. Additionally, the study offers a comprehensive comparative analysis of diverse brain connectivity modeling methods.

MetaBCI: An open-source platform for brain-computer interfaces

BACKGROUND

Recently, brain-computer interfaces (BCIs) have attracted worldwide attention for their great potential in clinical and real-life applications. To implement a complete BCI system, one must set up several links to translate the brain intent into computer commands. However, there is not an open-source software platform that can cover all links of the BCI chain.

METHOD

This study developed a one-stop open-source BCI software, namely MetaBCI, to facilitate the construction of a BCI system. MetaBCI is written in Python, and has the functions of stimulus presentation (Brainstim), data loading and processing (Brainda), and online information flow (Brainflow). This paper introduces the detailed information of MetaBCI and presents four typical application cases.

RESULTS

The results showed that MetaBCI was an extensible and feature-rich software platform for BCI research and application, which could effectively encode, decode, and feedback brain activities.

CONCLUSIONS

MetaBCI can greatly lower the BCI's technical threshold for BCI beginners and can save time and cost to build up a practical BCI system. The source code is available at https://github.com/TBC-TJU/MetaBCI, expecting new contributions from the BCI community.

An in-depth survey on Deep Learning-based Motor Imagery Electroencephalogram (EEG) classification

Electroencephalogram (EEG)-based Brain-Computer Interfaces (BCIs) build a communication path between human brain and external devices. Among EEG-based BCI paradigms, the most commonly used one is motor imagery (MI). As a hot research topic, MI EEG-based BCI has largely contributed to medical fields and smart home industry. However, because of the low signal-to-noise ratio (SNR) and the non-stationary characteristic of EEG data, it is difficult to correctly classify different types of MI-EEG signals. Recently, the advances in Deep Learning (DL) significantly facilitate the development of MI EEG-based BCIs. In this paper, we provide a systematic survey of DL-based MI-EEG classification methods. Specifically, we first comprehensively discuss several important aspects of DL-based MI-EEG classification, covering input formulations, network architectures, public datasets, etc. Then, we summarize problems in model performance comparison and give guidelines to future studies for fair performance comparison. Next, we fairly evaluate the representative DL-based models using source code released by the authors and meticulously analyse the evaluation results. By performing ablation study on the network architecture, we found that (1) effective feature fusion is indispensable for multi-stream CNN-based models. (2) LSTM should be combined with spatial feature extraction techniques to obtain good classification performance. (3) the use of dropout contributes little to improving the model performance, and that (4) adding fully connected layers to the models significantly increases their parameters but it might not improve their performance. Finally, we raise several open issues in MI-EEG classification and provide possible future research directions.

Multimodal pre-screening can predict BCI performance variability: A novel subject-specific experimental scheme

BACKGROUND

Brain-computer interface (BCI) systems currently lack the required robustness for long-term daily use due to inter- and intra-subject performance variability. In this study, we propose a novel personalized scheme for a multimodal BCI system, primarily using functional near-infrared spectroscopy (fNIRS) and electroencephalography (EEG), to identify, predict, and compensate for factors affecting competence-related and interfering factors associated with performance.

METHOD

11 (out of 13 recruited) participants, including five participants with motor deficits, completed four sessions on average. During the training sessions, the subjects performed a short pre-screening phase, followed by three variations of a novel visou-mental (VM) protocol. Features extracted from the pre-screening phase were used to construct predictive platforms using stepwise multivariate linear regression (MLR) models. In the test sessions, we employed a task-correction phase where our predictive models were used to predict the ideal task variation to maximize performance, followed by an interference-correction phase. We then investigated the associations between predicted and actual performances and evaluated the outcome of correction strategies.

RESULT

The predictive models resulted in respective adjusted R-squared values of 0.942, 0.724, and 0.939 for the first, second, and third variation of the task, respectively. The statistical analyses showed significant associations between the performances predicted by predictive models and the actual performances for the first two task variations, with rhos of 0.7289 (p-value = 0.011) and 0.6970 (p-value = 0.017), respectively. For 81.82 % of the subjects, the task/workload correction stage correctly determined which task variation provided the highest accuracy, with an average performance gain of 5.18 % when applying the correction strategies.

CONCLUSION

Our proposed method can lead to an integrated multimodal predictive framework to compensate for BCI performance variability, particularly, for people with severe motor deficits.

An overview of machine learning and deep learning techniques for predicting epileptic seizures

Epilepsy is a neurological disorder (the third most common, following stroke and migraines). A key aspect of its diagnosis is the presence of seizures that occur without a known cause and the potential for new seizures to occur. Machine learning has shown potential as a cost-effective alternative for rapid diagnosis. In this study, we review the current state of machine learning in the detection and prediction of epileptic seizures. The objective of this study is to portray the existing machine learning methods for seizure prediction. Internet bibliographical searches were conducted to identify relevant literature on the topic. Through cross-referencing from key articles, additional references were obtained to provide a comprehensive overview of the techniques. As the aim of this paper aims is not a pure bibliographical review of the subject, the publications here cited have been selected among many others based on their number of citations. To implement accurate diagnostic and treatment tools, it is necessary to achieve a balance between prediction time, sensitivity, and specificity. This balance can be achieved using deep learning algorithms. The best performance and results are often achieved by combining multiple techniques and features, but this approach can also increase computational requirements.

Explainable cross-task adaptive transfer learning for motor imagery EEG classification

Objective. In the field of motor imagery (MI) electroencephalography (EEG)-based brain-computer interfaces, deep transfer learning (TL) has proven to be an effective tool for solving the problem of limited availability in subject-specific data for the training of robust deep learning (DL) models. Although considerable progress has been made in the cross-subject/session and cross-device scenarios, the more challenging problem of cross-task deep TL remains largely unexplored.Approach. We propose a novel explainable cross-task adaptive TL method for MI EEG decoding. Firstly, similarity analysis and data alignment are performed for EEG data of motor execution (ME) and MI tasks. Afterwards, the MI EEG decoding model is obtained via pre-training with extensive ME EEG data and fine-tuning with partial MI EEG data. Finally, expected gradient-based post-hoc explainability analysis is conducted for the visualization of important temporal-spatial features.Main results. Extensive experiments are conducted on one large ME EEG High-Gamma dataset and two large MI EEG datasets (openBMI and GIST). The best average classification accuracy of our method reaches 80.00% and 72.73% for OpenBMI and GIST respectively, which outperforms several state-of-the-art algorithms. In addition, the results of the explainability analysis further validate the correlation between ME and MI EEG data and the effectiveness of ME/MI cross-task adaptation.Significance. This paper confirms that the decoding of MI EEG can be well facilitated by pre-existing ME EEG data, which largely relaxes the constraint of training samples for MI EEG decoding and is important in a practical sense.

A multi-band centroid contrastive reconstruction fusion network for motor imagery electroencephalogram signal decoding

Motor imagery (MI) brain-computer interface (BCI) assist users in establishing direct communication between their brain and external devices by decoding the movement intention of human electroencephalogram (EEG) signals. However, cerebral cortical potentials are highly rhythmic and sub-band features, different experimental situations and subjects have different categories of semantic information in specific sample target spaces. Feature fusion can lead to more discriminative features, but simple fusion of features from different embedding spaces leading to the model global loss is not easily convergent and ignores the complementarity of features. Considering the similarity and category contribution of different sub-band features, we propose a multi-band centroid contrastive reconstruction fusion network (MB-CCRF). We obtain multi-band spatio-temporal features by frequency division, preserving the task-related rhythmic features of different EEG signals; use a multi-stream cross-layer connected convolutional network to perform a deep feature representation for each sub-band separately; propose a centroid contrastive reconstruction fusion module, which maps different sub-band and category features into the same shared embedding space by comparing with category prototypes, reconstructing the feature semantic structure to ensure that the global loss of the fused features converges more easily. Finally, we use a learning mechanism to model the similarity between channel features and use it as the weight of fused sub-band features, thus enhancing the more discriminative features, suppressing the useless features. The experimental accuracy is 79.96% in the BCI competition Ⅳ-Ⅱa dataset. Moreover, the classification effect of sub-band features of different subjects is verified by comparison tests, the category propensity of different sub-band features is verified by confusion matrix tests and the distribution in different classes of each sub-band feature and fused feature are showed by visual analysis, revealing the importance of different sub-band features for the EEG-based MI classification task.

Rhythmic temporal prediction enhances neural representations of movement intention for brain-computer interface

Objective.Detecting movement intention is a typical use of brain-computer interfaces (BCI). However, as an endogenous electroencephalography (EEG) feature, the neural representation of movement is insufficient for improving motor-based BCI. This study aimed to develop a new movement augmentation BCI encoding paradigm by incorporating the cognitive function of rhythmic temporal prediction, and test the feasibility of this new paradigm in optimizing detections of movement intention.Methods.A visual-motion synchronization task was designed with two movement intentions (left vs. right) and three rhythmic temporal prediction conditions (1000 ms vs. 1500 ms vs. no temporal prediction). Behavioural and EEG data of 24 healthy participants were recorded. Event-related potentials (ERPs), event-related spectral perturbation induced by left- and right-finger movements, the common spatial pattern (CSP) and support vector machine, Riemann tangent space algorithm and logistic regression were used and compared across the three temporal prediction conditions, aiming to test the impact of temporal prediction on movement detection.Results.Behavioural results showed significantly smaller deviation time for 1000 ms and 1500 ms conditions. ERP analyses revealed 1000 ms and 1500 ms conditions led to rhythmic oscillations with a time lag in contralateral and ipsilateral areas of movement. Compared with no temporal prediction, 1000 ms condition exhibited greater beta event-related desynchronization (ERD) lateralization in motor area (P< 0.001) and larger beta ERD in frontal area (P< 0.001). 1000 ms condition achieved an averaged left-right decoding accuracy of 89.71% using CSP and 97.30% using Riemann tangent space, both significantly higher than no temporal prediction. Moreover, movement and temporal information can be decoded simultaneously, achieving 88.51% four-classification accuracy.Significance.The results not only confirm the effectiveness of rhythmic temporal prediction in enhancing detection ability of motor-based BCI, but also highlight the dual encodings of movement and temporal information within a single BCI paradigm, which is promising to expand the range of intentions that can be decoded by the BCI.

Attentional State Classification Using Amplitude and Phase Feature Extraction Method Based on Filter Bank and Riemannian Manifold

As a significant aspect of cognition, attention has been extensively studied and numerous measurements have been developed based on brain signal processing. Although existing attentional state classification methods have achieved good accuracy by extracting a variety of handcrafted features, spatial features have not been fully explored. This paper proposes an attentional state classification method based on Riemannian manifold to utilize spatial information. Based on the concept of Riemannian manifold of symmetric positive definite (SPD) matrix, the proposed method exploits the structure of covariance matrix to extract spatial features instead of using spatial filters. Specifically, Riemannian distances from intra-class Riemannian means are extracted as features for their robustness. To fully extend the potential of electroencephalograph (EEG) signal, both amplitude and phase information is utilized. In addition, to solve the variance of frequency bands, a filter bank is employed to process the signal of different frequency bands separately. Finally, features are fed into a support vector machine with a polynomial kernel to obtain classification results. The proposed attentional state classification using amplitude and phase feature extraction method based on filter bank and Riemannian manifold (AP-FBRM) method is evaluated on two open datasets including EEG data of 29 and 26 subjects. According to the experimental results, the optimal set of filter bank and the optimal technique to extract features containing both amplitude and phase information are determined. The proposed method respectively achieves accuracies of 88.06% and 80.00% and outperforms 8 baseline methods, which manifests that the proposed method creates an efficient way to recognize attentional state.

Variance characteristic preserving common spatial pattern for motor imagery BCI

Introduction

The common spatial patterns (CSP) algorithm is the most popular technique for extracting electroencephalogram (EEG) features in motor imagery based brain-computer interface (BCI) systems. CSP algorithm embeds the dimensionality of multichannel EEG data to extract features of motor imagery tasks. Most previous studies focused on the optimization of the time domain and the spectrum domain of EEG signal to improve the effectiveness of CSP, whereas ignoring the constraint on the projected feature space.

Methods

This study proposed a variance characteristic preserving CSP (VPCSP) that is modified by a regularization item based on graph theory. Specifically, we calculated the loss of abnormalities of the projected data while preserving the variance characteristic locally. Then the loss could be rewritten as a matrix with the introduction of the Laplace matrix, which turned it into a generalized eigenvalue problem equivalent to CSP. This study evaluated the proposed method on two public EEG datasets from the BCI competition. The modified method could extract robust and distinguishable features that provided higher classification performance. Experimental results showed that the proposed regularization improved the effectiveness of CSP significantly and achieved superior performance compared with reported modified CSP algorithms significantly.

Results

The classification accuracy of the proposed method achieved 87.88 %, 90.07 %, and 76.06 % on public dataset IV part I, III part IVa and the self-collected dataset, respectively. Comparative experiments are conducted on two public datasets and one self-collected dataset. Results showed that the proposed method outperformed the reported algorithm.

Discussion

The proposed method can extract robust features to increase the performance of BCI systems. And the proposal still has expandability. These results show that our proposal is a promising candidate for the performance improvement of MI-BCI.

Visual short-term memory-related EEG components in a virtual reality setup

Virtual reality (VR) offers a powerful tool for investigating cognitive processes, as it allows researchers to gauge behaviors and mental states in complex, yet highly controlled, scenarios. The use of VR head-mounted displays in combination with physiological measures such as EEG presents new challenges and raises the question whether established findings also generalize to a VR setup. Here, we used a VR headset to assess the spatial constraints underlying two well-established EEG correlates of visual short-term memory: the amplitude of the contralateral delay activity (CDA) and the lateralization of induced alpha power during memory retention. We tested observers' visual memory in a change detection task with bilateral stimulus arrays of either two or four items while varying the horizontal eccentricity of the memory arrays (4, 9, or 14 degrees of visual angle). The CDA amplitude differed between high and low memory load at the two smaller eccentricities, but not at the largest eccentricity. Neither memory load nor eccentricity significantly influenced the observed alpha lateralization. We further fitted time-resolved spatial filters to decode memory load from the event-related potential as well as from its time-frequency decomposition. Classification performance during the retention interval was above-chance level for both approaches and did not vary significantly across eccentricities. We conclude that commercial VR hardware can be utilized to study the CDA and lateralized alpha power, and we provide caveats for future studies targeting these EEG markers of visual memory in a VR setup.

Recognition of motor intentions from EEGs of the same upper limb by signal traceability and Riemannian geometry features

Introduction

The electroencephalographic (EEG) based on the motor imagery task is derived from the physiological electrical signal caused by the autonomous activity of the brain. Its weak potential difference changes make it easy to be overwhelmed by noise, and the EEG acquisition method has a natural limitation of low spatial resolution. These have brought significant obstacles to high-precision recognition, especially the recognition of the motion intention of the same upper limb.

Methods

This research proposes a method that combines signal traceability and Riemannian geometric features to identify six motor intentions of the same upper limb, including grasping/holding of the palm, flexion/extension of the elbow, and abduction/adduction of the shoulder. First, the EEG data of electrodes irrelevant to the task were screened out by low-resolution brain electromagnetic tomography. Subsequently, tangential spatial features are extracted by the Riemannian geometry framework in the covariance matrix estimated from the reconstructed EEG signals. The learned Riemannian geometric features are used for pattern recognition by a support vector machine with a linear kernel function.

Results

The average accuracy of the six classifications on the data set of 15 participants is 22.47%, the accuracy is 19.34% without signal traceability, the accuracy is 18.07% when the features are the filter bank common spatial pattern (FBCSP), and the accuracy is 16.7% without signal traceability and characterized by FBCSP.

Discussion

The results show that the proposed method can significantly improve the accuracy of intent recognition. In addressing the issue of temporal variability in EEG data for active Brain-Machine Interfaces, our method achieved an average standard deviation of 2.98 through model transfer on different days' data.

Real-time assessment of hypnotic depth, using an EEG-based brain-computer interface: a preliminary study

OBJECTIVE

Hypnosis can be an effective treatment for many conditions, and there have been attempts to develop instrumental approaches to continuously monitor hypnotic state level ("depth"). However, there is no method that addresses the individual variability of electrophysiological hypnotic correlates. We explore the possibility of using an EEG-based passive brain-computer interface (pBCI) for real-time, individualised estimation of the hypnosis deepening process.

RESULTS

The wakefulness and deep hypnosis intervals were manually defined and labelled in 27 electroencephalographic (EEG) recordings obtained from eight outpatients after hypnosis sessions. Spectral analysis showed that EEG correlates of deep hypnosis were relatively stable in each patient throughout the treatment but varied between patients. Data from each first session was used to train classification models to continuously assess deep hypnosis probability in subsequent sessions. Models trained using four frequency bands (1.5-45, 1.5-8, 1.5-14, and 4-15 Hz) showed accuracy mostly exceeding 85% in a 10-fold cross-validation. Real-time classification accuracy was also acceptable, so at least one of the four bands yielded results exceeding 74% in any session. The best results averaged across all sessions were obtained using 1.5-14 and 4-15 Hz, with an accuracy of 82%. The revealed issues are also discussed.

A Multi-Domain Convolutional Neural Network for EEG-Based Motor Imagery Decoding

Motor imagery (MI) decoding plays a crucial role in the advancement of electroencephalography (EEG)-based brain-computer interface (BCI) technology. Currently, most researches focus on complex deep learning structures for MI decoding. The growing complexity of networks may result in overfitting and lead to inaccurate decoding outcomes due to the redundant information. To address this limitation and make full use of the multi-domain EEG features, a multi-domain temporal-spatial-frequency convolutional neural network (TSFCNet) is proposed for MI decoding. The proposed network provides a novel mechanism that utilize the spatial and temporal EEG features combined with frequency and time-frequency characteristics. This network enables powerful feature extraction without complicated network structure. Specifically, the TSFCNet first employs the MixConv-Residual block to extract multiscale temporal features from multi-band filtered EEG data. Next, the temporal-spatial-frequency convolution block implements three shallow, parallel and independent convolutional operations in spatial, frequency and time-frequency domain, and captures high discriminative representations from these domains respectively. Finally, these features are effectively aggregated by average pooling layers and variance layers, and the network is trained with the joint supervision of the cross-entropy and the center loss. Our experimental results show that the TSFCNet outperforms the state-of-the-art models with superior classification accuracy and kappa values (82.72% and 0.7695 for dataset BCI competition IV 2a, 86.39% and 0.7324 for dataset BCI competition IV 2b). These competitive results demonstrate that the proposed network is promising for enhancing the decoding performance of MI BCIs.

Novel channel selection model based on graph convolutional network for motor imagery

Multi-channel electroencephalography (EEG) is used to capture features associated with motor imagery (MI) based brain-computer interface (BCI) with a wide spatial coverage across the scalp. However, redundant EEG channels are not conducive to improving BCI performance. Therefore, removing irrelevant channels can help improve the classification performance of BCI systems. We present a new method for identifying relevant EEG channels. Our method is based on the assumption that useful channels share related information and that this can be measured by inter-channel connectivity. Specifically, we treat all candidate EEG channels as a graph and define channel selection as the problem of node classification on a graph. Then we design a graph convolutional neural network (GCN) model for channels classification. Channels are selected based on the outputs of our GCN model. We evaluate our proposed GCN-based channel selection (GCN-CS) method on three MI datasets. On three datasets, GCN-CS achieves performance improvements by reducing the number of channels. Specifically, we achieve classification accuracies of 79.76% on Dataset 1, 89.14% on Dataset 2 and 87.96% on Dataset 3, which outperform competing methods significantly.

Online semi-supervised learning for motor imagery EEG classification

OBJECTIVE

Time-consuming data labeling in brain-computer interfaces (BCIs) raises many problems such as mental fatigue and is one key factor that hinders the real-world adoption of motor imagery (MI)-based BCIs. An alternative approach is to integrate readily available, as well as informative, unlabeled data online, whereas this approach is less investigated.

APPROACH

We proposed an online semi-supervised learning scheme to improve the classification performance of MI-based BCI. This scheme uses regularized weighted online sequential extreme learning machine (RWOS-ELM) as the base classifier and updates its model parameters with incoming balanced data chunk-by-chunk. In the initial stage, we designed a technique that combines the synthetic minority oversampling with the edited nearest neighbor rule for data augmentation to construct more discriminative initial classifiers. When used online, the incoming chunk of data is first pseudo-labeled by RWOS-ELM as well as an auxiliary classifier, and then balanced again by the above-mentioned technique. Initial classifiers are further updated based on these class-balanced data.

MAIN RESULTS

Offline experimental results on two publicly available MI datasets demonstrate the superiority of the proposed scheme over its counterparts. Further online experiments on six subjects show that their BCI performance gradually improved by learning from incoming unlabeled data.

SIGNIFICANCE

Our proposed online semi-supervised learning scheme has higher computation and memory usage efficiency, which is promising for online MI-based BCIs, especially in the case of insufficient labeled training data.

Motor intent recognition of multi-feature fusion EEG signals by UMAP algorithm

The key to the analysis of electroencephalogram (EEG) signals lies in the extraction of effective features from the raw EEG signals, which can then be utilized to augment the classification accuracy of motor imagery (MI) applications in brain-computer interface (BCI). It can be argued that the utilization of features from multiple domains can be a more effective approach to feature extraction for MI pattern classification, as it can provide a more comprehensive set of information that the traditional single feature extraction method may not be able to capture. In this paper, a multi-feature fusion algorithm based on uniform manifold approximate and projection (UMAP) is proposed for motor imagery EEG signals. The brain functional network and common spatial pattern (CSP) are initially extracted as features. Subsequently, UMAP is utilized to fuse the extracted multi-domain features to generate low-dimensional features with improved discriminative capability. Finally, the k-nearest neighbor (KNN) classifier is applied in a lower dimensional space. The proposed method is evaluated using left-right hand EEG signals, and achieved the average accuracy of over 92%. The results indicate that, compared with single-domain-based feature extraction methods, multi-feature fusion EEG signal classification based on the UMAP algorithm yields superior classification and visualization performance. Feature extraction and fusion based on UMAP algorithm of left-right hand motor imagery.

Challenges of neural interfaces for stroke motor rehabilitation

More than 85% of stroke survivors suffer from different degrees of disability for the rest of their lives. They will require support that can vary from occasional to full time assistance. These conditions are also associated to an enormous economic impact for their families and health care systems. Current rehabilitation treatments have limited efficacy and their long-term effect is controversial. Here we review different challenges related to the design and development of neural interfaces for rehabilitative purposes. We analyze current bibliographic evidence of the effect of neuro-feedback in functional motor rehabilitation of stroke patients. We highlight the potential of these systems to reconnect brain and muscles. We also describe all aspects that should be taken into account to restore motor control. Our aim with this work is to help researchers designing interfaces that demonstrate and validate neuromodulation strategies to enforce a contingent and functional neural linkage between the central and the peripheral nervous system. We thus give clues to design systems that can improve or/and re-activate neuroplastic mechanisms and open a new recovery window for stroke patients.

Predicting Motor Imagery BCI Performance Based on EEG Microstate Analysis

Motor imagery (MI) electroencephalography (EEG) is natural and comfortable for controllers, and has become a research hotspot in the field of the brain-computer interface (BCI). Exploring the inter-subject MI-BCI performance variation is one of the fundamental problems in MI-BCI application. EEG microstates with high spatiotemporal resolution and multichannel information can represent brain cognitive function. In this paper, four EEG microstates (MS1, MS2, MS3, MS4) were used in the analysis of the differences in the subjects' MI-BCI performance, and the four microstate feature parameters (the mean duration, the occurrences per second, the time coverage ratio, and the transition probability) were calculated. The correlation between the resting-state EEG microstate feature parameters and the subjects' MI-BCI performance was measured. Based on the negative correlation of the occurrence of MS1 and the positive correlation of the mean duration of MS3, a resting-state microstate predictor was proposed. Twenty-eight subjects were recruited to participate in our MI experiments to assess the performance of our resting-state microstate predictor. The experimental results show that the average area under curve (AUC) value of our resting-state microstate predictor was 0.83, and increased by 17.9% compared with the spectral entropy predictor, representing that the microstate feature parameters can better fit the subjects' MI-BCI performance than spectral entropy predictor. Moreover, the AUC of microstate predictor is higher than that of spectral entropy predictor at both the single-session level and average level. Overall, our resting-state microstate predictor can help MI-BCI researchers better select subjects, save time, and promote MI-BCI development.

A large EEG database with users' profile information for motor imagery brain-computer interface research

We present and share a large database containing electroencephalographic signals from 87 human participants, collected during a single day of brain-computer interface (BCI) experiments, organized into 3 datasets (A, B, and C) that were all recorded using the same protocol: right and left hand motor imagery (MI). Each session contains 240 trials (120 per class), which represents more than 20,800 trials, or approximately 70 hours of recording time. It includes the performance of the associated BCI users, detailed information about the demographics, personality profile as well as some cognitive traits and the experimental instructions and codes (executed in the open-source platform OpenViBE). Such database could prove useful for various studies, including but not limited to: (1) studying the relationships between BCI users' profiles and their BCI performances, (2) studying how EEG signals properties varies for different users' profiles and MI tasks, (3) using the large number of participants to design cross-user BCI machine learning algorithms or (4) incorporating users' profile information into the design of EEG signal classification algorithms.

Multi-Band CNN With Band-Dependent Kernels and Amalgamated Cross Entropy Loss for Motor Imagery Classification

In this paper, we present a novel MI classification method based on multi-band convolutional neural network (CNN) with band-dependent kernel sizes, named MBK-CNN, to improve classification performance, by resolving the subject dependency issue of the widely used CNN-based approaches due to the kernel size optimization problem. The proposed structure exploits the frequency diversity of the EEG signals and simultaneously resolves the subject dependent kernel size issue. EEG signal is decomposed into overlapping multi-band and passed through multiple CNNs (termed 'branch-CNNs') with different kernel sizes to generate frequency dependent features, which are combined by a simple weighted sum. In contrast to the existing works where single-band multi-branch CNNs with different kernel sizes are used to resolve the subject dependency issue, a unique kernel size per frequency band is used. To prevent possible overfitting induced by a weighted sum, each branch-CNN is additionally trained by tentative cross entropy loss while overall network is optimized by the end-to-end cross entropy loss, which is named amalgamated cross entropy loss. In addition, we further propose multi-band CNN with enhanced spatial diversity, named MBK-LR-CNN, by replacing each branch-CNN with several sub branch-CNNs applied for channel subsets (termed 'local region') to improve the classification performance. We evaluated the performance of the proposed methods, MBK-CNN and MBK-LR-CNN, on publicly available datasets, BCI Competition IV dataset 2a and High Gamma Dataset. The experimental results confirm the performance improvement of the proposed methods compared to the currently existing MI classification methods.

A Temporal Dependency Learning CNN With Attention Mechanism for MI-EEG Decoding

Deep learning methods have been widely explored in motor imagery (MI)-based brain computer interface (BCI) systems to decode electroencephalography (EEG) signals. However, most studies fail to fully explore temporal dependencies among MI-related patterns generated in different stages during MI tasks, resulting in limited MI-EEG decoding performance. Apart from feature extraction, learning temporal dependencies is equally important to develop a subject-specific MI-based BCI because every subject has their own way of performing MI tasks. In this paper, a novel temporal dependency learning convolutional neural network (CNN) with attention mechanism is proposed to address MI-EEG decoding. The network first learns spatial and spectral information from multi-view EEG data via the spatial convolution block. Then, a series of non-overlapped time windows is employed to segment the output data, and the discriminative feature is further extracted from each time window to capture MI-related patterns generated in different stages. Furthermore, to explore temporal dependencies among discriminative features in different time windows, we design a temporal attention module that assigns different weights to features in various time windows and fuses them into more discriminative features. The experimental results on the BCI Competition IV-2a (BCIC-IV-2a) and OpenBMI datasets show that our proposed network outperforms the state-of-the-art algorithms and achieves the average accuracy of 79.48%, improved by 2.30% on the BCIC-IV-2a dataset. We demonstrate that learning temporal dependencies effectively improves MI-EEG decoding performance. The code is available at https://github.com/Ma-Xinzhi/LightConvNet.

A novel noninvasive brain-computer interface by imagining isometric force levels

Physiological circuits differ across increasing isometric force levels during unilateral contraction. Therefore, we first explored the possibility of predicting the force level based on electroencephalogram (EEG) activity recorded during a single trial of unilateral 5% or 40% of maximal isometric voluntary contraction (MVC) in right-hand grip imagination. Nine healthy subjects were involved in this study. The subjects were required to randomly perform 20 trials for each force level while imagining a right-hand grip. We proposed the use of common spatial patterns (CSPs) and coherence between EEG signals as features in a support vector machine for force level prediction. The results showed that the force levels could be predicted through single-trial EEGs while imagining the grip (mean accuracy = 81.4 ± 13.29%). Additionally, we tested the possibility of online control of a ball game using the above paradigm through unilateral grip imagination at different force levels (i.e., 5% of MVC imagination and 40% of MVC imagination for right-hand movement control). Subjects played the ball games effectively by controlling direction with our novel BCI system (n = 9, mean accuracy = 76.67 ± 9.35%). Data analysis validated the use of our BCI system in the online control of a ball game. This information may provide additional commands for the control of robots by users through combinations with other traditional brain-computer interfaces, e.g., different limb imaginations.

MSATNet: multi-scale adaptive transformer network for motor imagery classification

Motor imagery brain-computer interface (MI-BCI) can parse user motor imagery to achieve wheelchair control or motion control for smart prostheses. However, problems of poor feature extraction and low cross-subject performance exist in the model for motor imagery classification tasks. To address these problems, we propose a multi-scale adaptive transformer network (MSATNet) for motor imagery classification. Therein, we design a multi-scale feature extraction (MSFE) module to extract multi-band highly-discriminative features. Through the adaptive temporal transformer (ATT) module, the temporal decoder and multi-head attention unit are used to adaptively extract temporal dependencies. Efficient transfer learning is achieved by fine-tuning target subject data through the subject adapter (SA) module. Within-subject and cross-subject experiments are performed to evaluate the classification performance of the model on the BCI Competition IV 2a and 2b datasets. The MSATNet outperforms benchmark models in classification performance, reaching 81.75 and 89.34% accuracies for the within-subject experiments and 81.33 and 86.23% accuracies for the cross-subject experiments. The experimental results demonstrate that the proposed method can help build a more accurate MI-BCI system.

An EEG database for the cognitive assessment of motor imagery during walking with a lower-limb exoskeleton

One important point in the development of a brain-machine Interface (BMI) commanding an exoskeleton is the assessment of the cognitive engagement of the subject during the motor imagery tasks conducted. However, there are not many databases that provide electroencephalography (EEG) data during the use of a lower-limb exoskeleton. The current paper presents a database designed with an experimental protocol aiming to assess not only motor imagery during the control of the device, but also the attention to gait on flat and inclined surfaces. The research was conducted as an EUROBENCH subproject in the facilities sited in Hospital Los Madroños, Brunete (Madrid). The data validation reaches accuracies over 70% in the assessment of motor imagery and attention to gait, which marks the present database as a valuable resource for researches interested on developing and testing new EEG-based BMIs.

Subject-independent EEG classification based on a hybrid neural network

A brain-computer interface (BCI) based on the electroencephalograph (EEG) signal is a novel technology that provides a direct pathway between human brain and outside world. For a traditional subject-dependent BCI system, a calibration procedure is required to collect sufficient data to build a subject-specific adaptation model, which can be a huge challenge for stroke patients. In contrast, subject-independent BCI which can shorten or even eliminate the pre-calibration is more time-saving and meets the requirements of new users for quick access to the BCI. In this paper, we design a novel fusion neural network EEG classification framework that uses a specially designed generative adversarial network (GAN), called a filter bank GAN (FBGAN), to acquire high-quality EEG data for augmentation and a proposed discriminative feature network for motor imagery (MI) task recognition. Specifically, multiple sub-bands of MI EEG are first filtered using a filter bank approach, then sparse common spatial pattern (CSP) features are extracted from multiple bands of filtered EEG data, which constrains the GAN to maintain more spatial features of the EEG signal, and finally we design a convolutional recurrent network classification method with discriminative features (CRNN-DF) to recognize MI tasks based on the idea of feature enhancement. The hybrid neural network proposed in this study achieves an average classification accuracy of 72.74 ± 10.44% (mean ± std) in four-class tasks of BCI IV-2a, which is 4.77% higher than the state-of-the-art subject-independent classification method. A promising approach is provided to facilitate the practical application of BCI.

Brain-computer interface enhanced by virtual reality training for controlling a lower limb exoskeleton

This study explores the use of a brain-computer interface (BCI) based on motor imagery (MI) for the control of a lower limb exoskeleton to aid in motor recovery after a neural injury. The BCI was evaluated in ten able-bodied subjects and two patients with spinal cord injuries. Five able-bodied subjects underwent a virtual reality (VR) training session to accelerate training with the BCI. Results from this group were compared with a control group of five able-bodied subjects, and it was found that the employment of shorter training by VR did not reduce the effectiveness of the BCI and even improved it in some cases. Patients gave positive feedback about the system and were able to handle experimental sessions without reaching high levels of physical and mental exertion. These results are promising for the inclusion of BCI in rehabilitation programs, and future research should investigate the potential of the MI-based BCI system.

Transformer-Based Network with Optimization for Cross-Subject Motor Imagery Identification

Exploring the effective signal features of electroencephalogram (EEG) signals is an important issue in the research of brain-computer interface (BCI), and the results can reveal the motor intentions that trigger electrical changes in the brain, which has broad research prospects for feature extraction from EEG data. In contrast to previous EEG decoding methods that are based solely on a convolutional neural network, the traditional convolutional classification algorithm is optimized by combining a transformer mechanism with a constructed end-to-end EEG signal decoding algorithm based on swarm intelligence theory and virtual adversarial training. The use of a self-attention mechanism is studied to expand the receptive field of EEG signals to global dependence and train the neural network by optimizing the global parameters in the model. The proposed model is evaluated on a real-world public dataset and achieves the highest average accuracy of 63.56% in cross-subject experiments, which is significantly higher than that found for recently published algorithms. Additionally, good performance is achieved in decoding motor intentions. The experimental results show that the proposed classification framework promotes the global connection and optimization of EEG signals, which can be further applied to other BCI tasks.

Functional-oriented, portable brain-computer interface training for hand motor recovery after stroke: a randomized controlled study

Background

Brain-computer interfaces (BCIs) have been proven to be effective for hand motor recovery after stroke. Facing kinds of dysfunction of the paretic hand, the motor task of BCIs for hand rehabilitation is relatively single, and the operation of many BCI devices is complex for clinical use. Therefore, we proposed a functional-oriented, portable BCI equipment and explored the efficiency of hand motor recovery after a stroke.

Materials and methods

Stroke patients were randomly assigned to the BCI group and the control group. The BCI group received BCI-based grasp/open motor training, while the control group received task-oriented guidance training. Both groups received 20 sessions of motor training in 4 weeks, and each session lasted for 30 min. The Fugl-Meyer assessment of the upper limb (FMA-UE) was applied for the assessment of rehabilitation outcomes, and the EEG signals were obtained for processing.

Results

The progress of FMA-UE between the BCI group [10.50 (5.75, 16.50)] and the control group [5.00 (4.00, 8.00)] was significantly different (Z = -2.834, P = 0.005). Meanwhile, the FMA-UE of both groups improved significantly (P < 0.001). A total of 24 patients in the BCI group achieved the minimal clinically important difference (MCID) of FMA-UE with an effective rate of 80%, and 16 in the control group achieved the MCID, with an effective rate of 51.6%. The lateral index of the open task in the BCI group was significantly decreased (Z = -2.704, P = 0.007). The average BCI accuracy for 24 stroke patients in 20 sessions was 70.7%, which was improved by 5.0% in the final session compared with the first session.

Conclusion

Targeted hand movement and two motor task modes, namely grasp and open, to be applied in a BCI design may be suitable in stroke patients with hand dysfunction. The functional-oriented, portable BCI training can promote hand recovery after a stroke, and it is expected to be widely used in clinical practice. The lateral index change of inter-hemispheric balance may be the mechanism of motor recovery.

Trial registration number

ChiCTR2100044492.

Correlation between the ERD in grasp/open tasks of BCIs and hand function of stroke patients: a cross-sectional study

BACKGROUND AND AIMS

Brain-computer interfaces (BCIs) are emerging as a promising tool for upper limb recovery after stroke, and motor tasks are an essential part of BCIs for patient training and control of rehabilitative/assistive BCIs. However, the correlation between brain activation with different levels of motor impairment and motor tasks in BCIs is still not so clear. Thus, we aim to compare the brain activation of different levels of motor impairment in performing the hand grasping and opening tasks in BCIs.

METHODS

We instructed stroke patients to perform motor attempts (MA) to grasp and open the affected hand for 30 trials, respectively. During this period, they underwent EEG acquisition and BCIs accuracy recordings. They also received detailed history records and behavioral scale assessments (the Fugl-Meyer assessment of upper limb, FMA-UE).

RESULTS

The FMA-UE was negatively correlated with the event-related desynchronization (ERD) of the affected hemisphere during open MA (R = - 0.423, P = 0.009) but not with grasp MA (R = - 0.058, P = 0.733). Then we divided the stroke patients into group 1 (Brunnstrom recovery stages between I to II, n = 19) and group 2 (Brunnstrom recovery stages between III to VI, n = 23). No difference during the grasping task (t = 0.091, P = 0.928), but a significant difference during the open task (t = 2.156, P = 0.037) was found between the two groups on the affected hemisphere. No significant difference was found in the unaffected hemisphere.

CONCLUSIONS

The study indicated that brain activation is positively correlated with the hand function of stroke in open-hand tasks. In the grasping task, the patients in the different groups have a similar brain response, while in the open task, mildly injured patients have more brain activation in open the hand than the poor hand function patients.

Enhancing motor imagery detection efficacy using multisensory virtual reality priming

Brain-computer interfaces (BCI) have been developed to allow users to communicate with the external world by translating brain activity into control signals. Motor imagery (MI) has been a popular paradigm in BCI control where the user imagines movements of e.g., their left and right limbs and classifiers are then trained to detect such intent directly from electroencephalography (EEG) signals. For some users, however, it is difficult to elicit patterns in the EEG signal that can be detected with existing features and classifiers. As such, new user control strategies and training paradigms have been highly sought-after to help improve motor imagery performance. Virtual reality (VR) has emerged as one potential tool where improvements in user engagement and level of immersion have shown to improve BCI accuracy. Motor priming in VR, in turn, has shown to further enhance BCI accuracy. In this pilot study, we take the first steps to explore if multisensory VR motor priming, where haptic and olfactory stimuli are present, can improve motor imagery detection efficacy in terms of both improved accuracy and faster detection. Experiments with 10 participants equipped with a biosensor-embedded VR headset, an off-the-shelf scent diffusion device, and a haptic glove with force feedback showed that significant improvements in motor imagery detection could be achieved. Increased activity in the six common spatial pattern filters used were also observed and peak accuracy could be achieved with analysis windows that were 2 s shorter. Combined, the results suggest that multisensory motor priming prior to motor imagery could improve detection efficacy.

Review of public motor imagery and execution datasets in brain-computer interfaces

The demand for public datasets has increased as data-driven methodologies have been introduced in the field of brain-computer interfaces (BCIs). Indeed, many BCI datasets are available in various platforms or repositories on the web, and the studies that have employed these datasets appear to be increasing. Motor imagery is one of the significant control paradigms in the BCI field, and many datasets related to motor tasks are open to the public already. However, to the best of our knowledge, these studies have yet to investigate and evaluate the datasets, although data quality is essential for reliable results and the design of subject- or system-independent BCIs. In this study, we conducted a thorough investigation of motor imagery/execution EEG datasets recorded from healthy participants published over the past 13 years. The 25 datasets were collected from six repositories and subjected to a meta-analysis. In particular, we reviewed the specifications of the recording settings and experimental design, and evaluated the data quality measured by classification accuracy from standard algorithms such as Common Spatial Pattern (CSP) and Linear Discriminant Analysis (LDA) for comparison and compatibility across the datasets. As a result, we found that various stimulation types, such as text, figure, or arrow, were used to instruct subjects what to imagine and the length of each trial also differed, ranging from 2.5 to 29 s with a mean of 9.8 s. Typically, each trial consisted of multiple sections: pre-rest (2.38 s), imagination ready (1.64 s), imagination (4.26 s, ranging from 1 to 10 s), the post-rest (3.38 s). In a meta-analysis of the total of 861 sessions from all datasets, the mean classification accuracy of the two-class (left-hand vs. right-hand motor imagery) problem was 66.53%, and the population of the BCI poor performers, those who are unable to reach proficiency in using a BCI system, was 36.27% according to the estimated accuracy distribution. Further, we analyzed the CSP features and found that each dataset forms a cluster, and some datasets overlap in the feature space, indicating a greater similarity among them. Finally, we checked the minimal essential information (continuous signals, event type/latency, and channel information) that should be included in the datasets for convenient use, and found that only 71% of the datasets met those criteria. Our attempts to evaluate and compare the public datasets are timely, and these results will contribute to understanding the dataset's quality and recording settings as well as the use of using public datasets for future work on BCIs.

CatSight, a direct path to proper multi-variate time series change detection: perceiving a concept drift through common spatial pattern

Detecting changes in data streams, with the data flowing continuously, is an important problem which Industry 4.0 has to deal with. In industrial monitoring, the data distribution may vary after a change in the machine’s operating point; this situation is known as concept drift, and it is key to detecting this change. One drawback of conventional machine learning algorithms is that they are usually static, trained offline, and require monitoring at the input level. A change in the distribution of data, in the relationship between the input and the output data, would result in the deterioration of the predictive performance of the models due to the lack of an ability to generalize the model to new concepts. Drift detecting methods emerge as a solution to identify the concept drift in the data. This paper proposes a new approach for concept drift detection—a novel approach to deal with sudden or abrupt drift, the most common drift found in industrial processes-, called CatSight. Briefly, this method is composed of two steps: (i) Use of Common Spatial Patterns (a statistical approach to deal with data streaming, closely related to Principal Component Analysis) to maximize the difference between two different distributions of a multivariate temporal data, and (ii) Machine Learning conventional algorithms to detect whether a change in the data flow has been occurred or not. The performance of the CatSight method, has been evaluated on a real use case, training six state of the art Machine Learning (ML) classifiers; obtained results indicate how adequate the new approach is.

Improvement of brain-computer interface in motor imagery training through the designing of a dynamic experiment and FBCSP

Motor imagery (MI) can produce a specific brain pattern when the subject imagines performing a particular action without any actual body movements. According to related previous research, the improvement of the training of MI brainwaves can be adopted by feedback methods in which the analysis of brainwave characteristics is very important. The aim of this study was to improve the subject's MI and the accuracy of classification. In order to ameliorate the accuracy of the MI of the left and right hand, the present study designed static and dynamic visual stimuli in experiments so as to evaluate which one can improve subjects' imagination training. Additionally, the filter bank common spatial pattern (FBCSP) method was used to divide the frequency band range of the brainwaves into multiple segments, following which linear discriminant analysis (LDA) was adopted for classification. The results revealed that the averaged false positive rate (FPR) under FBCSP-LDA in the dynamic MI experiment was the lowest FPR (23.76%). As such, this study suggested that a combination of the dynamic MI experiment and the FBCSP-LDA method improved the overall prediction error rate and ameliorated the performance of the MI brain-computer interface.

Comparison of subject-independent and subject-specific EEG-based BCI using LDA and SVM classifiers

Motor imagery brain-computer interface (MI-BCI) is one of the most used paradigms in EEG-based brain-computer interface (BCI). The current state-of-the-art in BCI involves tuning classifiers to subject-specific training data, acquired over several sessions, in order to perform calibration prior to actual use of the so-called subject-specific BCI system (SS-BCI). Herein, the goal is to provide a ready-to-use system requiring minimal effort for setup. Thus, our challenge was to design a subject-independent BCI (SI-BCI) to be used by any new user without the constraint of individual calibration. Outcomes from other studies with the same purpose were used to undertake comparisons and validate our findings. For the EEG signal processing, we used a combination of the delta (0.5-4 Hz), alpha (8-13 Hz), and beta+gamma (13-40 Hz) bands at a stage prior to feature extraction. Next, we extracted features from the 27-channel EEG using common spatial pattern (CSP) and performed binary classification (MI of right- and left-hand) with linear discriminant analysis (LDA) and support vector machine (SVM) classifiers. These analyses were done for both the SS-BCI and SI-BCI models. We employed "leave-one-subject-out" (LOSO) arrangement and 10-fold cross-validation to evaluate our SI-BCI and SS-BCI systems, respectively. Compared with other two studies, our work was the only one that showed higher accuracy for the LDA classifier in SI-BCI as compared to SS-BCI. On the other hand, LDA accuracy was lower than accuracy achieved with SVM in both conditions (SI-BCI and SS-BCI). Our SS-BCI accuracy reached 76.85% using LDA and 94.20% using SVM and for SI-BCI we got 80.30% with LDA and 83.23% with SVM. We conclude that SI-BCI may be a feasible and relevant option, which can be used in scenarios where subjects are not able to submit themselves to long training sessions or to fast evaluation of the so called "BCI illiteracy." Comparatively, our strategy proved to be more efficient, giving us the best result for SI-BCI when faced against the classification performances of other three studies, even considering the caveat that different datasets were used in the comparison of the four studies.

A Model Combining Multi Branch Spectral-Temporal CNN, Efficient Channel Attention, and LightGBM for MI-BCI Classification

Accurately decoding motor imagery (MI) brain-computer interface (BCI) tasks has remained a challenge for both neuroscience research and clinical diagnosis. Unfortunately, less subject information and low signal-to-noise ratio of MI electroencephalography (EEG) signals make it difficult to decode the movement intentions of users. In this study, we proposed an end-to-end deep learning model, a multi-branch spectral-temporal convolutional neural network with channel attention and LightGBM model (MBSTCNN-ECA-LightGBM), to decode MI-EEG tasks. We first constructed a multi branch CNN module to learn spectral-temporal domain features. Subsequently, we added an efficient channel attention mechanism module to obtain more discriminative features. Finally, LightGBM was applied to decode the MI multi-classification tasks. The within-subject cross-session training strategy was used to validate classification results. The experimental results showed that the model achieved an average accuracy of 86% on the two-class MI-BCI data and an average accuracy of 74% on the four-class MI-BCI data, which outperformed current state-of-the-art methods. The proposed MBSTCNN-ECA-LightGBM can efficiently decode the spectral and temporal domain information of EEG, improving the performance of MI-based BCIs.

Classification of Motor Imagery EEG Signals Based on Data Augmentation and Convolutional Neural Networks

In brain-computer interface (BCI) systems, motor imagery electroencephalography (MI-EEG) signals are commonly used to detect participant intent. Many factors, including low signal-to-noise ratios and few high-quality samples, make MI classification difficult. In order for BCI systems to function, MI-EEG signals must be studied. In pattern recognition and other fields, deep learning approaches have recently been successfully applied. In contrast, few effective deep learning algorithms have been applied to BCI systems, especially MI-based systems. In this paper, we address these problems from two aspects based on the characteristics of EEG signals: first, we proposed a combined time-frequency domain data enhancement method. This method guarantees that the size of the training data is effectively increased while maintaining the intrinsic composition of the data. Second, our design consists of a parallel CNN that takes both raw EEG images and images transformed through continuous wavelet transform (CWT) as inputs. We conducted classification experiments on a public data set to verify the effectiveness of the algorithm. According to experimental results based on the BCI Competition IV Dataset2a, the average classification accuracy is 97.61%. A comparison of the proposed algorithm with other algorithms shows that it performs better in classification. The algorithm can be used to improve the classification performance of MI-based BCIs and BCI systems created for people with disabilities.

Classification of Motor-Imagery Tasks Using a Large EEG Dataset by Fusing Classifiers Learning on Wavelet-Scattering Features

Brain-computer or brain-machine interface technology allows humans to control machines using their thoughts via brain signals. In particular, these interfaces can assist people with neurological diseases for speech understanding or physical disabilities for operating devices such as wheelchairs. Motor-imagery tasks play a basic role in brain-computer interfaces. This study introduces an approach for classifying motor-imagery tasks in a brain-computer interface environment, which remains a challenge for rehabilitation technology using electroencephalogram sensors. Methods used and developed for addressing the classification include wavelet time and image scattering networks, fuzzy recurrence plots, support vector machines, and classifier fusion. The rationale for combining outputs from two classifiers learning on wavelet-time and wavelet-image scattering features of brain signals, respectively, is that they are complementary and can be effectively fused using a novel fuzzy rule-based system. A large-scale challenging electroencephalogram dataset of motor imagery-based brain-computer interface was used to test the efficacy of the proposed approach. Experimental results obtained from within-session classification show the potential application of the new model that achieves an improvement of 7% in classification accuracy over the best existing classifier using state-of-the-art artificial intelligence (76% versus 69%, respectively). For the cross-session experiment, which imposes a more challenging and practical classification task, the proposed fusion model improves the accuracy by 11% (54% versus 65%). The technical novelty presented herein and its further exploration are promising for developing a reliable sensor-based intervention for assisting people with neurodisability to improve their quality of life.

Towards Practical BCI-Driven Wheelchairs: A Systematic Review Study

The use of brain signals in controlling wheelchairs is a promising solution for many disabled individuals, specifically those who are suffering from motor neuron disease affecting the proper functioning of their motor units. Almost two decades since the first work, the applicability of EEG-driven wheelchairs is still limited to laboratory environments. In this work, a systematic review study has been conducted to identify the state-of-the-art and the different models adopted in the literature. Furthermore, a strong emphasis is devoted to introducing the challenges impeding a broad use of the technology as well as the latest research trends in each of those areas.

Double Stage Transfer Learning for Brain-Computer Interfaces

In the application of brain-computer interfaces (BCIs), electroencephalogram (EEG) signals are difficult to collect in large quantities due to the non-stationary nature and long calibration time required. Transfer learning (TL), which transfers knowledge learned from existing subjects to new subjects, can be applied to solve this problem. Some existing EEG-based TL algorithms cannot achieve good results because they only extract partial features. To achieve effective transfer, a double-stage transfer learning (DSTL) algorithm which applied transfer learning to both preprocessing stage and feature extraction stage of typical BCIs was proposed. First, Euclidean alignment (EA) was used to align EEG trials from different subjects. Second, aligned EEG trials in the source domain were reweighted by the distance between the covariance matrix of each trial in the source domain and the mean covariance matrix of the target domain. Lastly, after extracting spatial features with common spatial patterns (CSP), transfer component analysis (TCA) was adopted to reduce the differences between different domains further. Experiments on two public datasets in two transfer paradigms (multi-source to single-target (MTS) and single-source to single-target (STS)) verified the effectiveness of the proposed method. The proposed DSTL achieved better classification accuracy on two datasets: 84.64% and 77.16% in MTS, 73.38% and 68.58% in STS, which shows that DSTL performs better than other state-of-the-art methods. The proposed DSTL can reduce the difference between the source domain and the target domain, providing a new method for EEG data classification without training dataset.

Intra- and Inter-Subject Common Spatial Pattern for Reducing Calibration Effort in MI-Based BCI

One major problem limiting the practicality of a brain-computer interface (BCI) is the need for large amount of labeled data to calibrate its classification model. Although the effectiveness of transfer learning (TL) for conquering this problem has been evidenced by many studies, a highly recognized approach has not yet been established. In this paper, we propose a Euclidean alignment (EA)-based intra- and inter-subject common spatial pattern (EA-IISCSP) algorithm for estimating four spatial filters, which aim at exploiting intra- and inter-subject similarities and variability to enhance the robustness of feature signals. Based on the algorithm, a TL-based classification framework was developed for enhancing the performance of motor imagery (MI) BCIs, in which the feature vector extracted by each filter is dimensionally reduced by linear discriminant analysis (LDA) and a support vector machine (SVM) is used for classification. The performance of the proposed algorithm was evaluated on two MI data sets and compared with that of three state-of-the-art TL algorithms. Experimental results showed that the proposed algorithm significantly outperforms these competing algorithms for training trials per class from 15 to 50 and can reduce the amount of training data while maintaining an acceptable accuracy, thus facilitating the practical application of MI-based BCIs.

Mutual Information-Driven Subject-Invariant and Class-Relevant Deep Representation Learning in BCI

In recent years, deep learning-based feature representation methods have shown a promising impact on electroencephalography (EEG)-based brain-computer interface (BCI). Nonetheless, owing to high intra- and inter-subject variabilities, many studies on decoding EEG were designed in a subject-specific manner by using calibration samples, with no concern of its practical use, hampered by time-consuming steps and a large data requirement. To this end, recent studies adopted a transfer learning strategy, especially domain adaptation techniques. Among those, we have witnessed the potential of adversarial learning-based transfer learning in BCIs. In the meantime, it is known that adversarial learning-based domain adaptation methods are prone to negative transfer that disrupts learning generalized feature representations, applicable to diverse domains, for example, subjects or sessions in BCIs. In this article, we propose a novel framework that learns class-relevant and subject-invariant feature representations in an information-theoretic manner, without using adversarial learning. To be specific, we devise two operational components in a deep network that explicitly estimate mutual information between feature representations: 1) to decompose features in an intermediate layer into class-relevant and class-irrelevant ones and 2) to enrich class-discriminative feature representation. On two large EEG datasets, we validated the effectiveness of our proposed framework by comparing with several comparative methods in performance. Furthermore, we conducted rigorous analyses by performing an ablation study in regard to the components in our network, explaining our model's decision on input EEG signals via layer-wise relevance propagation, and visualizing the distribution of learned features via t-SNE.

Targeting targeted memory reactivation: Characteristics of cued reactivation in sleep

Targeted memory reactivation (TMR) is a technique in which sensory cues associated with memories during wake are used to trigger memory reactivation during subsequent sleep. The characteristics of such cued reactivation, and the optimal placement of TMR cues, remain to be determined. We built an EEG classification pipeline that discriminated reactivation of right- and left-handed movements and found that cues which fall on the up-going transition of the slow oscillation (SO) are more likely to elicit a classifiable reactivation. We also used a novel machine learning pipeline to predict the likelihood of eliciting a classifiable reactivation after each TMR cue using the presence of spindles and features of SOs. Finally, we found that reactivations occurred either immediately after the cue or one second later. These findings greatly extend our understanding of memory reactivation and pave the way for development of wearable technologies to efficiently enhance memory through cueing in sleep.

Subject Separation Network for Reducing Calibration Time of MI-Based BCI

Motor imagery brain-computer interface (MI-based BCIs) have demonstrated great potential in various applications. However, to well generalize classifiers to new subjects, a time-consuming calibration process is necessary due to high inter-subject variabilities of EEG signals. This process is costly and tedious, hindering the further expansion of MI-based BCIs outside of the laboratory. To reduce the calibration time of MI-based BCIs, we propose a novel domain adaptation framework that adapts multiple source subjects' labeled data to the unseen trials of target subjects. Firstly, we train one Subject Separation Network(SSN) for each of the source subjects in the dataset. Based on adversarial domain adaptation, a shared encoder is constructed to learn similar representations for both domains. Secondly, to model the factors that cause subject variabilities and eliminate the correlated noise existing in common feature space, private feature spaces orthogonal to the shared counterpart are learned for each subject. We use a shared decoder to validate that the model is actually learning from task-relevant neurophysiological information. At last, an ensemble classifier is built by the integration of the SSNs using the information extracted from each subject's task-relevant characteristics. To quantify the efficacy of the framework, we analyze the accuracy-calibration cost trade-off in MI-based BCIs, and theoretically guarantee a generalization bound on the target error. Visualizations of the transformed features illustrate the effectiveness of domain adaptation. The experimental results on the BCI Competition IV-IIa dataset prove the effectiveness of the proposed framework compared with multiple classification methods. We infer from our results that users could learn to control MI-based BCIs without a heavy calibration process. Our study further shows how to design and train Neural Networks to decode task-related information from different subjects and highlights the potential of deep learning methods for inter-subject EEG decoding.

Enhancement of motor imagery training efficiency by an online adaptive training paradigm integrated with error related potential

Objective. Motor imagery (MI) is a process of autonomously modulating the motor area to rehearse action mentally without actual execution. Based on the neuroplasticity of the cerebral cortex, MI can promote the functional rehabilitation of the injured cerebral cortex motor area. However, it usually takes several days to a few months to train individuals to acquire the necessary MI ability to control rehabilitation equipment in current studies, which greatly limits the clinical application of rehabilitation training systems based on the MI brain-computer interface (BCI).Approach. A novel MI training paradigm combined with the error related potential (ErrP) is proposed, and online adaptive training of the MI classifier was performed using ErrP. ErrP is used to correct the output of the MI classification to obtain a higher accuracy of kinesthetic feedback based on the imagination intention of subjects while generating simulated labels for MI online adaptive training. In this way, we improved the MI training efficiency. Thirteen subjects were randomly divided into an experimental group using the proposed paradigm and a control group using the traditional MI training paradigm to participate in six MI training experiments.Main results. The proposed paradigm enabled the experimental group to obtain a higher event-related desynchronization modulation level in the contralateral brain region compared with the control group and 69.76% online classification accuracy of MI after three MI training experiments. The online classification accuracy reached 72.76% and the whole system recognized the MI intention of the subjects with an online accuracy of 82.61% after six experiments.Significance. Compared with the conventional unimodal MI training strategy, the proposed approach enables subjects to use the MI-BCI based system directly and achieve a better performance after only three training experiments with training left and right hands simultaneously. This greatly improves the usability of the MI-BCI-based rehabilitation system and makes it more convenient for clinical use.

Robust Motor Imagery Tasks Classification Approach Using Bayesian Neural Network

The development of Brain-Computer Interfaces based on Motor Imagery (MI) tasks is a relevant research topic worldwide. The design of accurate and reliable BCI systems remains a challenge, mainly in terms of increasing performance and usability. Classifiers based on Bayesian Neural Networks are proposed in this work by using the variational inference, aiming to analyze the uncertainty during the MI prediction. An adaptive threshold scheme is proposed here for MI classification with a reject option, and its performance on both datasets 2a and 2b from BCI Competition IV is compared with other approaches based on thresholds. The results using subject-specific and non-subject-specific training strategies are encouraging. From the uncertainty analysis, considerations for reducing computational cost are proposed for future work.

Classification of BCI Multiclass Motor Imagery Task Based on Artificial Neural Network

Motor imagery (MI) signals recorded by electroencephalography provide the most practical basis for conceiving brain-computer interfaces (BCI). These interfaces offer a high degree of freedom. This helps people with motor disabilities communicate with the device by tackling a sequence of motor imagery tasks. However, the extracting user-specific features and increasing the accuracy of the classifier remain as difficult tasks in MI-based BCI. In this work, we propose a new method using artificial neural network (ANN) enhancing the performance of the motor imagery classification. Feature extraction techniques, like time domain parameters, band power features, signal power features, and wavelet packet decomposition (WPD), are studied and compared. Four classification algorithms are implemented which are Quadratic Discriminant Analysis, k-Nearest Neighbors, Linear Discriminant Analysis, and proposed ANN architecture. We added Batch Normalization layers to the proposed ANN architecture to improve the learning time and accuracy of the neural network. These layers also alleviate the effect of weight initialization and the addition of a regularization effect on the network. Our proposed method using ANN architecture achieves 0.5545 of kappa and 58.42% of accuracy on the BCI Competition IV-2a dataset. Our results show that the modified ANN method, with frequency and spatial features extracted by WPD and Common Spatial Pattern, respectively, offers a better classification compared to other current methods.