Optimal spatial filtering of single trial EEG during imagined hand movement

Multiscale time-frequency method for multiclass Motor Imagery Brain Computer Interface

Motor Imagery Brain Computer Interface (MI-BCI) has become a promising technology in the field of neurorehabilitation. However, the performance and computational complexity of the current multiclass MI-BCI have not been fully optimized, and the intuitive interpretation of individual differences on motor imagery tasks is seldom investigated. In this paper, a well-designed multiscale time-frequency segmentation scheme is first applied to multichannel EEG recordings to obtain Time-Frequency Segments (TFSs). Then, the TFS selection based on a specific wrapper feature selection rule is utilized to determine optimum TFSs. Next, One-Versus-One (OvO)-divCSP implemented in divergence framework is used to extract discriminative features. Finally, One-Versus-Rest (OvR)-SVM is utilized to predict the class label based on selected multiclass MI features. Experimental results indicate our method yields a superior performance on two publicly available multiclass MI datasets with a mean accuracy of 80.00% and a mean kappa of 0.73. Meanwhile, the proposed TFS selection method can significantly alleviate the computational burden with little accuracy reduction, demonstrating the feasibility of real-time multiclass MI-BCI. Furthermore, the Motor Imagery Time-Frequency Reaction Map (MI-TFRM) is visualized, contributing to analyzing and interpreting the performance differences between different subjects.

Evaluation of Real-Time Endogenous Brain-Computer Interface Developed Using Ear-Electroencephalography

While previous studies have demonstrated the feasibility of using ear-electroencephalography (ear-EEG) for the development of brain-computer interfaces (BCIs), most of them have been performed using exogenous paradigms in offline environments. To verify the reliable feasibility of constructing ear-EEG-based BCIs, the feasibility of using ear-EEG should be further demonstrated using another BCI paradigm, namely the endogenous paradigm, in real-time online environments. Exogenous and endogenous BCIs are to use the EEG evoked by external stimuli and induced by self-modulation, respectively. In this study, we investigated whether an endogenous ear-EEG-based BCI with reasonable performance can be implemented in online environments that mimic real-world scenarios. To this end, we used three different mental tasks, i.e., mental arithmetic, word association, and mental singing, and performed BCI experiments with fourteen subjects on three different days to investigate not only the reliability of a real-time endogenous ear-EEG-based BCI, but also its test-retest reliability. The mean online classification accuracy was almost 70%, which was equivalent to a marginal accuracy for a practical two-class BCI (70%), demonstrating the feasibility of using ear-EEG for the development of real-time endogenous BCIs, but further studies should follow to improve its performance enough to be used for practical ear-EEG-based BCI applications.

Enhancing the decoding accuracy of EEG signals by the introduction of anchored-STFT and adversarial data augmentation method

Brain-computer interfaces (BCIs) enable communication between humans and machines by translating brain activity into control commands. Electroencephalography (EEG) signals are one of the most used brain signals in non-invasive BCI applications but are often contaminated with noise. Therefore, it is possible that meaningful patterns for classifying EEG signals are deeply hidden. State-of-the-art deep-learning algorithms are successful in learning hidden, meaningful patterns. However, the quality and the quantity of the presented inputs are pivotal. Here, we propose a feature extraction method called anchored Short Time Fourier Transform (anchored-STFT), which is an advanced version of STFT, as it minimizes the trade-off between temporal and spectral resolution presented by STFT. In addition, we propose a data augmentation method derived from l2-norm fast gradient sign method (FGSM), called gradient norm adversarial augmentation (GNAA). GNAA is not only an augmentation method but is also used to harness adversarial inputs in EEG data, which not only improves the classification accuracy but also enhances the robustness of the classifier. In addition, we also propose a CNN architecture, namely Skip-Net, for the classification of EEG signals. The proposed pipeline outperforms the current state-of-the-art methods and yields classification accuracies of 90.7% on BCI competition II dataset III and 89.5%, 81.8%, 76.0% and 85.4%, 69.1%, 80.9% on different data distributions of BCI Competition IV dataset 2b and 2a, respectively.

A Two-Branch CNN Fusing Temporal and Frequency Features for Motor Imagery EEG Decoding

With the development of technology and the rise of the meta-universe concept, the brain-computer interface (BCI) has become a hotspot in the research field, and the BCI based on motor imagery (MI) EEG has been widely concerned. However, in the process of MI-EEG decoding, the performance of the decoding model needs to be improved. At present, most MI-EEG decoding methods based on deep learning cannot make full use of the temporal and frequency features of EEG data, which leads to a low accuracy of MI-EEG decoding. To address this issue, this paper proposes a two-branch convolutional neural network (TBTF-CNN) that can simultaneously learn the temporal and frequency features of EEG data. The structure of EEG data is reconstructed to simplify the spatio-temporal convolution process of CNN, and continuous wavelet transform is used to express the time-frequency features of EEG data. TBTF-CNN fuses the features learned from the two branches and then inputs them into the classifier to decode the MI-EEG. The experimental results on the BCI competition IV 2b dataset show that the proposed model achieves an average classification accuracy of 81.3% and a kappa value of 0.63. Compared with other methods, TBTF-CNN achieves a better performance in MI-EEG decoding. The proposed method can make full use of the temporal and frequency features of EEG data and can improve the decoding accuracy of MI-EEG.

Time-Distributed Attention Network for EEG-based Motor Imagery Decoding from the Same Limb

A brain-computer interface (BCI) based on motor imagery (MI) from the same limb can provide an intuitive control pathway but has received limited attention. It is still a challenge to classify multiple MI tasks from the same limb. The goal of this study is to propose a novel decoding method to classify the MI tasks of four joints of the same upper limb and the resting state. EEG signals were collected from 20 participants. A time-distributed attention network (TD-Atten) was proposed to adaptively assign different weights to different classes and frequency bands of the input multiband Common Spatial Pattern (CSP) features. The long short-term memory (LSTM) and dense layers were then used to learn sequential information from the reweight features and perform the classification. Our proposed method outperformed other baseline and deep learning-based methods and obtained the accuracies of 46.8% in the 5-class scenario and 53.4% in the 4-class scenario. The visualization results of attention weights indicated that the proposed framework can adaptively pay attention to alpha-band related features in MI tasks, which was consistent with the analysis of brain activation patterns. These results demonstrated the feasibility and interpretability of the attention mechanism in MI decoding and the potential of this fine MI paradigm to be applied for the control of a robotic arm or a neural prosthesis.

Evaluation of classification approaches for distinguishing brain states predictive of episodic memory performance from electroencephalography: Abbreviated Title: Evaluating methods of classifying memory states from EEG

Previous studies have attempted to separate single trial neural responses for events a person is likely to remember from those they are likely to forget using machine learning classification methods. Successful single trial classification holds potential for translation into the clinical realm for real-time detection of memory and other cognitive states to provide real-time interventions (i.e., brain-computer interfaces). However, most of these studies-and classification analyses in general- do not make clear if the chosen methodology is optimally suited for the classification of memory-related brain states. To address this problem, we systematically compared different methods for every step of classification (i.e., feature extraction, feature selection, classifier selection) to investigate which methods work best for decoding episodic memory brain states-the first analysis of its kind. Using an adult lifespan sample EEG dataset collected during performance of an episodic context encoding and retrieval task, we found that no specific feature type (including Common Spatial Pattern (CSP)-based features, mean, variance, correlation, features based on AR model, entropy, phase, and phase synchronization) outperformed others consistently in distinguishing different memory classes. However, extracting all of these feature types consistently outperformed extracting only one type of feature. Additionally, the combination of filtering and sequential forward selection was the optimal method to select the effective features compared to filtering alone or performing no feature selection at all. Moreover, although all classifiers performed at a fairly similar level, LASSO was consistently the highest performing classifier compared to other commonly used options (i.e., naïve Bayes, SVM, and logistic regression) while naïve Bayes was the fastest classifier. Lastly, for multiclass classification (i.e., levels of context memory confidence and context feature perception), generalizing the binary classification using the binary decision tree performed better than the voting or one versus rest method. These methods were shown to outperform alternative approaches for three orthogonal datasets (i.e., EEG working memory, EEG motor imagery, and MEG working memory), supporting their generalizability. Our results provide an optimized methodological process for classifying single-trial neural data and provide important insight and recommendations for a cognitive neuroscientist's ability to make informed choices at all stages of the classification process for predicting memory and other cognitive states.

Reducing calibration time in motor imagery-based BCIs by data alignment and empirical mode decomposition

One of the major reasons that limit the practical applications of a brain-computer interface (BCI) is its long calibration time. In this paper, we propose a novel approach to reducing the calibration time of motor imagery (MI)-based BCIs without sacrificing classification accuracy. The approach aims to augment the training set size of a new subject by generating artificial electroencephalogram (EEG) data from a few training trials initially available. The artificial EEG data are obtained by first performing empirical mode decomposition (EMD) and then mixing resulting intrinsic mode functions (IMFs). The original training trials are aligned to common reference point with Euclidean alignment (EA) method prior to EMD and pooled together with artificial trials as the expended training set, which is input into a linear discriminant analysis (LDA) classifier or a logistic regression (LR) classifier. The performance of the proposed algorithm is evaluated on two motor imagery (MI) data sets and compared with that of the algorithm trained with only real EEG data (Baseline) and the algorithm trained with expanded EEG data by EMD without data alignment. The experimental results showed that the proposed algorithm can significantly reduce the amount of training data needed to achieve a given performance level and thus is expected to facilitate the real-world applications of MI-based BCIs.

Cognitive Workload of Tugboat Captains in Realistic Scenarios: Adaptive Spatial Filtering for Transfer Between Conditions

Changing and often class-dependent non-stationarities of signals are a big challenge in the transfer of common findings in cognitive workload estimation using Electroencephalography (EEG) from laboratory experiments to realistic scenarios or other experiments. Additionally, it often remains an open question whether actual cognitive workload reflected by brain signals was the main contribution to the estimation or discriminative and class-dependent muscle and eye activity, which can be secondary effects of changing workload levels. Within this study, we investigated a novel approach to spatial filtering based on beamforming adapted to changing settings. We compare it to no spatial filtering and Common Spatial Patterns (CSP). We used a realistic maneuvering task, as well as an auditory n-back secondary task on a tugboat simulator as two different conditions to induce workload changes on professional tugboat captains. Apart from the typical within condition classification, we investigated the ability of the different classification methods to transfer between the n-back condition and the maneuvering task. The results show a clear advantage of the proposed approach over the others in the challenging transfer setting. While no filtering leads to lowest within-condition normalized classification loss on average in two scenarios (22 and 10%), our approach using adaptive beamforming (30 and 18%) performs comparably to CSP (33 and 15%). Importantly, in the transfer from one to another setting, no filtering and CSP lead to performance around chance level (45 to 53%), while our approach in contrast is the only one capable of classifying in all other scenarios (34 and 35%) with a significant difference from chance level. The changing signal composition over the scenarios leads to a need to adapt the spatial filtering in order to be transferable. With our approach, the transfer is successful due to filtering being optimized for the extraction of neural components and additional investigation of their scalp patterns revealed mainly neural origin. Interesting findings are that rather the patterns slightly change between conditions. We conclude that the approaches with low normalized loss depend on eye and muscle activity which is successful for classification within conditions, but fail in the classifier transfer since eye and muscle contributions are highly condition-specific.

A Collaborative Brain-Computer Interface Framework for Enhancing Group Detection Performance of Dynamic Visual Targets

The superiority of collaborative brain-computer interface (cBCI) in performance enhancement makes it an effective way to break through the performance bottleneck of the BCI-based dynamic visual target detection. However, the existing cBCIs focus on multi-mind information fusion with a static and unidirectional mode, lacking the information interaction and learning guidance among multiple agents. Here, we propose a novel cBCI framework to enhance the group detection performance of dynamic visual targets. Specifically, a mutual learning domain adaptation network (MLDANet) with information interaction, dynamic learning, and individual transferring abilities is developed as the core of the cBCI framework. MLDANet takes P3-sSDA network as individual network unit, introduces mutual learning strategy, and establishes a dynamic interactive learning mechanism between individual networks and collaborative decision-making at the neural decision level. The results indicate that the proposed MLDANet-cBCI framework can achieve the best group detection performance, and the mutual learning strategy can improve the detection ability of individual networks. In MLDANet-cBCI, the F1 scores of collaborative detection and individual network are 0.12 and 0.19 higher than those in the multi-classifier cBCI, respectively, when three minds collaborate. Thus, the proposed framework breaks through the traditional multi-mind collaborative mode and exhibits a superior group detection performance of dynamic visual targets, which is also of great significance for the practical application of multi-mind collaboration.

STAnet: A Spatiotemporal Attention Network for Decoding Auditory Spatial Attention from EEG

OBJECTIVE

Humans are able to localize the source of a sound. This enables them to direct attention to a particular speaker in a cocktail party. Psycho-acoustic studies show that the sensory cortices of the human brain respond to the location of sound sources differently, and the auditory attention itself is a dynamic and temporally based brain activity. In this work, we seek to build a computational model which uses both spatial and temporal information manifested in EEG signals for auditory spatial attention detection (ASAD).

METHODS

We propose an end-to-end spatiotemporal attention network, denoted as STAnet, to detect auditory spatial attention from EEG. The STAnet is designed to assign differentiated weights dynamically to EEG channels through a spatial attention mechanism, and to temporal patterns in EEG signals through a temporal attention mechanism.

RESULTS

We report the ASAD experiments on two publicly available datasets. The STAnet outperforms other competitive models by a large margin under various experimental conditions. Its attention decision for 1-second decision window outperforms that of the state-of-the-art techniques for 10-second decision window. Experimental results also demonstrate that the STAnet achieves competitive performance on EEG signals ranging from 64 to as few as 16 channels.

CONCLUSION

This study provides evidence suggesting that efficient low-density EEG online decoding is within reach.

SIGNIFICANCE

This study also marks an important step towards the practical implementation of ASAD in real life applications.

Depression Diagnosis Modeling With Advanced Computational Methods: Frequency-Domain eMVAR and Deep Learning

Electroencephalogram (EEG)-based automated depression diagnosis systems have been suggested for early and accurate detection of mood disorders. EEG signals are highly irregular, nonlinear, and nonstationary in nature and are traditionally studied from a linear viewpoint by means of statistical and frequency features. Since, linear metrics present certain limitations and nonlinear methods have proven to be an efficient tool in understanding the complexities of the brain in the identification of underlying behavior of biological signals, such as electrocardiogram, EEG and magnetoencephalogram and thus, can be applied to all nonstationary signals. Various nonlinear algorithms can be used in the analysis of EEG signals. In this research paper, we aim to develop a novel methodology for EEG-based depression diagnosis utilizing 2 advanced computational techniques: frequency-domain extended multivariate autoregressive (eMVAR) and deep learning (DL). We proposed a hybrid method comprising a pretrained ResNet-50 and long-short term memory (LSTM) to capture depression-specific information and compared with a strong conventional machine learning (ML) framework having eMVAR connectivity features. The following 8 causality measures, which interpret the interaction mechanisms among spectrally decomposed oscillations, were used to extract features from multivariate EEG time series: directed coherence (DC), directed transfer function (DTF), partial DC (PDC), generalized PDC (gPDC), extended DC (eDC), delayed DC (dDC), extended PDC (ePDC), and delayed PDC (dPDC). The classification accuracies were 84% with DC, 85% with DTF, 95.3% with PDC, 95.1% with gPDC, 84.8% with eDC, 84.6% with dDC, 84.2% with ePDC, and 95.9% with dPDC for the eMVAR framework. Through a DL framework (ResNet-50 + LSTM), the classification accuracy was achieved as 90.22%. The results demonstrate that our DL methodology is a competitive alternative to the strong feature extraction-based ML methods in depression classification.

Multiple graph fusion based on Riemannian geometry for motor imagery classification

In motor imagery-based brain-computer interfaces (BCIs), the spatial covariance features of electroencephalography (EEG) signals that lie on Riemannian manifolds are used to enhance the classification performance of motor imagery BCIs. However, the problem of subject-specific bandpass frequency selection frequently arises in Riemannian manifold-based methods. In this study, we propose a multiple Riemannian graph fusion (MRGF) model to optimize the subject-specific frequency band for a Riemannian manifold. After constructing multiple Riemannian graphs corresponding to multiple bandpass frequency bands, graph embedding based on bilinear mapping and graph fusion based on mutual information were applied to simultaneously extract the spatial and spectral features of the EEG signals from Riemannian graphs. Furthermore, with a support vector machine (SVM) classifier performed on learned features, we obtained an efficient algorithm, which achieves higher classification performance on various datasets, such as BCI competition IIa and in-house BCI datasets. The proposed methods can also be used in other classification problems with sample data in the form of covariance matrices.

Multi-view optimization of time-frequency common spatial patterns for brain-computer interfaces

BACKGROUND

Common spatial pattern (CSP) is a prevalent method applied to feature extraction in motor imagery (MI)-based brain-computer interfaces (BCIs) recorded by electroencephalogram (EEG). The selection of time windows and frequency bands prominently affects the performance of CSP algorithms. Concerning the joint optimization of these two parameters, several studies have utilized a unified framework based on different feature selection strategies and achieved considerable improvement. However, during the feature selection process, useful information could be discarded inevitably and the underlying internal structure of features could be neglected.

NEW METHOD

In this paper, we proposed a novel framework termed time window filter bank common spatial pattern with multi-view optimization (TWFBCSP-MVO) to further boost the decoding of MI tasks. Concretely, after extracting CSP features from different time-frequency decompositions of EEG signals, a preliminary screening strategy based on variance ratio was devised to filter out the unrelated spatial patterns. We then introduced a multi-view learning strategy for the simultaneous optimization of time windows and frequency bands. A support vector machine classifier was trained to determine the output of the brain.

RESULTS

An experimental study was conducted on two public datasets to verify the effectiveness of TWFBCSP-MVO. Results showed that the proposed TWFBCSP-MVO could help improve the performance of MI classification.

COMPARISON WITH EXISTING METHODS

In comparison to other competing methods, the proposed method performed significantly better (p<0.01).

CONCLUSIONS

The proposed method is a promising contestant to improve the performance of practical MI-based BCIs.

Efficacy of Brain-Computer Interface and the Impact of Its Design Characteristics on Poststroke Upper-limb Rehabilitation: A Systematic Review and Meta-analysis of Randomized Controlled Trials

Background. A number of recent randomized controlled trials reported the efficacy of brain-computer interface (BCI) for upper-limb stroke rehabilitation compared with other therapies. Despite the encouraging results reported, there is a significant variance in the reported outcomes. This paper aims to investigate the effectiveness of different BCI designs on poststroke upper-limb rehabilitation. Methods. The effect sizes of pooled and individual studies were assessed by computing Hedge's g values with a 95% confidence interval. Subgroup analyses were also performed to examine the impact of different BCI designs on the treatment effect. Results. The study included 12 clinical trials involving 298 patients. The analysis showed that the BCI yielded significant superior short-term and long-term efficacy in improving the upper-limb motor function compared to the control therapies (Hedge's g = 0.73 and 0.33, respectively). Based on our subgroup analyses, the BCI studies that used the intention of movement had a higher effect size compared to those used motor imagery (Hedge's g = 1.21 and 0.55, respectively). The BCI studies using band power features had a significantly higher effect size than those using filter bank common spatial patterns features (Hedge's g = 1.25 and - 0.23, respectively). Finally, the studies that used functional electrical stimulation as the BCI feedback had the highest effect size compared to other devices (Hedge's g = 1.2). Conclusion. This meta-analysis confirmed the effectiveness of BCI for upper-limb rehabilitation. Our findings support the use of band power features, the intention of movement, and the functional electrical stimulation in future BCI designs for poststroke upper-limb rehabilitation.

A Novel Classification Framework Using the Graph Representations of Electroencephalogram for Motor Imagery based Brain-Computer Interface

The motor imagery (MI) based brain-computer interfaces (BCIs) have been proposed as a potential physical rehabilitation technology. However, the low classification accuracy achievable with MI tasks is still a challenge when building effective BCI systems. We propose a novel MI classification model based on measurement of functional connectivity between brain regions and graph theory. Specifically, motifs describing local network structures in the brain are extracted from functional connectivity graphs. A graph embedding model called Ego-CNNs is then used to build a classifier, which can convert the graph from a structural representation to a fixed-dimensional vector for detecting critical structure in the graph. We validate our proposed method on four datasets, and the results show that our proposed method produces high classification accuracies in two-class classification tasks (92.8% for dataset 1, 93.4% for dataset 2, 96.5% for dataset 3, and 80.2% for dataset 4) and multiclass classification tasks (90.33% for dataset 1). Our proposed method achieves a mean Kappa value of 0.88 across nine participants, which is superior to other methods we compared it to. These results indicate that there is a local structural difference in functional connectivity graphs extracted under different motor imagery tasks. Our proposed method has great potential for motor imagery classification in future studies.

Artifacts in EEG-Based BCI Therapies: Friend or Foe?

EEG-based brain-computer interfaces (BCI) have promising therapeutic potential beyond traditional neurofeedback training, such as enabling personalized and optimized virtual reality (VR) neurorehabilitation paradigms where the timing and parameters of the visual experience is synchronized with specific brain states. While BCI algorithms are often designed to focus on whichever portion of a signal is most informative, in these brain-state-synchronized applications, it is of critical importance that the resulting decoder is sensitive to physiological brain activity representative of various mental states, and not to artifacts, such as those arising from naturalistic movements. In this study, we compare the relative classification accuracy with which different motor tasks can be decoded from both extracted brain activity and artifacts contained in the EEG signal. EEG data were collected from 17 chronic stroke patients while performing six different head, hand, and arm movements in a realistic VR-based neurorehabilitation paradigm. Results show that the artifactual component of the EEG signal is significantly more informative than brain activity with respect to classification accuracy. This finding is consistent across different feature extraction methods and classification pipelines. While informative brain signals can be recovered with suitable cleaning procedures, we recommend that features should not be designed solely to maximize classification accuracy, as this could select for remaining artifactual components. We also propose the use of machine learning approaches that are interpretable to verify that classification is driven by physiological brain states. In summary, whereas informative artifacts are a helpful friend in BCI-based communication applications, they can be a problematic foe in the estimation of physiological brain states.

Multi-Parameter Physiological State Monitoring in Target Detection Under Real-World Settings

Mental state changes induced by stimuli under experimental settings or by daily events in real life affect task performance and are entwined with physical and mental health. In this study, we developed a physiological state indicator with five parameters that reflect the subject's real-time physiological states based on online EEG signal processing. These five parameters are attention, fatigue, stress, and the brain activity shifts of the left and right hemispheres. We designed a target detection experiment modified by a cognitive attention network test for validating the effectiveness of the proposed indicator, as such conditions would better approximate a real chaotic environment. Results demonstrated that attention levels while performing the target detection task were significantly higher than during rest periods, but also exhibited a decay over time. In contrast, the fatigue level increased gradually and plateaued by the third rest period. Similar to attention levels, the stress level decreased as the experiment proceeded. These parameters are therefore shown to be highly correlated to different stages of the experiment, suggesting their usage as primary factors in passive brain-computer interfaces (BCI). In addition, the left and right brain activity indexes reveal the EEG neural modulations of the corresponding hemispheres, which set a feasible reference of activation for an active BCI control system, such as one executing motor imagery tasks. The proposed indicator is applicable to potential passive and active BCI applications for monitoring the subject's physiological state change in real-time, along with providing a means of evaluating the associated signal quality to enhance the BCI performance.

MIN2Net: End-to-End Multi-Task Learning for Subject-Independent Motor Imagery EEG Classification

OBJECTIVE

Advances in the motor imagery (MI)-based brain-computer interfaces (BCIs) allow control of several applications by decoding neurophysiological phenomena, which are usually recorded by electroencephalography (EEG) using a non-invasive technique. Despite significant advances in MI-based BCI, EEG rhythms are specific to a subject and various changes over time. These issues point to significant challenges to enhance the classification performance, especially in a subject-independent manner.

METHODS

To overcome these challenges, we propose MIN2Net, a novel end-to-end multi-task learning to tackle this task. We integrate deep metric learning into a multi-task autoencoder to learn a compact and discriminative latent representation from EEG and perform classification simultaneously.

RESULTS

This approach reduces the complexity in pre-processing, results in significant performance improvement on EEG classification. Experimental results in a subject-independent manner show that MIN2Net outperforms the state-of-the-art techniques, achieving an F1-score improvement of 6.72 %, and 2.23 % on the SMR-BCI, and OpenBMI datasets, respectively.

CONCLUSION

We demonstrate that MIN2Net improves discriminative information in the latent representation.

SIGNIFICANCE

This study indicates the possibility and practicality of using this model to develop MI-based BCI applications for new users without calibration.

Mental State Detection Using Riemannian Geometry on Electroencephalogram Brain Signals

The goal of this study was to implement a Riemannian geometry (RG)-based algorithm to detect high mental workload (MWL) and mental fatigue (MF) using task-induced electroencephalogram (EEG) signals. In order to elicit high MWL and MF, the participants performed a cognitively demanding task in the form of the letter n-back task. We analyzed the time-varying characteristics of the EEG band power (BP) features in the theta and alpha frequency band at different task conditions and cortical areas by employing a RG-based framework. MWL and MF were considered as too high, when the Riemannian distances of the task-run EEG reached or surpassed the threshold of the baseline EEG. The results of this study showed a BP increase in the theta and alpha frequency bands with increasing experiment duration, indicating elevated MWL and MF that impedes/hinders the task performance of the participants. High MWL and MF was detected in 8 out of 20 participants. The Riemannian distances also showed a steady increase toward the threshold with increasing experiment duration, with the most detections occurring toward the end of the experiment. To support our findings, subjective ratings (questionnaires concerning fatigue and workload levels) and behavioral measures (performance accuracies and response times) were also considered.

Optimal Selection of EEG Electrodes Using Interval Type-2 Fuzzy-Logic-Based Semiseparating Signaling Game

This article addresses the noise contamination in spatial filtering of brain responses using a novel signaling game-based approach to the optimal selection of EEG electrodes. The proposed method takes the standard common spatial pattern (CSP) filter as an input and produces an optimal electrode set as output for effective classification of different cognitive tasks. The standard CSP algorithms are highly prone to the inclusion of noise in the EEG data and may select noisy electrodes/signal sources that are redundant for a specific cognitive task which, in turn, may lead to a lower classification accuracy. A lot of literature exists in this area of research, most of which deals with adding the regularization term in the standard CSP algorithm. However, all of these methods lack capturing the uncertainty present in the EEG responses due to intrasession and intersession variations of subjective brain response. The novelty of this article lies in designing the fuzzy signaling game-based approach for optimal electrode selection using an interval type-2 fuzzy set, which can capture both the intrasession and intersession variability of EEG responses acquired from a subject's scalp. Experiments are undertaken over a wide variety of possible cognitive task classification problems which reveal that the proposed method yields superior results in electrode selection with respect to classification accuracy. Statistical tests undertaken using the Friedman test also confirm the superiority of the proposed method over its competitors.

Selective Cross-Subject Transfer Learning Based on Riemannian Tangent Space for Motor Imagery Brain-Computer Interface

A motor imagery (MI) brain-computer interface (BCI) plays an important role in the neurological rehabilitation training for stroke patients. Electroencephalogram (EEG)-based MI BCI has high temporal resolution, which is convenient for real-time BCI control. Therefore, we focus on EEG-based MI BCI in this paper. The identification of MI EEG signals is always quite challenging. Due to high inter-session/subject variability, each subject should spend long and tedious calibration time in collecting amounts of labeled samples for a subject-specific model. To cope with this problem, we present a supervised selective cross-subject transfer learning (sSCSTL) approach which simultaneously makes use of the labeled samples from target and source subjects based on Riemannian tangent space. Since the covariance matrices representing the multi-channel EEG signals belong to the smooth Riemannian manifold, we perform the Riemannian alignment to make the covariance matrices from different subjects close to each other. Then, all aligned covariance matrices are converted into the Riemannian tangent space features to train a classifier in the Euclidean space. To investigate the role of unlabeled samples, we further propose semi-supervised and unsupervised versions which utilize the total samples and unlabeled samples from target subject, respectively. Sequential forward floating search (SFFS) method is executed for source selection. All our proposed algorithms transfer the labeled samples from most suitable source subjects into the feature space of target subject. Experimental results on two publicly available MI datasets demonstrated that our algorithms outperformed several state-of-the-art algorithms using small number of the labeled samples from target subject, especially for good target subjects.

Mobile Electroencephalography for Studying Neural Control of Human Locomotion

Walking or running in real-world environments requires dynamic multisensory processing within the brain. Studying supraspinal neural pathways during human locomotion provides opportunities to better understand complex neural circuity that may become compromised due to aging, neurological disorder, or disease. Knowledge gained from studies examining human electrical brain dynamics during gait can also lay foundations for developing locomotor neurotechnologies for rehabilitation or human performance. Technical barriers have largely prohibited neuroimaging during gait, but the portability and precise temporal resolution of non-invasive electroencephalography (EEG) have expanded human neuromotor research into increasingly dynamic tasks. In this narrative mini-review, we provide a (1) brief introduction and overview of modern neuroimaging technologies and then identify considerations for (2) mobile EEG hardware, (3) and data processing, (4) including technical challenges and possible solutions. Finally, we summarize (5) knowledge gained from human locomotor control studies that have used mobile EEG, and (6) discuss future directions for real-world neuroimaging research.

A Generalizable and Discriminative Learning Method for Deep EEG-Based Motor Imagery Classification

Convolutional neural networks (CNNs) have been widely applied to the motor imagery (MI) classification field, significantly improving the state-of-the-art (SoA) performance in terms of classification accuracy. Although innovative model structures are thoroughly explored, little attention was drawn toward the objective function. In most of the available CNNs in the MI area, the standard cross-entropy loss is usually performed as the objective function, which only ensures deep feature separability. Corresponding to the limitation of current objective functions, a new loss function with a combination of smoothed cross-entropy (with label smoothing) and center loss is proposed as the supervision signal for the model in the MI recognition task. Specifically, the smoothed cross-entropy is calculated by the entropy between the predicted labels and the one-hot hard labels regularized by a noise of uniform distribution. The center loss learns a deep feature center for each class and minimizes the distance between deep features and their corresponding centers. The proposed loss tries to optimize the model in two learning objectives, preventing overconfident predictions and increasing deep feature discriminative capacity (interclass separability and intraclass invariant), which guarantee the effectiveness of MI recognition models. We conduct extensive experiments on two well-known benchmarks (BCI competition IV-2a and IV-2b) to evaluate our method. The result indicates that the proposed approach achieves better performance than other SoA models on both datasets. The proposed learning scheme offers a more robust optimization for the CNN model in the MI classification task, simultaneously decreasing the risk of overfitting and increasing the discriminative power of deeply learned features.

Improving Motor Imagery-Based Brain-Computer Interface Performance Based on Sensory Stimulation Training: An Approach Focused on Poorly Performing Users

The motor imagery (MI)-based brain-computer interface (BCI) is an intuitive interface that provides control over computer applications directly from brain activity. However, it has shown poor performance compared to other BCI systems such as P300 and SSVEP BCI. Thus, this study aimed to improve MI-BCI performance by training participants in MI with the help of sensory inputs from tangible objects (i.e., hard and rough balls), with a focus on poorly performing users. The proposed method is a hybrid of training and imagery, combining motor execution and somatosensory sensation from a ball-type stimulus. Fourteen healthy participants participated in the somatosensory-motor imagery (SMI) experiments (within-subject design) involving EEG data classification with a three-class system (signaling with left hand, right hand, or right foot). In the scenario of controlling a remote robot to move it to the target point, the participants performed MI when faced with a three-way intersection. The SMI condition had a better classification performance than did the MI condition, achieving a 68.88% classification performance averaged over all participants, which was 6.59% larger than that in the MI condition (p < 0.05). In poor performers, the classification performance in SMI was 10.73% larger than in the MI condition (62.18% vs. 51.45%). However, good performers showed a slight performance decrement (0.86%) in the SMI condition compared to the MI condition (80.93% vs. 81.79%). Combining the brain signals from the motor and somatosensory cortex, the proposed hybrid MI-BCI system demonstrated improved classification performance, this phenomenon was predominant in poor performers (eight out of nine subjects). Hybrid MI-BCI systems may significantly contribute to reducing the proportion of BCI-inefficiency users and closing the performance gap with other BCI systems.

The time-varying networks of the wrist extension in post-stroke hemiplegic patients

Hemiplegia is a common dysfunction caused by the brain stroke and leads to movement disability. Although the lateralization of movement-related potential, the event-related desynchronization, and more complicated inter-regional information coupling have been investigated, seldom studies have focused on investigating the dynamic information exchanging among multiple brain regions during motor execution for post-stroke hemiplegic patients. With high temporal-resolution electroencephalogram (EEG), the time-varying network is able to reflect the dynamical complex network modalities corresponding to the movements at a millisecond level. In our present study, the wrist extension experiment was designed, along with related EEG datasets being collected. Thereafter, the corresponding time-varying networks underlying the wrist extension were accordingly constructed by adopting the adaptive directed transfer function and then statistically explored, to further uncover the dynamic network deficits (i.e., motor dysfunction) in post-stroke hemiplegic patients. Results of this study found the effective connectivity between the stroked motor area and other areas decreased in patients when compared to healthy controls; on the contrary, the enhanced connectivity between non-stroked motor areas and other areas, especially the frontal and parietal-occipital lobes, were further identified for patients during their accomplishing the designed wrist extension, which might dynamically compensate for the deficited patients' motor behaviors. These findings not only helped deepen our knowledge of the mechanism underlying the patients' motor behaviors, but also facilitated the real-time strategies for clinical therapy of brain stroke, as well as providing a reliable biomarker to predict the future rehabilitation.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11571-021-09738-2.

The Paradigm Design of a Novel 2-class Unilateral Upper Limb Motor Imagery Tasks and its EEG Signal Classification

Multitasking motor imagery (MI) of the unilateral upper limb is potentially more valuable in stroke rehabilitation than the current conventional MI in both hands. In this paper, a novel experimental paradigm was designed to imagine two motions of unilateral upper limb, which is hand gripping and releasing, and elbow reciprocating left and right. During this experiment, the electroencephalogram (EEG) signals were collected from 10 subjects. The time and frequency domains of the EEG signals were analyzed and visualized, indicating the presence of different Event-Related Desynchronization (ERD) or Event-Related Synchronization (ERS) for the two tasks. Then the two tasks were classified through three different EEG decoding methods, in which the optimized convolutional neural network (CNN) based on FBCNet achieved an average accuracy of 67.8%, obtaining a good recognition result. This work not only can advance the studies of MI decoding of unilateral upper limb, but also can provide a basis for better upper limb stroke rehabilitation in MI-BCI.

Internal Feature Selection Method of CSP Based on L1-Norm and Dempster-Shafer Theory

The common spatial pattern (CSP) algorithm is a well-recognized spatial filtering method for feature extraction in motor imagery (MI)-based brain-computer interfaces (BCIs). However, due to the influence of nonstationary in electroencephalography (EEG) and inherent defects of the CSP objective function, the spatial filters, and their corresponding features are not necessarily optimal in the feature space used within CSP. In this work, we design a new feature selection method to address this issue by selecting features based on an improved objective function. Especially, improvements are made in suppressing outliers and discovering features with larger interclass distances. Moreover, a fusion algorithm based on the Dempster-Shafer theory is proposed, which takes into consideration the distribution of features. With two competition data sets, we first evaluate the performance of the improved objective functions in terms of classification accuracy, feature distribution, and embeddability. Then, a comparison with other feature selection methods is carried out in both accuracy and computational time. Experimental results show that the proposed methods consume less additional computational cost and result in a significant increase in the performance of MI-based BCI systems.

Filter Bank Sinc-ShallowNet with EMD-based Mixed Noise Adding Data Augmentation for Motor Imagery Classification

Motor imagery-based brain computer interface (MI-BCI) is a representative active BCI paradigm which is widely employed in the rehabilitation field. In MI-BCI, a classification model is built to identify the target limb from MI-based EEG signals, but the performance of models cannot meet the demand for practical use. Lightweight neural networks in deep learning methods are used to build high performance models in MI-BCI. Small sample sizes and the lack of multi-scale information extraction in frequency domain limit the performance improvement of lightweight neural networks. To solve these problems, the Filter Bank Sinc-ShallowNet (FB-Sinc-ShallowNet) algorithm combined with the mixed noise adding method based on empirical mode decomposition (EMD) was proposed. The FB-Sinc-ShallowNet algorithm improves a lightweight neural network Sinc-ShallowNet with a filter bank structure corresponding to four sensory motor rhythms. The mixed noise adding method employs the EMD method to improve the quality of generated data. The proposed method was evaluated on the BCI competition IV IIa dataset and can achieve highest average accuracy of 77.2%, about 6.34% higher than state-of-the-art method Sinc-ShallowNet. This work implies the effectiveness of filter bank structure in lightweight neural networks and provides a novel option for data augmentation and classification of MI-based EEG signals, which can be applied in the rehabilitation field for decoding MI-EEG with few samples.

Is Riemannian Geometry Better than Euclidean in Averaging Covariance Matrices for CSP-based Subject-Independent Classification of Motor Imagery?

Common Spatial Pattern (CSP) is a popular feature extraction algorithm used for electroencephalogram (EEG) data classification in brain-computer interfaces. One of the critical operations used in CSP is taking the average of trial covariance matrices for each class. In this regard, the arithmetic mean, which minimizes the sum of squared Euclidean distances to the data points, is conventionally used; however, this operation ignores the Riemannian geometry in the manifold of covariance matrices. To alleviate this problem, Fréchet mean determined using different Riemannian distances have been used. In this paper, we are primarily concerned with the following question: Does using the Fréchet mean with Riemannian distances instead of arithmetic mean in averaging CSP covariance matrices improve the subject-independent classification of motor imagery (MI)? To answer this question we conduct a comparative study using the largest MI dataset to date, with 54 subjects and a total of 21,600 trials of left-and right-hand MI. The results indicate a general trend of having a statistically significant better performance when the Riemannian geometry is used.

On the interpretation of linear Riemannian tangent space model parameters in M/EEG

Riemannian tangent space methods offer state-of-the-art performance in magnetoencephalography (MEG) and electroencephalography (EEG) based applications such as brain-computer interfaces and biomarker development. One limitation, particularly relevant for biomarker development, is limited model interpretability compared to established component-based methods. Here, we propose a method to transform the parameters of linear tangent space models into interpretable patterns. Using typical assumptions, we show that this approach identifies the true patterns of latent sources, encoding a target signal. In simulations and two real MEG and EEG datasets, we demonstrate the validity of the proposed approach and investigate its behavior when the model assumptions are violated. Our results confirm that Riemannian tangent space methods are robust to differences in the source patterns across observations. We found that this robustness property also transfers to the associated patterns.

Continuously Decoding Grasping Movements using Stereotactic Depth Electrodes

Brain-Computer Interfaces (BCIs) that decode a patient's movement intention to control a prosthetic device could restore some independence to paralyzed patients. An important step on the road towards naturalistic prosthetic control is to decode movement continuously with low-latency. BCIs based on intracortical micro-arrays provide continuous control of robotic arms, but require a minor craniotomy. Surface recordings of neural activity using EEG have made great advances over the last years, but suffer from high noise levels and large intra-session variance. Here, we investigate the use of minimally invasive recordings using stereotactically implanted EEG (sEEG). These electrodes provide a sparse sampling across many brain regions. So far, promising decoding results have been presented using data measured from the subthalamic nucleus or trial-to-trial based methods using depth electrodes. In this work, we demonstrate that grasping movements can continuously be decoded using sEEG electrodes, as well. Beta and high-gamma activity was extracted from eight participants performing a grasping task. We demonstrate above chance level decoding of movement vs rest and left vs right, from both frequency bands with accuracies up to 0.94 AUC. The vastly different electrode locations between participants lead to large variability. In the future, we hope that sEEG recordings will provide additional information for the decoding process in neuroprostheses.

Noise-assisted Multivariate Empirical Mode Decomposition based Causal Decomposition for Detecting Upper Limb Movement in EEG-EMG Hybrid Brain Computer Interface

EEG-EMG based hybrid Brain Computer Interface (hBCI) utilizes the brain-muscle physiological system to interpret and identify motor behaviors, and transmit human intelligence to automated machines in AI applications such as neurorehabilitations and brain-like intelligence. The study introduces a hBCI method for motor behaviors, where multiple time series of the brain neuromuscular network are introduced to indicate brain-muscle causal interactions, and features are extracted based on Relative Causal Strengths (RCSs) derived by Noise-assisted Multivariate Empirical Mode Decomposition (NA-MEMD) based Causal Decomposition. The complex process in brain neuromuscular interactions is specifically investigated towards a monitoring task of upper limb movement, whose 63-channel EEGs and 2-channel EMGs are composed of data inputs. The energy and frequency factors counted from RCSs were extracted as Core Features (CFs). Results showed accuracies of 91.4% and 81.4% with CFs for identifying cascaded (No Movement and Movement Execution) and 3-class (No Movement, Right Movement, and Left Movement) using Naive Bayes classifier, respectively. Moreover, those reached 100% and 94.3% when employing CFs combined with eigenvalues processed by Common Spatial Pattern (CSP). This initial work implies a novel causality inference based hBCI solution for the detection of human upper limb movement.

Decoding subjective emotional arousal from EEG during an immersive virtual reality experience

Immersive virtual reality (VR) enables naturalistic neuroscientific studies while maintaining experimental control, but dynamic and interactive stimuli pose methodological challenges. We here probed the link between emotional arousal, a fundamental property of affective experience, and parieto-occipital alpha power under naturalistic stimulation: 37 young healthy adults completed an immersive VR experience, which included rollercoaster rides, while their EEG was recorded. They then continuously rated their subjective emotional arousal while viewing a replay of their experience. The association between emotional arousal and parieto-occipital alpha power was tested and confirmed by (1) decomposing the continuous EEG signal while maximizing the comodulation between alpha power and arousal ratings and by (2) decoding periods of high and low arousal with discriminative common spatial patterns and a long short-term memory recurrent neural network. We successfully combine EEG and a naturalistic immersive VR experience to extend previous findings on the neurophysiology of emotional arousal towards real-world neuroscience.

A Review of the Role of Machine Learning Techniques towards Brain–Computer Interface Applications

This review article provides a deep insight into the Brain–Computer Interface (BCI) and the application of Machine Learning (ML) technology in BCIs. It investigates the various types of research undertaken in this realm and discusses the role played by ML in performing different BCI tasks. It also reviews the ML methods used for mental state detection, mental task categorization, emotion classification, electroencephalogram (EEG) signal classification, event-related potential (ERP) signal classification, motor imagery categorization, and limb movement classification. This work explores the various methods employed in BCI mechanisms for feature extraction, selection, and classification and provides a comparative study of reviewed methods. This paper assists the readers to gain information regarding the developments made in BCI and ML domains and future improvements needed for improving and designing better BCI applications.

An Optimal Transport Based Transferable System for Detection of Erroneous Somato-Sensory Feedback from Neural Signals

This study is aimed at the detection of single-trial feedback, perceived as erroneous by the user, using a transferable classification system while conducting a motor imagery brain-computer interfacing (BCI) task. The feedback received by the users are relayed from a functional electrical stimulation (FES) device and hence are somato-sensory in nature. The BCI system designed for this study activates an electrical stimulator placed on the left hand, right hand, left foot, and right foot of the user. Trials containing erroneous feedback can be detected from the neural signals in form of the error related potential (ErrP). The inclusion of neuro-feedback during the experiments indicated the possibility that ErrP signals can be evoked when the participant perceives an error from the feedback. Hence, to detect such feedback using ErrP, a transferable (offline) decoder based on optimal transport theory is introduced herein. The offline system detects single-trial erroneous trials from the feedback period of an online neuro-feedback BCI system. The results of the FES-based feedback BCI system were compared to a similar visual-based (VIS) feedback system. Using our framework, the error detector systems for both the FES and VIS feedback paradigms achieved an F1-score of 92.66% and 83.10%, respectively, and are significantly superior to a comparative system where an optimal transport was not used. It is expected that this form of transferable and automated error detection system compounded with a motor imagery system will augment the performance of a BCI and provide a better BCI-based neuro-rehabilitation protocol that has an error control mechanism embedded into it.

Enhancement of capability for motor imagery using vestibular imbalance stimulation during brain computer interface

Objective.Motor imagery (MI), based on the theory of mirror neurons and neuroplasticity, can promote motor cortical activation in neurorehabilitation. The strategy of MI based on brain-computer interface (BCI) has been used in rehabilitation training and daily assistance for patients with hemiplegia in recent years. However, it is difficult to maintain the consistency and timeliness of receiving external stimulation to neural activation in most subjects owing to the high variability of electroencephalogram (EEG) representation across trials/subjects. Moreover, in practical application, MI-BCI cannot highly activate the motor cortex and provide stable interaction owing to the weakness of the EEG feature and lack of an effective mode of activation.Approach.In this study, a novel hybrid BCI paradigm based on MI and vestibular stimulation motor imagery (VSMI) was proposed to enhance the capability of feature response for MI. Twelve subjects participated in a group of controlled experiments containing VSMI and MI. Three indicators, namely, activation degree, timeliness, and classification accuracy, were adopted to evaluate the performance of the task.Main results.Vestibular stimulation could significantly strengthen the suppression ofαandβbands of contralateral brain regions during MI, that is, enhance the activation degree of the motor cortex (p< 0.01). Compared with MI, the timeliness of EEG feature-response achieved obvious improvements in VSMI experiments. Moreover, the averaged classification accuracy of VSMI and MI was 80.56% and 69.38%, respectively.Significance.The experimental results indicate that specific vestibular activity contributes to the oscillations of the motor cortex and has a positive effect on spontaneous imagery, which provides a novel MI paradigm and enables the preliminary exploration of sensorimotor integration of MI.

Directional Decoding From EEG in a Center-Out Motor Imagery Task With Visual and Vibrotactile Guidance

Motor imagery is a popular technique employed as a motor rehabilitation tool, or to control assistive devices to substitute lost motor function. In both said areas of application, artificial somatosensory input helps to mirror the sensorimotor loop by providing kinesthetic feedback or guidance in a more intuitive fashion than via visual input. In this work, we study directional and movement-related information in electroencephalographic signals acquired during a visually guided center-out motor imagery task in two conditions, i.e., with and without additional somatosensory input in the form of vibrotactile guidance. Imagined movements to the right and forward could be discriminated in low-frequency electroencephalographic amplitudes with group level peak accuracies of 70% with vibrotactile guidance, and 67% without vibrotactile guidance. The peak accuracies with and without vibrotactile guidance were not significantly different. Furthermore, the motor imagery could be classified against a resting baseline with group level accuracies between 76 and 83%, using either low-frequency amplitude features or μ and β power spectral features. On average, accuracies were higher with vibrotactile guidance, while this difference was only significant in the latter set of features. Our findings suggest that directional information in low-frequency electroencephalographic amplitudes is retained in the presence of vibrotactile guidance. Moreover, they hint at an enhancing effect on motor-related μ and β spectral features when vibrotactile guidance is provided.

Multi-Domain Convolutional Neural Networks for Lower-Limb Motor Imagery Using Dry vs. Wet Electrodes

Motor imagery (MI) brain–computer interfaces (BCIs) have been used for a wide variety of applications due to their intuitive matching between the user’s intentions and the performance of tasks. Applying dry electroencephalography (EEG) electrodes to MI BCI applications can resolve many constraints and achieve practicality. In this study, we propose a multi-domain convolutional neural networks (MD-CNN) model that learns subject-specific and electrode-dependent EEG features using a multi-domain structure to improve the classification accuracy of dry electrode MI BCIs. The proposed MD-CNN model is composed of learning layers for three domain representations (time, spatial, and phase). We first evaluated the proposed MD-CNN model using a public dataset to confirm 78.96% classification accuracy for multi-class classification (chance level accuracy: 30%). After that, 10 healthy subjects participated and performed three classes of MI tasks related to lower-limb movement (gait, sitting down, and resting) over two sessions (dry and wet electrodes). Consequently, the proposed MD-CNN model achieved the highest classification accuracy (dry: 58.44%; wet: 58.66%; chance level accuracy: 43.33%) with a three-class classifier and the lowest difference in accuracy between the two electrode types (0.22%, d = 0.0292) compared with the conventional classifiers (FBCSP, EEGNet, ShallowConvNet, and DeepConvNet) that used only a single domain. We expect that the proposed MD-CNN model could be applied for developing robust MI BCI systems with dry electrodes.

Common Spatial Pattern Reformulated for Regularizations in Brain-Computer Interfaces

Common spatial pattern (CSP) is one of the most successful feature extraction algorithms for brain-computer interfaces (BCIs). It aims to find spatial filters that maximize the projected variance ratio between the covariance matrices of the multichannel electroencephalography (EEG) signals corresponding to two mental tasks, which can be formulated as a generalized eigenvalue problem (GEP). However, it is challenging in principle to impose additional regularization onto the CSP to obtain structural solutions (e.g., sparse CSP) due to the intrinsic nonconvexity and invariance property of GEPs. This article reformulates the CSP as a constrained minimization problem and establishes the equivalence of the reformulated and the original CSPs. An efficient algorithm is proposed to solve this optimization problem by alternately performing singular value decomposition (SVD) and least squares. Under this new formulation, various regularization techniques for linear regression can then be easily implemented to regularize the CSPs for different learning paradigms, such as the sparse CSP, the transfer CSP, and the multisubject CSP. Evaluations on three BCI competition datasets show that the regularized CSP algorithms outperform other baselines, especially for the high-dimensional small training set. The extensive results validate the efficiency and effectiveness of the proposed CSP formulation in different learning contexts.

Design a novel BCI for neurorehabilitation using concurrent LFP and EEG features: a case study

Brain-computer interfaces (BCI) that enables people with severe motor disabilities to use their brain signals for direct control of objects have attracted increased interest in rehabilitation. To date, no study has investigated feasibility of the BCI framework incorporating both intracortical and scalp signals. Methods: Concurrent local field potential (LFP) from the hand-knob area and scalp EEG were recorded in a paraplegic patient undergoing a spike-based close-loop neurorehabilitation training. Based upon multimodal spatio-spectral feature extraction and Naive Bayes classification, we developed, for the first time, a novel LFP-EEG-BCI for motor intention decoding. A transfer learning (TL) approach was employed to further improve the feasibility. The performance of the proposed LFP-EEG-BCI for four-class upper-limb motor intention decoding was assessed. Results: Using a decision fusion strategy, we showed that the LFP-EEG-BCI significantly (p <0.05) outperformed single modal BCI (LFP-BCI and EEG-BCI) in terms of decoding accuracy with the best performance achieved using regularized common spatial pattern features. Interrogation of feature characteristics revealed discriminative spatial and spectral patterns, which may lead to new insights for better understanding of brain dynamics during different motor imagery tasks and promote development of efficient decoding algorithms. Moreover, we showed that similar classification performance could be obtained with few training trials, therefore highlighting the efficacy of TL. Conclusion: The present findings demonstrated the superiority of the novel LFP-EEG-BCI in motor intention decoding. Significance: This work introduced a novel LFP-EEG-BCI that may lead to new directions for developing practical neurorehabilitation systems with high detection accuracy and multi-paradigm feasibility in clinical applications.