Optimal spatial filtering of single trial EEG during imagined hand movement

Towards identifying optimal biased feedback for various user states and traits in motor imagery BCI

OBJECTIVE

Neural self-regulation is necessary for achieving control over brain-computer interfaces (BCIs). This can be an arduous learning process esspecially for motor imagery BCI. Various training methods were proposed to assist users in accomplishing BCI control and increase performance. Notably the use of biased feedback, i.e. non-realistic representation of performance. Benefits of biased feedback on performance and learning vary between users (e.g. depending on their initial level of BCI control) and remain speculative. To disentangle the speculations, we investigate what personality type, initial state and calibration performance (CP) could benefit from biased feedback.

METHODS

We conduct an experiment (n=30 for 2 sessions). The feedback provided to each group (n=10) is either positively, negatively or not biased.

RESULTS

Statistical analyses suggest that interactions between bias and: 1) workload, 2) anxiety, and 3) self-control significantly affect online performance. For instance, low initial workload paired with negative bias is associated to higher peak performances (86%) than without any bias (69%). High anxiety relates negatively to performance no matter the bias (60%), while low anxiety matches best with negative bias (76%). For low CP, learning rate (LR) increases with negative bias only short term (LR=2%) as during the second session it severely drops (LR=-1%).

CONCLUSION

We unveil many interactions between said human factors and bias. Additionally, we use prediction models to confirm and reveal even more interactions.

SIGNIFICANCE

This paper is a first step towards identifying optimal biased feedback for a personality type, state, and CP in order to maximize BCI performance and learning.

Motor-Imagery EEG-Based BCIs in Wheelchair Movement and Control: A Systematic Literature Review

The pandemic emergency of the coronavirus disease 2019 (COVID-19) shed light on the need for innovative aids, devices, and assistive technologies to enable people with severe disabilities to live their daily lives. EEG-based Brain-Computer Interfaces (BCIs) can lead individuals with significant health challenges to improve their independence, facilitate participation in activities, thus enhancing overall well-being and preventing impairments. This systematic review provides state-of-the-art applications of EEG-based BCIs, particularly those using motor-imagery (MI) data, to wheelchair control and movement. It presents a thorough examination of the different studies conducted since 2010, focusing on the algorithm analysis, features extraction, features selection, and classification techniques used as well as on wheelchair components and performance evaluation. The results provided in this paper could highlight the limitations of current biomedical instrumentations applied to people with severe disabilities and bring focus to innovative research topics.

Brain Symmetry Analysis during the Use of a BCI Based on Motor Imagery for the Control of a Lower-Limb Exoskeleton

Brain–Computer Interfaces (BCI) are systems that allow external devices to be controlled by means of brain activity. There are different such technologies, and electroencephalography (EEG) is an example. One of the most common EEG control methods is based on detecting changes in sensorimotor rhythms (SMRs) during motor imagery (MI). The aim of this study was to assess the laterality of cortical function when performing MI of the lower limb. Brain signals from five subjects were analyzed in two conditions, during exoskeleton-assisted gait and while static. Three different EEG electrode configurations were evaluated: covering both hemispheres, covering the non-dominant hemisphere and covering the dominant hemisphere. In addition, the evolution of performance and laterality with practice was assessed. Although sightly superior results were achieved with information from all electrodes, differences between electrode configurations were not statistically significant. Regarding the evolution during the experimental sessions, the performance of the BCI generally evolved positively the higher the experience was.

Does Real-Time Feedback Affect Sensorimotor EEG Patterns in Routine Motor Imagery Practice?

Background. Motor imagery engages much of the same neural circuits as an overt movement. Therefore, the mental rehearsal of movements is often used to supplement physical training and might aid motor neurorehabilitation after stroke. One attempt to capture the brain’s involvement in imagery involves the use, as a marker, of the depression or event-related desynchronization (ERD) of thalamocortical sensorimotor rhythms found in a human electroencephalogram (EEG). Using fast real-time processing, it is possible to make the subject aware of their own brain reactions or—even better—to turn them into actions through a technology called the brain–computer interface (BCI). However, it remains unclear whether BCI-enabled imagery facilitates a stronger or qualitatively different brain response compared to the open-loop training. Methods. Seven healthy volunteers who were experienced in both closed and open-loop motor imagery took part in six experimental sessions over a period of 4.5 months, in which they performed kinesthetic imagery of a previously known set of finger and arm movements with simultaneous 30-channel EEG acquisition. The first and the last session mostly consisted of feedback trials in which the subjects were presented with the classification results of the EEG patterns in real time; during the other sessions, no feedback was provided. Spatiotemporal and amplitude features of the ERD patterns concomitant with imagery were compared across experimental days and between feedback conditions using linear mixed-effects modeling. Results. The main spatial sources of ERD appeared to be highly stable across the six experimental days, remaining nearly identical in five of seven subjects (Pearson’s ρ > 0.94). Only in one subject did the spatial pattern of activation statistically significantly differ (p = 0.009) between the feedback and no-feedback conditions. Real-time visual feedback delivered through the BCI did not significantly increase the ERD strength. Conclusion. The results imply that the potential benefits of MI could be yielded by well-habituated subjects with a simplified open-loop setup, e.g., through at-home self-practice.

A novel method for classification of multi-class motor imagery tasks based on feature fusion

Motor imagery based brain computer interface (MI-BCI) has the advantage of strong independence that can rely on the spontaneous brain activity of the user to operate external devices. However, MI-BCI still has the problem of poor control effect, which requires more effective feature extraction algorithms and classification methods to extract distinctly separable features from electroencephalogram (EEG) signals. This paper proposes a novel framework based on Bispectrum, Entropy and common spatial pattern (BECSP). Here we use three methods of bispectrum in higher order spectra, entropy and CSP to extract MI-EEG signal features, and then select the most contributing features through tree-based feature selection algorithm. By comparing the classification results of SVM, Random Forest, Naive Bayes, LDA, KNN, Xgboost and Adaboost, we finally decide to use the SVM algorithm based on RBF kernel function which obtained the best result among them for classification. The proposed method is applied to the BCI competition IV data set 2a and BCI competition III data set IVa. On data set 2a, the highest accuracy on the evaluation data set reaches 85%. The experiment on data set IVa can also achieve good results. Compared with other algorithms that use the same data set, the performance of our algorithm has also been improved.

Reducing False Triggering Caused by Irrelevant Mental Activities in Brain-Computer Interface Based on Motor Imagery

In recent years, the brain-computer interface (BCI) based on motor imagery (MI) has been considered as a potential post-stroke rehabilitation technology. However, the recognition of MI relies on the event-related desynchronization (ERD) feature, which has poor task specificity. Further, there is the problem of false triggering (irrelevant mental activities recognized as the MI of the target limb). In this paper, we discuss the feasibility of reducing the false triggering rate using a novel paradigm, in which the steady-state somatosensory evoked potential (SSSEP) is combined with the MI (MI-SSSEP). Data from the target (right hand MI) and nontarget task (rest) were used to establish the recognition model, and three kinds of interference tasks were used to test the false triggering performance. In the MI-SSSEP paradigm, ERD and SSSEP features modulated by MI could be used for recognition, while in the MI paradigm, only ERD features could be used. The results showed that the false triggering rate of interference tasks with SSSEP features was reduced to 29.3%, which was far lower than the 55.5% seen under the MI paradigm with ERD features. Moreover, in the MI-SSSEP paradigm, the recognition rate of the target and nontarget task was also significantly improved. Further analysis showed that the specificity of SSSEP was significantly higher than that of ERD (p < 0.05), but the sensitivity was not significantly different. These results indicated that SSSEP modulated by MI could more specifically decode the target task MI, and thereby may have potential in achieving more accurate rehabilitation training.

A hybrid environment control system combining EMG and SSVEP signal based on brain-computer interface technology

The patients who are impaired with neurodegenerative disorders cannot command their muscles through the neural pathways. These patients are given an alternative from their neural path through Brain-Computer Interface (BCI) systems, which are the explicit use of brain impulses without any need for a computer's vocal muscle. Nowadays, the steady-state visual evoked potential (SSVEP) modality offers a robust communication pathway to introduce a non-invasive BCI. There are some crucial constituents, including window length of SSVEP response, the number of electrodes in the acquisition device and system accuracy, which are the critical performance components in any BCI system based on SSVEP signal. In this study, a real-time hybrid BCI system consists of SSVEP and EMG has been proposed for the environmental control system. The feature in terms of the common spatial pattern (CSP) has been extracted from four classes of SSVEP response, and extracted feature has been classified using K-nearest neighbors (k-NN) based classification algorithm. The obtained classification accuracy of eight participants was 97.41%. Finally, a control mechanism that aims to apply for the environmental control system has also been developed. The proposed system can identify 18 commands (i.e., 16 control commands using SSVEP and two commands using EMG). This result represents very encouraging performance to handle real-time SSVEP based BCI system consists of a small number of electrodes. The proposed framework can offer a convenient user interface and a reliable control method for realistic BCI technology.

EEG-Based Brain-Computer Interfaces (BCIs): A Survey of Recent Studies on Signal Sensing Technologies and Computational Intelligence Approaches and Their Applications

Brain-Computer interfaces (BCIs) enhance the capability of human brain activities to interact with the environment. Recent advancements in technology and machine learning algorithms have increased interest in electroencephalographic (EEG)-based BCI applications. EEG-based intelligent BCI systems can facilitate continuous monitoring of fluctuations in human cognitive states under monotonous tasks, which is both beneficial for people in need of healthcare support and general researchers in different domain areas. In this review, we survey the recent literature on EEG signal sensing technologies and computational intelligence approaches in BCI applications, compensating for the gaps in the systematic summary of the past five years. Specifically, we first review the current status of BCI and signal sensing technologies for collecting reliable EEG signals. Then, we demonstrate state-of-the-art computational intelligence techniques, including fuzzy models and transfer learning in machine learning and deep learning algorithms, to detect, monitor, and maintain human cognitive states and task performance in prevalent applications. Finally, we present a couple of innovative BCI-inspired healthcare applications and discuss future research directions in EEG-based BCI research.

Manipulating Single-Trial Motor Performance in Chronic Stroke Patients by Closed-Loop Brain State Interaction

Motor impaired patients performing repetitive motor tasks often reveal large single-trial performance variations. Based on a data-driven framework, we extracted robust oscillatory brain states from pre-trial intervals, which are predictive for the upcoming motor performance on the level of single trials. Based on the brain state estimate, i.e. whether the brain state predicts a good or bad upcoming performance, we implemented a novel gating strategy for the start of trials by selecting specifically suitable or unsuitable trial starting time points. In a pilot study with four chronic stroke patients with hand motor impairments, we conducted a total of 41 sessions. After few initial calibration sessions, patients completed approximately 15 hours of effective hand motor training during eight online sessions using the gating strategy. Patients' reaction times were significantly reduced for suitable trials compared to unsuitable trials and shorter overall trial durations under suitable states were found in two patients. Overall, this successful proof-of-concept pilot study motivates to transfer this closed-loop training framework to a clinical study and to other application fields, such as cognitive rehabilitation, sport sciences or systems neuroscience.

Anytime collaborative brain-computer interfaces for enhancing perceptual group decision-making

In this paper we present, and test in two realistic environments, collaborative Brain-Computer Interfaces (cBCIs) that can significantly increase both the speed and the accuracy of perceptual group decision-making. The key distinguishing features of this work are: (1) our cBCIs combine behavioural, physiological and neural data in such a way as to be able to provide a group decision at any time after the quickest team member casts their vote, but the quality of a cBCI-assisted decision improves monotonically the longer the group decision can wait; (2) we apply our cBCIs to two realistic scenarios of military relevance (patrolling a dark corridor and manning an outpost at night where users need to identify any unidentified characters that appear) in which decisions are based on information conveyed through video feeds; and (3) our cBCIs exploit Event-Related Potentials (ERPs) elicited in brain activity by the appearance of potential threats but, uniquely, the appearance time is estimated automatically by the system (rather than being unrealistically provided to it). As a result of these elements, in the two test environments, groups assisted by our cBCIs make both more accurate and faster decisions than when individual decisions are integrated in more traditional manners.

Transfer Learning based on Optimal Transport for Motor Imagery Brain-Computer Interfaces

OBJECTIVE

This paper tackles the cross-sessions variability of electroencephalography-based brain-computer interfaces (BCIs) in order to avoid the lengthy recalibration step of the decoding method before every use.

METHODS

We develop a new approach of domain adaptation based on optimal transport to tackle brain signal variability between sessions of motor imagery BCIs. We propose a backward method where, unlike the original formulation, the data from a new session are transported to a calibration session, and thereby avoiding model retraining. Several domain adaptation approaches are evaluated and compared. We simulated two possible online scenarios: i) block-wise adaptation and ii) sample-wise adaptation. In this study, we collect a dataset of 10 subjects performing a hand motor imagery task in 2 sessions. A publicly available dataset is also used.

RESULTS

For the first scenario, results indicate that classifier retraining can be avoided by means of our backward formulation yielding to equivalent classification performance as compared to retraining solutions. In the second scenario, classification performance rises up to 90.23\% overall accuracy when the label of the indicated mental task is used to learn the transport. Adaptive time is between 10 and 80 times faster than the other methods.

CONCLUSIONS

The proposed method is able to mitigate the cross-session variability in motor imagery BCIs.

SIGNIFICANCE

The backward formulation is an efficient retraining-free approach built to avoid lengthy calibration times. Thus, the BCI can be actively used after just a few minutes of setup. This is important for practical applications such as BCI-based motor rehabilitation.

Selective Subject Pooling Strategy to Improve Model Generalization for a Motor Imagery BCI

Brain–computer interfaces (BCIs) facilitate communication for people who cannot move their own body. A BCI system requires a lengthy calibration phase to produce a reasonable classifier. To reduce the duration of the calibration phase, it is natural to attempt to create a subject-independent classifier with all subject datasets that are available; however, electroencephalogram (EEG) data have notable inter-subject variability. Thus, it is very challenging to achieve subject-independent BCI performance comparable to subject-specific BCI performance. In this study, we investigate the potential for achieving better subject-independent motor imagery BCI performance by conducting comparative performance tests with several selective subject pooling strategies (i.e., choosing subjects who yield reasonable performance selectively and using them for training) rather than using all subjects available. We observed that the selective subject pooling strategy worked reasonably well with public MI BCI datasets. Finally, based upon the findings, criteria to select subjects for subject-independent BCIs are proposed here.

Education 4.0: Teaching the Basis of Motor Imagery Classification Algorithms for Brain-Computer Interfaces

Education 4.0 is looking to prepare future scientists and engineers not only by granting them with knowledge and skills but also by giving them the ability to apply them to solve real life problems through the implementation of disruptive technologies. As a consequence, there is a growing demand for educational material that introduces science and engineering students to technologies, such as Artificial Intelligence (AI) and Brain–Computer Interfaces (BCI). Thus, our contribution towards the development of this material is to create a test bench for BCI given the basis and analysis on how they can be discriminated against. This is shown using different AI methods: Fisher Linear Discriminant Analysis (LDA), Support Vector Machines (SVM), Artificial Neural Networks (ANN), Restricted Boltzmann Machines (RBM) and Self-Organizing Maps (SOM), allowing students to see how input changes alter their performance. These tests were done against a two-class Motor Image database. First, using a large frequency band and no filtering eye movement. Secondly, the band was reduced and the eye movement was filtered. The accuracy was analyzed obtaining values around 70∼80% for all methods, excluding SVM and SOM mapping. Accuracy and mapping differentiability increased for some subjects for the second scenario 70∼85%, meaning either their band with the most significant information is on that limited space or the contamination because of eye movement was better mitigated by the regression method. This can be translated to saying that these methods work better under limited spaces. The outcome of this work is useful to show future scientists and engineers how BCI experiments are conducted while teaching them the basics of some AI techniques that can be used in this and other several experiments that can be carried on the framework of Education 4.0.

Review of brain encoding and decoding mechanisms for EEG-based brain-computer interface

A brain-computer interface (BCI) can connect humans and machines directly and has achieved successful applications in the past few decades. Many new BCI paradigms and algorithms have been developed in recent years. Therefore, it is necessary to review new progress in BCIs. This paper summarizes progress for EEG-based BCIs from the perspective of encoding paradigms and decoding algorithms, which are two key elements of BCI systems. Encoding paradigms are grouped by their underlying neural meachanisms, namely sensory- and motor-related, vision-related, cognition-related and hybrid paradigms. Decoding algorithms are reviewed in four categories, namely decomposition algorithms, Riemannian geometry, deep learning and transfer learning. This review will provide a comprehensive overview of both modern primary paradigms and algorithms, making it helpful for those who are developing BCI systems.

Sub-band target alignment common spatial pattern in brain-computer interface

BACKGROUND AND OBJECTIVE

In the brain computer interface (BCI) field, using sub-band common spatial pattern (SBCSP) and filter bank common spatial pattern (FBCSP) can improve the accuracy of classification by selection a specific frequency band. However, in the cross-subject classification, due to the individual differences between different subjects, the performance is limited.

METHODS

This paper introduces the idea of transfer learning and presents the sub-band target alignment common spatial pattern (SBTACSP) method and applies it to the cross-subject classification of motor imagery (MI) EEG signals. First, the EEG signals are bandpass-filtered into multiple frequency bands (sub-band filtering). Subsequently, the source domain trails are aligned into the target domain space in each frequency band. The CSP algorithm is then employed to extract features among which more representative features are selected by the minimum redundancy maximum relevance (mRMR) approach from each sub-band. Then the features of all sub-bands are fused. Finally, conventional linear discriminant analysis (LDA) algorithm is used for MI classification.

RESULTS

Our method is evaluated on Datasets Ⅱa and Ⅱb of the BCI Competition Ⅳ. Compared with six state-of-the-art algorithms, the proposed SBTACSP method performed relatively the best and achieved a mean classification accuracy of 75.15% and 66.85% in cross-subject classification of Datasets Ⅱa and Ⅱb respectively.

CONCLUSION

Therefore, the combination of sub-band filtering and transfer learning achieves superior classification performance compared to either one. The proposed algorithms will greatly promote the practical application of MI based BCIs.

Auditory Attention Detection via Cross-Modal Attention

Humans show a remarkable perceptual ability to select the speech stream of interest among multiple competing speakers. Previous studies demonstrated that auditory attention detection (AAD) can infer which speaker is attended by analyzing a listener's electroencephalography (EEG) activities. However, previous AAD approaches perform poorly on short signal segments, more advanced decoding strategies are needed to realize robust real-time AAD. In this study, we propose a novel approach, i.e., cross-modal attention-based AAD (CMAA), to exploit the discriminative features and the correlation between audio and EEG signals. With this mechanism, we hope to dynamically adapt the interactions and fuse cross-modal information by directly attending to audio and EEG features, thereby detecting the auditory attention activities manifested in brain signals. We also validate the CMAA model through data visualization and comprehensive experiments on a publicly available database. Experiments show that the CMAA achieves accuracy values of 82.8, 86.4, and 87.6% for 1-, 2-, and 5-s decision windows under anechoic conditions, respectively; for a 2-s decision window, it achieves an average of 84.1% under real-world reverberant conditions. The proposed CMAA network not only achieves better performance than the conventional linear model, but also outperforms the state-of-the-art non-linear approaches. These results and data visualization suggest that the CMAA model can dynamically adapt the interactions and fuse cross-modal information by directly attending to audio and EEG features in order to improve the AAD performance.

Improving motor imagery classification during induced motor perturbations

Objective.Motor imagery is the mental simulation of movements. It is a common paradigm to design brain-computer interfaces (BCIs) that elicits the modulation of brain oscillatory activity similar to real, passive and induced movements. In this study, we used peripheral stimulation to provoke movements of one limb during the performance of motor imagery tasks. Unlike other works, in which induced movements are used to support the BCI operation, our goal was to test and improve the robustness of motor imagery based BCI systems to perturbations caused by artificially generated movements.Approach.We performed a BCI session with ten participants who carried out motor imagery of three limbs. In some of the trials, one of the arms was moved by neuromuscular stimulation. We analysed 2-class motor imagery classifications with and without movement perturbations. We investigated the performance decrease produced by these disturbances and designed different computational strategies to attenuate the observed classification accuracy drop.Main results.When the movement was induced in a limb not coincident with the motor imagery classes, extracting oscillatory sources of the movement imagination tasks resulted in BCI performance being similar to the control (undisturbed) condition; when the movement was induced in a limb also involved in the motor imagery tasks, the performance drop was significantly alleviated by spatially filtering out the neural noise caused by the stimulation. We also show that the loss of BCI accuracy was accompanied by weaker power of the sensorimotor rhythm. Importantly, this residual power could be used to predict whether a BCI user will perform with sufficient accuracy under the movement disturbances.Significance.We provide methods to ameliorate and even eliminate motor related afferent disturbances during the performance of motor imagery tasks. This can help improving the reliability of current motor imagery based BCI systems.

On the characterization of cognitive tasks using activity-specific short-lived synchronization between electroencephalography channels

Accurate characterization of brain activity during a cognitive task is challenging due to the dynamically changing and the complex nature of the brain. The majority of the proposed approaches assume stationarity in brain activity and disregard the systematic timing organization among brain regions during cognitive tasks. In this study, we propose a novel cognitive activity recognition method that captures the activity-specific timing parameters from training data that elicits maximal average short-lived pairwise synchronization between electroencephalography signals. We evaluated the characterization power of the activity-specific timing parameter triplets in a motor imagery activity recognition framework. The activity-specific timing parameter triplets consist of latency of the maximally synchronized signal segments from activity onset Δt, the time lag between maximally synchronized signal segments τ, and the duration of the maximally synchronized signal segments w. We used cosine-based similarity, wavelet bi-coherence, phase-locking value, phase coherence value, linearized mutual information, and cross-correntropy to calculate the channel synchronizations at the specific timing parameters. Recognition performances as well as statistical analyses on both BCI Competition-III dataset IVa and PhysioNet Motor Movement/Imagery dataset, indicate that the inter-channel short-lived synchronization calculated using activity-specific timing parameter triplets elicit significantly distinct synchronization profiles for different motor imagery tasks and can thus reliably be used for cognitive task recognition purposes.

Universal adversarial perturbations for CNN classifiers in EEG-based BCIs

Objective. Multiple convolutional neural network (CNN) classifiers have been proposed for electroencephalogram (EEG) based brain-computer interfaces (BCIs). However, CNN models have been found vulnerable to universal adversarial perturbations (UAPs), which are small and example-independent, yet powerful enough to degrade the performance of a CNN model, when added to a benign example.Approach. This paper proposes a novel total loss minimization (TLM) approach to generate UAPs for EEG-based BCIs.Main results. Experimental results demonstrated the effectiveness of TLM on three popular CNN classifiers for both target and non-target attacks. We also verified the transferability of UAPs in EEG-based BCI systems.Significance. To our knowledge, this is the first study on UAPs of CNN classifiers in EEG-based BCIs. UAPs are easy to construct, and can attack BCIs in real-time, exposing a potentially critical security concern of BCIs.

Evaluating deep learning EEG-based mental stress classification in adolescents with autism for breathing entrainment BCI

Mental stress is a major individual and societal burden and one of the main contributing factors that lead to pathologies such as depression, anxiety disorders, heart attacks, and strokes. Given that anxiety disorders are one of the most common comorbidities in youth with autism spectrum disorder (ASD), this population is particularly vulnerable to mental stress, severely limiting overall quality of life. To prevent this, early stress quantification with machine learning (ML) and effective anxiety mitigation with non-pharmacological interventions are essential. This study aims to investigate the feasibility of exploiting electroencephalography (EEG) signals for stress assessment by comparing several ML classifiers, namely support vector machine (SVM) and deep learning methods. We trained a total of eleven subject-dependent models-four with conventional brain-computer interface (BCI) methods and seven with deep learning approaches-on the EEG of neurotypical (n=5) and ASD (n=8) participants performing alternating blocks of mental arithmetic stress induction, guided and unguided breathing. Our results show that a multiclass two-layer LSTM RNN deep learning classifier is capable of identifying mental stress from ongoing EEG with an overall accuracy of 93.27%. Our study is the first to successfully apply an LSTM RNN classifier to identify stress states from EEG in both ASD and neurotypical adolescents, and offers promise for an EEG-based BCI for the real-time assessment and mitigation of mental stress through a closed-loop adaptation of respiration entrainment.

Sparse representation-based classification with two-dimensional dictionary optimization for motor imagery EEG pattern recognition

BACKGROUND

Sparse representation-based classification (SRC) has more advantages in motor imagery EEG pattern recognition, and the quality of dictionary construction directly determines the performance of SRC. In this paper, we proposed a two-dimensional dictionary optimization (TDDO) method to directly improve the performance of SRC.

NEW METHOD

Firstly, an initial dictionary was constructed with multi-band features extracted by filter band common spatial pattern (FBCSP). Then Lasso regression is used to select significant features in each atom synchronously in the horizontal direction, and the KNN-based method is used to clean up noise atoms in the vertical direction. Finally, an SRC method by training samples linearly representing test samples was implemented in classification.

RESULTS

The results show the necessity and rationality of TDDO-SRC method. The highest average classification accuracy of 86.5% and 92.4% is obtained on two public datasets.

COMPARISON WITH EXISTING METHOD(S)

The proposed method has more superior classification accuracy compared to traditional methods and existing winners' methods.

CONCLUSIONS

The quality of dictionary construction has a great impact on the robustness of SRC. And compared with the original SRC, the classification accuracy of the optimized TDDO-SRC is greatly improved.

EEG-controlled functional electrical stimulation rehabilitation for chronic stroke: system design and clinical application

Stroke is one of the most serious diseases that threaten human life and health. It is a major cause of death and disability in the clinic. New strategies for motor rehabilitation after stroke are undergoing exploration. We aimed to develop a novel artificial neural rehabilitation system, which integrates brain-computer interface (BCI) and functional electrical stimulation (FES) technologies, for limb motor function recovery after stroke. We conducted clinical trials (including controlled trials) in 32 patients with chronic stroke. Patients were randomly divided into the BCI-FES group and the neuromuscular electrical stimulation (NMES) group. The changes in outcome measures during intervention were compared between groups, and the trends of ERD values based on EEG were analyzed for BCI-FES group. Results showed that the increase in Fugl Meyer Assessment of the Upper Extremity (FMA-UE) and Kendall Manual Muscle Testing (Kendall MMT) scores of the BCI-FES group was significantly higher than that in the sham group, which indicated the practicality and superiority of the BCI-FES system in clinical practice. The change in the laterality coefficient (LC) values based on μ-ERD (ΔLC_m-ERD) had high significant positive correlation with the change in FMA-UE(r = 0.6093, P = 0.012), which provides theoretical basis for exploring novel objective evaluation methods.

A Survey on Deep Learning-Based Short/Zero-Calibration Approaches for EEG-Based Brain-Computer Interfaces

Brain-computer interfaces (BCIs) utilizing machine learning techniques are an emerging technology that enables a communication pathway between a user and an external system, such as a computer. Owing to its practicality, electroencephalography (EEG) is one of the most widely used measurements for BCI. However, EEG has complex patterns and EEG-based BCIs mostly involve a cost/time-consuming calibration phase; thus, acquiring sufficient EEG data is rarely possible. Recently, deep learning (DL) has had a theoretical/practical impact on BCI research because of its use in learning representations of complex patterns inherent in EEG. Moreover, algorithmic advances in DL facilitate short/zero-calibration in BCI, thereby suppressing the data acquisition phase. Those advancements include data augmentation (DA), increasing the number of training samples without acquiring additional data, and transfer learning (TL), taking advantage of representative knowledge obtained from one dataset to address the so-called data insufficiency problem in other datasets. In this study, we review DL-based short/zero-calibration methods for BCI. Further, we elaborate methodological/algorithmic trends, highlight intriguing approaches in the literature, and discuss directions for further research. In particular, we search for generative model-based and geometric manipulation-based DA methods. Additionally, we categorize TL techniques in DL-based BCIs into explicit and implicit methods. Our systematization reveals advances in the DA and TL methods. Among the studies reviewed herein, ~45% of DA studies used generative model-based techniques, whereas ~45% of TL studies used explicit knowledge transferring strategy. Moreover, based on our literature review, we recommend an appropriate DA strategy for DL-based BCIs and discuss trends of TLs used in DL-based BCIs.

Optimal channel-based sparse time-frequency blocks common spatial pattern feature extraction method for motor imagery classification

Common spatial pattern (CSP) as a spatial filtering method has been most widely applied to electroencephalogram (EEG) feature extraction to classify motor imagery (MI) in brain-computer interface (BCI) applications. The effectiveness of CSP is determined by the quality of interception in a specific time window and frequency band. Although numerous algorithms have been designed to optimize CSP by splitting the EEG data with a sliding time window and dividing the frequency bands with a set of band-pass filters, simultaneously. However, they did not consider the drawbacks of the rapid increase in data volume and feature dimensions brought about by this method, which would reduce the classification accuracy and calculation efficiency of the model. Therefore, we propose an optimal channel-based sparse time-frequency blocks common spatial pattern (OCSB-CSP) feature extraction method to improve the model classification accuracy and computational efficiency. Comparative experiments on two public EEG datasets show that the proposed method can quickly select significant time-frequency blocks and improve classification performance. The average classification accuracies are higher than those of other winners' methods, providing a new idea for the improvement of BCI applications.

AsEmo: Automatic Approach for EEG-Based Multiple Emotional State Identification

An electroencephalogram (EEG) is the most extensively used physiological signal in emotion recognition using biometric data. However, these EEG data are difficult to analyze, because of their anomalous characteristic where statistical elements vary according to time as well as spatial-temporal correlations. Therefore, new methods that can clearly distinguish emotional states in EEG data are required. In this paper, we propose a new emotion recognition method, named AsEmo. The proposed method extracts effective features boosting classification performance on various emotional states from multi-class EEG data. AsEmo Automatically determines the number of spatial filters needed to extract significant features using the explained variance ratio (EVR) and employs a Subject-independent method for real-time processing of Emotion EEG data. The advantages of this method are as follows: (a) it automatically determines the spatial filter coefficients distinguishing emotional states and extracts the best features; (b) it is very robust for real-time analysis of new data using a subject-independent technique that considers subject sets, and not a specific subject; (c) it can be easily applied to both binary-class and multi-class data. Experimental results on real-world EEG emotion recognition tasks demonstrate that AsEmo outperforms other state-of-the-art methods with a 2-8% improvement in terms of classification accuracy.

A BMI Based on Motor Imagery and Attention for Commanding a Lower-Limb Robotic Exoskeleton: A Case Study

Lower-limb robotic exoskeletons are wearable devices that can be beneficial for people with lower-extremity motor impairment because they can be valuable in rehabilitation or assistance. These devices can be controlled mentally by means of brain–machine interfaces (BMI). The aim of the present study was the design of a BMI based on motor imagery (MI) to control the gait of a lower-limb exoskeleton. The evaluation is carried out with able-bodied subjects as a preliminary study since potential users are people with motor limitations. The proposed control works as a state machine, i.e., the decoding algorithm is different to start (standing still) and to stop (walking). The BMI combines two different paradigms for reducing the false triggering rate (when the BMI identifies irrelevant brain tasks as MI), one based on motor imagery and another one based on the attention to the gait of the user. Research was divided into two parts. First, during the training phase, results showed an average accuracy of 68.44 ± 8.46% for the MI paradigm and 65.45 ± 5.53% for the attention paradigm. Then, during the test phase, the exoskeleton was controlled by the BMI and the average performance was 64.50 ± 10.66%, with very few false positives. Participants completed various sessions and there was a significant improvement over time. These results indicate that, after several sessions, the developed system may be employed for controlling a lower-limb exoskeleton, which could benefit people with motor impairment as an assistance device and/or as a therapeutic approach with very limited false activations.

Learning Common Time-Frequency-Spatial Patterns for Motor Imagery Classification

The common spatial patterns (CSP) algorithm is the most popular spatial filtering method applied to extract electroencephalogram (EEG) features for motor imagery (MI) based brain-computer interface (BCI) systems. The effectiveness of the CSP algorithm depends on optimal selection of the frequency band and time window from the EEG. Many algorithms have been designed to optimize frequency band selection for CSP, while few algorithms seek to optimize the time window. This study proposes a novel framework, termed common time-frequency-spatial patterns (CTFSP), to extract sparse CSP features from multi-band filtered EEG data in multiple time windows. Specifically, the whole MI period is first segmented into multiple subseries using a sliding time window approach. Then, sparse CSP features are extracted from multiple frequency bands in each time window. Finally, multiple support vector machine (SVM) classifiers with the Radial Basis Function (RBF) kernel are trained to identify the MI tasks and the voting result of these classifiers determines the final output of the BCI. This study applies the proposed CTFSP algorithm to three public EEG datasets (BCI competition III dataset IVa, BCI competition III dataset IIIa, and BCI competition IV dataset 1) to validate its effectiveness, compared against several other state-of-the-art methods. The experimental results demonstrate that the proposed algorithm is a promising candidate for improving the performance of MI-BCI systems.

SUPFUNSIM: Spatial Filtering Toolbox for EEG

Brain activity pattern recognition from EEG or MEG signal analysis is one of the most important method in cognitive neuroscience. The supFunSim library is a new Matlab toolbox which generates accurate EEG forward model and implements a collection of spatial filters for EEG source reconstruction, including the linearly constrained minimum-variance (LCMV), eigenspace LCMV, nulling (NL), and minimum-variance pseudo-unbiased reduced-rank (MV-PURE) filters in various versions. It also enables source-level directed connectivity analysis using partial directed coherence (PDC) measure. The supFunSim library is based on the well-known FieldTrip toolbox for EEG and MEG analysis and is written using object-oriented programming paradigm. The resulting modularity of the toolbox enables its simple extensibility. This paper gives a complete overview of the toolbox from both developer and end-user perspectives, including description of the installation process and use cases.

Correlation-based common spatial pattern (CCSP): A novel extension of CSP for classification of motor imagery signal

Common spatial pattern (CSP) is shown to be an effective pre-processing algorithm in order to discriminate different classes of motor-based EEG signals by obtaining suitable spatial filters. The performance of these filters can be improved by regularized CSP, in which available prior information is added in terms of regularization terms into the objective function of conventional CSP. Variety of prior information can be used in this way. In this paper, we used time correlation between different classes of EEG signal as the prior information, which is clarified similarity between different classes of signal for regularizing CSP. Furthermore, the proposed objective function can be easily extended to more than two-class problems. We used three different standard datasets to evaluate the performance of the proposed method. Correlation-based CSP (CCSP) outperformed original CSP as well as the existing regularized CSP, Principle Component Cnalysis (PCA) and Fisher Discriminate Analysis (FDA) in both two-class and multi-class scenarios. The simulation results showed that the proposed method outperformed conventional CSP by 6.9% in 2-class and 2.23% in multi-class problem in term of mean classification accuracy.

The Riemannian Spatial Pattern method: mapping and clustering movement imagery using Riemannian geometry

OBJECTIVE

Over the last decade, Riemannian geometry has shown promising results for motor imagery classification. However, extracting the underlying spatial features is not as straightforward as for applying Common Spatial Pattern (CSP) filtering prior to classification. In this article, we propose a simple way to extract the spatial patterns obtained from Riemannian classification: the Riemannian Spatial Pattern (RSP) method, which is based on the backward channel selection procedure.

APPROACH

The RSP method was compared to the CSP approach on ECoG data obtained from a quadriplegic patient while performing imagined movements of arm articulations and fingers.

MAIN RESULTS

Similar results were found between the RSP and CSP methods for mapping each motor imagery task with activations following the classical somatotopic organization. Clustering obtained by pairwise comparisons of imagined motor movements however, revealed higher differentiation for the RSP method compared to the CSP approach. Importantly, the RSP approach could provide a precise comparison of the imagined finger flexions which added supplementary information to the mapping results.

SIGNIFICANCE

Our new RSP method illustrates the interest of the Riemannian framework in the spatial domain and as such offers new avenues for the neuroimaging community. This study is part of an ongoing clinical trial registered with ClinicalTrials.gov, NCT02550522.

A Dense Long Short-Term Memory Model for Enhancing the Imagery-Based Brain-Computer Interface

Imagery-based brain-computer interfaces (BCIs) aim to decode different neural activities into control signals by identifying and classifying various natural commands from electroencephalogram (EEG) patterns and then control corresponding equipment. However, several traditional BCI recognition algorithms have the “one person, one model” issue, where the convergence of the recognition model’s training process is complicated. In this study, a new BCI model with a Dense long short-term memory (Dense-LSTM) algorithm is proposed, which combines the event-related desynchronization (ERD) and the event-related synchronization (ERS) of the imagery-based BCI; model training and testing were conducted with its own data set. Furthermore, a new experimental platform was built to decode the neural activity of different subjects in a static state. Experimental evaluation of the proposed recognition algorithm presents an accuracy of 91.56%, which resolves the “one person one model” issue along with the difficulty of convergence in the training process.

Massage Therapy's Effectiveness on the Decoding EEG Rhythms of Left/Right Motor Imagery and Motion Execution in Patients With Skeletal Muscle Pain

Objective: Most of effectiveness assessments of the widely-used Massage therapy were based on subjective routine clinical assessment tools, such as Visual Analogue Scale (VAS) score. However, few studies demonstrated the impact of massage on the Electroencephalograph (EEG) rhythm decoding of Motor imagery (MI) and motion execution (ME) with trunk left/right bending in patients with skeletal muscle pain. Method: We used the sample entropy (SampEn), permutation entropy (PermuEn), common spatial pattern (CSP) features, support vector machine (SVM) and logic regression (LR) classifiers. We also used the convolutional neural network (CNN) and attention-based bi-directional long short-term memory (BiLSTM) for classification. Results: The averaged SampEn and PermuEn values of alpha rhythm decreased in almost fourteen channels for five statuses (quiet, MI with left/right bending, ME with left/right bending). It indicated that massage alleviates the pain for the patients of skeletal pain. Furthermore, compared with the SVM and LR classifiers, the BiLSTM method achieved a better area under curve (AUC) of 0.89 for the classification of MI with trunk left/right bending before massage. The AUC became smaller after massage than that before massage for the classification of MI with trunk left/right bending using CNN and BiLSTM methods. The Permutation direct indicator (PDI) score showed the significant difference for patients in different statuses (before vs after massage, and MI vs ME). Conclusions: Massage not only affects the quiet status, but also affects the MI and ME. Clinical Impact: Massage therapy may affect a bit on the accuracy of MI with trunk left/right bending and it change the topography of MI and ME with trunk left/right bending for the patients with skeletal muscle pain.

Long-Term BCI Training of a Tetraplegic User: Adaptive Riemannian Classifiers and User Training

While often presented as promising assistive technologies for motor-impaired users, electroencephalography (EEG)-based Brain-Computer Interfaces (BCIs) remain barely used outside laboratories due to low reliability in real-life conditions. There is thus a need to design long-term reliable BCIs that can be used outside-of-the-lab by end-users, e.g., severely motor-impaired ones. Therefore, we propose and evaluate the design of a multi-class Mental Task (MT)-based BCI for longitudinal training (20 sessions over 3 months) of a tetraplegic user for the CYBATHLON BCI series 2019. In this BCI championship, tetraplegic pilots are mentally driving a virtual car in a racing video game. We aimed at combining a progressive user MT-BCI training with a newly designed machine learning pipeline based on adaptive Riemannian classifiers shown to be promising for real-life applications. We followed a two step training process: the first 11 sessions served to train the user to control a 2-class MT-BCI by performing either two cognitive tasks (REST and MENTAL SUBTRACTION) or two motor-imagery tasks (LEFT-HAND and RIGHT-HAND). The second training step (9 remaining sessions) applied an adaptive, session-independent Riemannian classifier that combined all 4 MT classes used before. Moreover, as our Riemannian classifier was incrementally updated in an unsupervised way it would capture both within and between-session non-stationarity. Experimental evidences confirm the effectiveness of this approach. Namely, the classification accuracy improved by about 30% at the end of the training compared to initial sessions. We also studied the neural correlates of this performance improvement. Using a newly proposed BCI user learning metric, we could show our user learned to improve his BCI control by producing EEG signals matching increasingly more the BCI classifier training data distribution, rather than by improving his EEG class discrimination. However, the resulting improvement was effective only on synchronous (cue-based) BCI and it did not translate into improved CYBATHLON BCI game performances. For the sake of overcoming this in the future, we unveil possible reasons for these limited gaming performances and identify a number of promising future research directions. Importantly, we also report on the evolution of the user's neurophysiological patterns and user experience throughout the BCI training and competition.

A New - Discriminative and Multi- Filter Bank Tangent Space Mapping Method for Recognition of Multiclass Motor

Objective: Tangent Space Mapping (TSM) using the geometric structure of the covariance matrices is an effective method to recognize multiclass motor imagery (MI). Compared with the traditional CSP method, the Riemann geometric method based on TSM takes into account the nonlinear information contained in the covariance matrix, and can extract more abundant and effective features. Moreover, the method is an unsupervised operation, which can reduce the time of feature extraction. However, EEG features induced by MI mental activities of different subjects are not the same, so selection of subject-specific discriminative EEG frequency components play a vital role in the recognition of multiclass MI. In order to solve the problem, a discriminative and multi-scale filter bank tangent space mapping (DMFBTSM) algorithm is proposed in this article to design the subject-specific Filter Bank (FB) so as to effectively recognize multiclass MI tasks. Methods: On the 4-class BCI competition IV-2a dataset, first, a non-parametric method of multivariate analysis of variance (MANOVA) based on the sum of squared distances is used to select discriminative frequency bands for a subject; next, a multi-scale FB is generated according to the range of these frequency bands, and then decompose multi-channel EEG of the subject into multiple sub-bands combined with several time windows. Then TSM algorithm is used to estimate Riemannian tangent space features in each sub-band and finally a liner Support Vector Machines (SVM) is used for classification. Main Results: The analysis results show that the proposed discriminative FB enhances the multi-scale TSM algorithm, improves the classification accuracy and reduces the execution time during training and testing. On the 4-class BCI competition IV-2a dataset, the average session to session classification accuracy of nine subjects reached 77.33 ± 12.3%. When the training time and the test time are similar, the average classification accuracy is 2.56% higher than the latest TSM method based on multi-scale filter bank analysis technology. When the classification accuracy is similar, the training speed is increased by more than three times, and the test speed is increased two times more. Compared with Supervised Fisher Geodesic Minimum Distance to the Mean (Supervised FGMDRM), another new variant based on Riemann geometry classifier, the average accuracy is 3.36% higher, we also compared with the latest Deep Learning method, and the average accuracy of 10-fold cross validation improved by 2.58%. Conclusion: Research shows that the proposed DMFBTSM algorithm can improve the classification accuracy of MI tasks. Significance: Compared with the MFBTSM algorithm, the algorithm proposed in this article is expected to select frequency bands with good separability for specific subject to improve the classification accuracy of multiclass MI tasks and reduce the feature dimension to reduce training time and testing time.

Dynamic Joint Domain Adaptation Network for Motor Imagery Classification

Electroencephalogram (EEG) has been widely used in brain computer interface (BCI) due to its convenience and reliability. The EEG-based BCI applications are majorly limited by the time-consuming calibration procedure for discriminative feature representation and classification. Existing EEG classification methods either heavily depend on the handcrafted features or require adequate annotated samples at each session for calibration. To address these issues, we propose a novel dynamic joint domain adaptation network based on adversarial learning strategy to learn domain-invariant feature representation, and thus improve EEG classification performance in the target domain by leveraging useful information from the source session. Specifically, we explore the global discriminator to align the marginal distribution across domains, and the local discriminator to reduce the conditional distribution discrepancy between sub-domains via conditioning on deep representation as well as the predicted labels from the classifier. In addition, we further investigate a dynamic adversarial factor to adaptively estimate the relative importance of alignment between the marginal and conditional distributions. To evaluate the efficacy of our method, extensive experiments are conducted on two public EEG datasets, namely, Datasets IIa and IIb of BCI Competition IV. The experimental results demonstrate that the proposed method achieves superior performance compared with the state-of-the-art methods.

Abdul Majeed A. The classification of motor imagery response: an accuracy enhancement through the ensemble of random subspace k-NN

Brain-computer interface (BCI) is a viable alternative communication strategy for patients of neurological disorders as it facilitates the translation of human intent into device commands. The performance of BCIs primarily depends on the efficacy of the feature extraction and feature selection techniques, as well as the classification algorithms employed. More often than not, high dimensional feature set contains redundant features that may degrade a given classifier's performance. In the present investigation, an ensemble learning-based classification algorithm, namely random subspace k-nearest neighbour (k-NN) has been proposed to classify the motor imagery (MI) data. The common spatial pattern (CSP) has been applied to extract the features from the MI response, and the effectiveness of random forest (RF)-based feature selection algorithm has also been investigated. In order to evaluate the efficacy of the proposed method, an experimental study has been implemented using four publicly available MI dataset (BCI Competition III dataset 1 (data-1), dataset IIIA (data-2), dataset IVA (data-3) and BCI Competition IV dataset II (data-4)). It was shown that the ensemble-based random subspace k-NN approach achieved the superior classification accuracy (CA) of 99.21%, 93.19%, 93.57% and 90.32% for data-1, data-2, data-3 and data-4, respectively against other models evaluated, namely linear discriminant analysis, support vector machine, random forest, Naïve Bayes and the conventional k-NN. In comparison with other classification approaches reported in the recent studies, the proposed method enhanced the accuracy by 2.09% for data-1, 1.29% for data-2, 4.95% for data-3 and 5.71% for data-4, respectively. Moreover, it is worth highlighting that the RF feature selection technique employed in the present study was able to significantly reduce the feature dimension without compromising the overall CA. The outcome from the present study implies that the proposed method may significantly enhance the accuracy of MI data classification.

Inhibitory control dysfunction in parkinsonian impulse control disorders

Impulse control disorders (ICDs) in Parkinson's disease have been associated with dysfunctions in the control of value- or reward-based responding (choice impulsivity) and abnormalities in mesocorticolimbic circuits. The hypothesis that dysfunctions in the control of response inhibition (action impulsivity) also play a role in Parkinson's disease ICDs has recently been raised, but the underlying neural mechanisms have not been probed directly. We used high-resolution EEG recordings from 41 patients with Parkinson's disease with and without ICDs to track the spectral and dynamical signatures of different mechanisms involved in inhibitory control in a simple visuomotor task involving no selection between competing responses and no reward to avoid potential confounds with reward-based decision. Behaviourally, patients with Parkinson's disease with ICDs proved to be more impulsive than those without ICDs. This was associated with decreased beta activity in the precuneus and in a region of the medial frontal cortex centred on the supplementary motor area. The underlying dynamical patterns pinpointed dysfunction of proactive inhibitory control, an executive mechanism intended to gate motor responses in anticipation of stimulation in uncertain contexts. The alteration of the cortical drive of proactive response inhibition in Parkinson's disease ICDs pinpoints the neglected role the precuneus might play in higher order executive functions in coordination with the supplementary motor area, specifically for switching between executive settings. Clinical perspectives are discussed in the light of the non-dopaminergic basis of this function.

An Introductory Tutorial on Brain–Computer Interfaces and Their Applications

The prospect and potentiality of interfacing minds with machines has long captured human imagination. Recent advances in biomedical engineering, computer science, and neuroscience are making brain–computer interfaces a reality, paving the way to restoring and potentially augmenting human physical and mental capabilities. Applications of brain–computer interfaces are being explored in applications as diverse as security, lie detection, alertness monitoring, gaming, education, art, and human cognition augmentation. The present tutorial aims to survey the principal features and challenges of brain–computer interfaces (such as reliable acquisition of brain signals, filtering and processing of the acquired brainwaves, ethical and legal issues related to brain–computer interface (BCI), data privacy, and performance assessment) with special emphasis to biomedical engineering and automation engineering applications. The content of this paper is aimed at students, researchers, and practitioners to glimpse the multifaceted world of brain–computer interfacing.