Common Spatial Pattern Reformulated for Regularizations in Brain-Computer Interfaces

Enhancing robustness of spatial filters in motor imagery based brain-computer interface via temporal learning

BACKGROUND

In motor imagery-based brain-computer interface (MI-BCI) EEG decoding, spatial filtering play a crucial role in feature extraction. Recent studies have emphasized the importance of temporal filtering for extracting discriminative features in MI tasks. While many efforts have been made to optimize feature extraction externally, stabilizing features from spatial filtering remains underexplored.

NEW METHOD

To address this problem, we propose an approach to improve the robustness of temporal features by minimizing instability in the temporal domain. Specifically, we utilize Jensen-Shannon divergence to quantify temporal instability and integrate decision variables to construct an objective function that minimizes this instability. Our method enhances the stability of variance and mean values in the extracted features, improving the identification of discriminative features and reducing the effects of instability.

RESULTS

The proposed method was applied to spatial filtering models, and tested on two publicly datasets as well as a self-collected dataset. Results demonstrate that the proposed method significantly boosts classification accuracy, confirming its effectiveness in enhancing temporal feature stability.

COMPARISON WITH EXISTING METHODS

We compared our method with spatial filtering methods, and the-state-of-the-art models. The proposed approach achieves the highest accuracy, with 92.43 % on BCI competition III IVa dataset, 84.45 % on BCI competition IV 2a dataset, and 73.18 % on self-collected dataset.

CONCLUSIONS

Enhancing the instability of temporal features contributes to improved MI-BCI performance. This not only improves classification performance but also provides a stable foundation for future advancements. The proposed method shows great potential for EEG decoding.

Dynamic Hierarchical Convolutional Attention Network for Recognizing Motor Imagery Intention

The neural activity patterns of localized brain regions are crucial for recognizing brain intentions. However, existing electroencephalogram (EEG) decoding models, especially those based on deep learning, predominantly focus on global spatial features, neglecting valuable local information, potentially leading to suboptimal performance. Therefore, this study proposed a dynamic hierarchical convolutional attention network (DH-CAN) that comprehensively learned discriminative information from both global and local spatial domains, as well as from time-frequency domains in EEG signals. Specifically, a multiscale convolutional block was designed to dynamically capture time-frequency information. The channels of EEG signals were mapped to different brain regions based on motor imagery neural activity patterns. The spatial features, both global and local, were then hierarchically extracted to fully exploit the discriminative information. Furthermore, regional connectivity was established using a graph attention network, incorporating it into the local spatial features. Particularly, this study shared network parameters between symmetrical brain regions to better capture asymmetrical motor imagery patterns. Finally, the learned multilevel features were integrated through a high-level fusion layer. Extensive experimental results on two datasets demonstrated that the proposed model performed excellently across multiple evaluation metrics, exceeding existing benchmark methods. These findings suggested that the proposed model offered a novel perspective for EEG decoding research.

Developing Brain-Based Bare-Handed Human-Machine Interaction via On-Skin Input

Developing natural, intuitive, and human-centric input systems for mobile human-machine interaction (HMI) poses significant challenges. Existing gaze or gesture-based interaction systems are often constrained by their dependence on continuous visual engagement, limited interaction surfaces, or cumbersome hardware. To address these challenges, we propose MetaSkin, a novel neurohaptic interface that uniquely integrates neural signals with on-skin interaction for bare-handed, eyes-free interaction by exploiting human's natural proprioceptive capabilities. To support the interface, we developed a deep learning framework that employs multiscale temporal-spectral feature representation and selective feature attention to effectively decode neural signals generated by on-skin touch and motion gestures. In experiments with 12 participants, our method achieved offline accuracies of 81.95% for touch location discrimination, 71.00% for motion type identification, and 46.08% for 10-class touch-motion classification. In pseudo-online settings, accuracies reached 99.43% for touch onset detection, and 80.34% and 67.02% for classification of touch location and motion type, respectively. Neurophysiological analyses revealed distinct neural activation patterns in the sensorimotor cortex, underscoring the efficacy of our multiscale approach in capturing rich temporal and spectral dynamics. Future work will focus on optimizing the system for diverse user populations and dynamic environments, with a long-term goal of advancing human-centered, neuroadaptive interfaces for next-generation HMI systems. This work represents a significant step toward a paradigm shift in design of brain-computer interfaces, bridging sensory and motor paradigms for building more sophisticated systems.

Multi-layer transfer learning algorithm based on improved common spatial pattern for brain-computer interfaces

In the application of brain-computer interface, the differences in imaging methods and brain structure between subjects hinder the effectiveness of decoding algorithms when applied on different subjects. Transfer learning has been designed to solve this problem. There have been many applications of transfer learning in motor imagery (MI), however the effectiveness is still limited due to the inconsistent domain alignment, lack of prominent data features and allocation of weights in trails. In this paper, a Multi-layer transfer learning algorithm based on improved Common Spatial Patterns (MTICSP) was proposed to solve these problems. Firstly, the source domain data and target domain data were aligned by Target Alignment (TA)method to reduce distribution differences between subjects. Secondly, the mean covariance matrix of the two classes was re-weighted by calculating the distance between the covariance matrix of each trial in the source domain and the target domain. Thirdly, the improved Common Spatial Patterns (CSP) by introducing regularization coefficient was proposed to further reduce the difference between source domain and target domain to extract features. Finally, the feature blocks of the source domain and target domain were aligned again by Joint Distribution Adaptation (JDA) method. Experiments on two public datasets in two transfer paradigms multi-source to single-target (MTS) and single-source to single-target (STS) verified the effectiveness of our proposed method. The MTS and STS in the 5-person dataset were 80.21% and 77.58%, respectively, and 80.10% and 73.91%, respectively, in the 9-person dataset. Experimental results also showed that the proposed algorithm was superior to other state-of-the-art algorithms. In addition, the generalization ability of our algorithm MTICSP was validated on the fatigue EEG dataset collected by ourselves, and obtained 94.83% and 87.41% accuracy in MTS and STS experiments respectively. The proposed method combines improved CSP with transfer learning to extract the features of source and target domains effectively, providing a new method for combining transfer learning with motor imagination.

Granger Causal Inference Based on Dual Laplacian Distribution and Its Application to MI-BCI Classification

Granger causality-based effective brain connectivity provides a powerful tool to probe the neural mechanism for information processing and the potential features for brain computer interfaces. However, in real applications, traditional Granger causality is prone to the influence of outliers, such as inevitable ocular artifacts, resulting in unreasonable brain linkages and the failure to decipher inherent cognition states. In this work, motivated by constructing the sparse causality brain networks under the strong physiological outlier noise conditions, we proposed a dual Laplacian Granger causality analysis (DLap-GCA) by imposing Laplacian distributions on both model parameters and residuals. In essence, the first Laplacian assumption on residuals will resist the influence of outliers in electroencephalogram (EEG) on causality inference, and the second Laplacian assumption on model parameters will sparsely characterize the intrinsic interactions among multiple brain regions. Through simulation study, we quantitatively verified its effectiveness in suppressing the influence of complex outliers, the stable capacity for model estimation, and sparse network inference. The application to motor-imagery (MI) EEG further reveals that our method can effectively capture the inherent hemispheric lateralization of MI tasks with sparse patterns even under strong noise conditions. The MI classification based on the network features derived from the proposed approach shows higher accuracy than other existing traditional approaches, which is attributed to the discriminative network structures being captured in a timely manner by DLap-GCA even under the single-trial online condition. Basically, these results consistently show its robustness to the influence of complex outliers and the capability of characterizing representative brain networks for cognition information processing, which has the potential to offer reliable network structures for both cognitive studies and future brain-computer interface (BCI) realization.

Online Privacy-Preserving EEG Classification by Source-Free Transfer Learning

Electroencephalogram (EEG) signals play an important role in brain-computer interface (BCI) applications. Recent studies have utilized transfer learning to assist the learning task in the new subject, i.e., target domain, by leveraging beneficial information from previous subjects, i.e., source domains. Nevertheless, EEG signals involve sensitive personal mental and health information. Thus, privacy concern becomes a critical issue. In addition, existing methods mostly assume that a portion of the new subject's data is available and perform alignment or adaptation between the source and target domains. However, in some practical scenarios, new subjects prefer prompt BCI utilization over the time-consuming process of collecting data for calibration and adaptation, which makes the above assumption difficult to hold. To address the above challenges, we propose Online Source-Free Transfer Learning (OSFTL) for privacy-preserving EEG classification. Specifically, the learning procedure contains offline and online stages. At the offline stage, multiple model parameters are obtained based on the EEG samples from multiple source subjects. OSFTL only needs access to these source model parameters to preserve the privacy of the source subjects. At the online stage, a target classifier is trained based on the online sequence of EEG instances. Subsequently, OSFTL learns a weighted combination of the source and target classifiers to obtain the final prediction for each target instance. Moreover, to ensure good transferability, OSFTL dynamically updates the transferred weight of each source domain based on the similarity between each source classifier and the target classifier. Comprehensive experiments on both simulated and real-world applications demonstrate the effectiveness of the proposed method, indicating the potential of OSFTL to facilitate the deployment of BCI applications outside of controlled laboratory settings.

A Least-Square Unified Framework for Spatial Filtering in SSVEP-Based BCIs

The steady-state visual evoked potential (SSVEP) has become one of the most prominent BCI paradigms with high information transfer rate, and has been widely applied in rehabilitation and assistive applications. This paper proposes a least-square (LS) unified framework to summarize the correlation analysis (CA)-based SSVEP spatial filtering methods from a machine learning perspective. Within this framework, the commonalities and differences between various spatial filtering methods appear apparent, the interpretation of computational factors becomes intuitive, and spatial filters can be determined by solving a generalized optimization problem with non-linear and regularization items. Moreover, the proposed LS framework provides the foundation of utilizing the knowledge behind these spatial filtering methods in further classification/regression model designs. Through a comparative analysis of existing representative spatial filtering methods, recommendations are made for the superior and robust design strategies. These recommended strategies are further integrated to fill the research gaps and demonstrate the ability of the proposed LS framework to promote algorithmic improvements, resulting in five new spatial filtering methods. This study could offer significant insights in understanding the relationships between various design strategies in the spatial filtering methods from the machine learning perspective, and would also contribute to the development of the SSVEP recognition methods with high performance.

ADFCNN: Attention-Based Dual-Scale Fusion Convolutional Neural Network for Motor Imagery Brain-Computer Interface

Convolutional neural networks (CNNs) have been successfully applied to motor imagery (MI)-based brain-computer interface (BCI). Nevertheless, single-scale CNN fail to extract abundant information over a wide spectrum from EEG signals, while typical multi-scale CNNs cannot effectively fuse information from different scales with concatenation-based methods. To overcome these challenges, we propose a new scheme equipped with attention-based dual-scale fusion convolutional neural network (ADFCNN), which jointly extracts and fuses EEG spectral and spatial information at different scales. This scheme also provides novel insight through self-attention for effective information fusion from different scales. Specifically, temporal convolutions with two different kernel sizes identify EEG μ and β rhythms, while spatial convolutions at two different scales generate global and detailed spatial information, respectively, and the self-attention mechanism performs feature fusion based on the internal similarity of the concatenated features extracted by the dual-scale CNN. The proposed scheme achieves the superior performance compared with state-of-the-art methods in subject-specific motor imagery recognition on BCI Competition IV dataset 2a, 2b and OpenBMI dataset, with the cross-session average classification accuracies of 79.39% and significant improvements of 9.14% on BCI-IV2a, 87.81% and 7.66% on BCI-IV2b, 65.26% and 7.2% on OpenBMI dataset, and the within-session average classification accuracies of 86.87% and significant improvements of 10.89% on BCI-IV2a, 87.26% and 8.07% on BCI-IV2b, 84.29% and 5.17% on OpenBMI dataset, respectively. What is more, ablation experiments are conducted to investigate the mechanism and demonstrate the effectiveness of the dual-scale joint temporal-spatial CNN and self-attention modules. Visualization is also used to reveal the learning process and feature distribution of the model.

TSPNet: a time-spatial parallel network for classification of EEG-based multiclass upper limb motor imagery BCI

The classification of electroencephalogram (EEG) motor imagery signals has emerged as a prominent research focus within the realm of brain-computer interfaces. Nevertheless, the conventional, limited categories (typically just two or four) offered by brain-computer interfaces fail to provide an extensive array of control modes. To address this challenge, we propose the Time-Spatial Parallel Network (TSPNet) for recognizing six distinct categories of upper limb motor imagery. Within TSPNet, temporal and spatial features are extracted separately, with the time dimension feature extractor and spatial dimension feature extractor performing their respective functions. Following this, the Time-Spatial Parallel Feature Extractor is employed to decouple the connection between temporal and spatial features, thus diminishing feature redundancy. The Time-Spatial Parallel Feature Extractor deploys a gating mechanism to optimize weight distribution and parallelize time-spatial features. Additionally, we introduce a feature visualization algorithm based on signal occlusion frequency to facilitate a qualitative analysis of TSPNet. In a six-category scenario, TSPNet achieved an accuracy of 49.1% ± 0.043 on our dataset and 49.7% ± 0.029 on a public dataset. Experimental results conclusively establish that TSPNet outperforms other deep learning methods in classifying data from these two datasets. Moreover, visualization results vividly illustrate that our proposed framework can generate distinctive classifier patterns for multiple categories of upper limb motor imagery, discerned through signals of varying frequencies. These findings underscore that, in comparison to other deep learning methods, TSPNet excels in intention recognition, which bears immense significance for non-invasive brain-computer interfaces.

Real-Time Deep Neurolinguistic Learning Enhances Noninvasive Neural Language Decoding for Brain-Machine Interaction

Electroencephalogram (EEG)-based brain-machine interface (BMI) has been utilized to help patients regain motor function and has recently been validated for its use in healthy people because of its ability to directly decipher human intentions. In particular, neurolinguistic research using EEGs has been investigated as an intuitive and naturalistic communication tool between humans and machines. In this study, the human mind directly decoded the neural languages based on speech imagery using the proposed deep neurolinguistic learning. Through real-time experiments, we evaluated whether BMI-based cooperative tasks between multiple users could be accomplished using a variety of neural languages. We successfully demonstrated a BMI system that allows a variety of scenarios, such as essential activity, collaborative play, and emotional interaction. This outcome presents a novel BMI frontier that can interact at the level of human-like intelligence in real time and extends the boundaries of the communication paradigm.

Application of multi-task transfer learning: The combination of EA and optimized subband regularized CSP to classification of 8-channel EEG signals with small dataset

Introduction

The volume conduction effect and high dimensional characteristics triggered by the excessive number of channels of EEG cap-acquired signals in BCI systems can increase the difficulty of classifying EEG signals and the lead time of signal acquisition. We aim to combine transfer learning to decode EEG signals in the few-channel case, improve the classification performance of the motor imagery BCI system across subject cases, reduce the cost of signal acquisition performed by the BCI system, and improve the usefulness of the system.

Methods

Dataset2a from BCI CompetitionIV(2008) was used as Dataset1, and our team's self-collected dataset was used as Dataset2. Dataset1 acquired EEG signals from 9 subjects using a 22-channel device with a sampling frequency of 250 Hz. Dataset2 acquired EEG signals from 10 healthy subjects (8 males and 2 females; age distribution between 21-30 years old; mean age 25 years old) using an 8-channel system with a sampling frequency of 1000 Hz. We introduced EA in the data preprocessing process to reduce the signal differences between subjects and proposed VFB-RCSP in combination with RCSP and FBCSP to optimize the effect of feature extraction.

Results

Experiments were conducted on Dataset1 with EEG data containing only 8 channels and achieved an accuracy of 78.01 and a kappa coefficient of 0.54. The accuracy exceeded most of the other methods proposed in recent years, even though the number of channels used was significantly reduced. On Dataset 2, an accuracy of 59.77 and a Kappa coefficient of 0.34 were achieved, which is a significant improvement compared to other poorly improved classical protocols.

Discussion

Our work effectively improves the classification of few-channel EEG data. It overcomes the dependence of existing algorithms on the number of channels, the number of samples, and the frequency band, which is significant for reducing the complexity of BCI models and improving the user-friendliness of BCI systems.

Analysis of instantaneous brain interactions contribution to a motor imagery classification task

The purpose of this study is to analyze the contribution of the interactions between electrodes, measured either as correlation or as Jaccard distance, to the classification of two actions in a motor imagery paradigm, namely, left-hand movement and right-hand movement. The analysis is performed in two classifier models, namely, a static (linear discriminant analysis, LDA) model and a dynamic (hidden conditional random field, HCRF) model. The impact of using the sliding window technique (SWT) in the static and dynamic models is also analyzed. The study proved that their combination with temporal features provides significant information to improve the classification in a two-class motor imagery task for LDA (average accuracy: 0.7192 no additional features, 0.7617 by adding correlation, 0.7606 by adding Jaccard distance; p < 0.001) and HCRF (average accuracy: 0.7370 no additional features, 0.7764 by adding correlation, 0.7793 by adding Jaccard distance; p < 0.001). Also, we showed that adding interactions between electrodes improves significantly the performance of each classifier, regarding the nature of the interaction measure or the classifier itself.

NeuroGrasp: Real-Time EEG Classification of High-Level Motor Imagery Tasks Using a Dual-Stage Deep Learning Framework

Brain-computer interfaces (BCIs) have been widely employed to identify and estimate a user's intention to trigger a robotic device by decoding motor imagery (MI) from an electroencephalogram (EEG). However, developing a BCI system driven by MI related to natural hand-grasp tasks is challenging due to its high complexity. Although numerous BCI studies have successfully decoded large body parts, such as the movement intention of both hands, arms, or legs, research on MI decoding of high-level behaviors such as hand grasping is essential to further expand the versatility of MI-based BCIs. In this study, we propose NeuroGrasp, a dual-stage deep learning framework that decodes multiple hand grasping from EEG signals under the MI paradigm. The proposed method effectively uses an EEG and electromyography (EMG)-based learning, such that EEG-based inference at test phase becomes possible. The EMG guidance during model training allows BCIs to predict hand grasp types from EEG signals accurately. Consequently, NeuroGrasp improved classification performance offline, and demonstrated a stable classification performance online. Across 12 subjects, we obtained an average offline classification accuracy of 0.68 (±0.09) in four-grasp-type classifications and 0.86 (±0.04) in two-grasp category classifications. In addition, we obtained an average online classification accuracy of 0.65 (±0.09) and 0.79 (±0.09) across six high-performance subjects. Because the proposed method has demonstrated a stable classification performance when evaluated either online or offline, in the future, we expect that the proposed method could contribute to different BCI applications, including robotic hands or neuroprosthetics for handling everyday objects.

Use of common spatial patterns for early detection of Parkinson's disease

One of the most common diseases that affects human brain is Parkinson's disease. Detection of Parkinson's disease (PD) poses a serious challenge. Robust methods for feature extraction allowing separation between the electroencephalograms (EEG) of healthy subjects and PD patients are required. We used the EEG records of healthy subjects and PD patients which were subject to auditory tasks. We used the common spatial patterns (CSP) and Laplacian mask as methods to allow robust selection and extraction of features. We used the derived CSP whitening matrix to determine those channels that are the most promising in the terms of differentiating between EEGs of healthy controls and of PD patients. Using the selection of features calculated using the CSP we managed to obtain the classification accuracy of 85% when classifying EEG records belonging to groups of controls or PD patients. Using the features calculated using the Laplacian operator we obtained the classification accuracy of 90%. Diagnosing the PD in early stages using EEG is possible. The CSP proved to be a promising technique to detect informative channels and to separate between the groups. Use of the combination of features calculated using the Laplacian offers good separability between the two groups.

An asynchronous artifact-enhanced electroencephalogram based control paradigm assisted by slight facial expression

In this study, an asynchronous artifact-enhanced electroencephalogram (EEG)-based control paradigm assisted by slight-facial expressions (sFE-paradigm) was developed. The brain connectivity analysis was conducted to reveal the dynamic directional interactions among brain regions under sFE-paradigm. The component analysis was applied to estimate the dominant components of sFE-EEG and guide the signal processing. Enhanced by the artifact within the detected electroencephalogram (EEG), the sFE-paradigm focused on the mainstream defect as the insufficiency of real-time capability, asynchronous logic, and robustness. The core algorithm contained four steps, including “obvious non-sFE-EEGs exclusion,” “interface ‘ON’ detection,” “sFE-EEGs real-time decoding,” and “validity judgment.” It provided the asynchronous function, decoded eight instructions from the latest 100 ms signal, and greatly reduced the frequent misoperation. In the offline assessment, the sFE-paradigm achieved 96.46% ± 1.07 accuracy for interface “ON” detection and 92.68% ± 1.21 for sFE-EEGs real-time decoding, with the theoretical output timespan less than 200 ms. This sFE-paradigm was applied to two online manipulations for evaluating stability and agility. In “object-moving with a robotic arm,” the averaged intersection-over-union was 60.03 ± 11.53%. In “water-pouring with a prosthetic hand,” the average water volume was 202.5 ± 7.0 ml. During online, the sFE-paradigm performed no significant difference (P = 0.6521 and P = 0.7931) with commercial control methods (i.e., FlexPendant and Joystick), indicating a similar level of controllability and agility. This study demonstrated the capability of sFE-paradigm, enabling a novel solution to the non-invasive EEG-based control in real-world challenges.

A Domain Adaptation-Based Method for Classification of Motor Imagery EEG

Non-stationarity of EEG signals lead to high variability across sessions, which results in low classification accuracy. To reduce the inter-session variability, an unsupervised domain adaptation method is proposed. Arithmetic mean and covariance are exploited to represent the data distribution. First, overall mean alignment is conducted between the source and target data. Then, the data in the target domain is labeled by a classifier trained with the source data. The per-class mean and covariance of the target data are estimated based on the predicted labels. Next, an alignment from the source domain to the target domain is performed according to the covariance of each class in the target domain. Finally, per-class mean adaptation is required after covariance alignment to remove the shift of data distribution caused by covariance alignment. Two public BCI competition datasets, namely the BCI competition III dataset IVa and the BCI competition IV dataset IIa were used to evaluate the proposed method. On both datasets, the proposed method effectively improved classification accuracy.

Multiscale time-frequency method for multiclass Motor Imagery Brain Computer Interface

Motor Imagery Brain Computer Interface (MI-BCI) has become a promising technology in the field of neurorehabilitation. However, the performance and computational complexity of the current multiclass MI-BCI have not been fully optimized, and the intuitive interpretation of individual differences on motor imagery tasks is seldom investigated. In this paper, a well-designed multiscale time-frequency segmentation scheme is first applied to multichannel EEG recordings to obtain Time-Frequency Segments (TFSs). Then, the TFS selection based on a specific wrapper feature selection rule is utilized to determine optimum TFSs. Next, One-Versus-One (OvO)-divCSP implemented in divergence framework is used to extract discriminative features. Finally, One-Versus-Rest (OvR)-SVM is utilized to predict the class label based on selected multiclass MI features. Experimental results indicate our method yields a superior performance on two publicly available multiclass MI datasets with a mean accuracy of 80.00% and a mean kappa of 0.73. Meanwhile, the proposed TFS selection method can significantly alleviate the computational burden with little accuracy reduction, demonstrating the feasibility of real-time multiclass MI-BCI. Furthermore, the Motor Imagery Time-Frequency Reaction Map (MI-TFRM) is visualized, contributing to analyzing and interpreting the performance differences between different subjects.

Online Adaptation Boosts SSVEP-Based BCI Performance

OBJECTIVE

A user-friendly steady-state visual evoked potential (SSVEP)-based brain-computer interface (BCI) prefers no calibration for its target recognition algorithm, however, the existing calibration-free schemes perform still far behind their calibration-based counterparts. To tackle this issue, learning online from the subject's unlabeled data is investigated as a potential approach to boost the performance of the calibration-free SSVEP-based BCIs.

METHODS

An online adaptation scheme is developed to tune the spatial filters using the online unlabeled data from previous trials, and then developing the online adaptive canonical correlation analysis (OACCA) method.

RESULTS

A simulation study on two public SSVEP datasets (Dataset I and II) with a total of 105 subjects demonstrated that the proposed online adaptation scheme can boost the CCA's averaged information transfer rate (ITR) from 94.60 to 158.87 bits/min in Dataset I and from 85.80 to 123.91 bits/min in Dataset II. Furthermore, in our online experiment it boosted the CCA's ITR from 55.81 bits/min to 95.73 bits/min. More importantly, this online adaptation scheme can be easily combined with any spatial filtering-based algorithms to achieve online learning.

CONCLUSION

By online adaptation, the proposed OACCA performed much better than the calibration-free CCA, and comparable to the calibration-based algorithms.

SIGNIFICANCE

This work provides a general way for the SSVEP-based BCIs to learn online from unlabeled data and thus avoid calibration.

CNN based classification of motor imaginary using variational mode decomposed EEG-spectrum image

A novel approach of preprocessing EEG signals by generating spectrum image for effective Convolutional Neural Network (CNN) based classification for Motor Imaginary (MI) recognition is proposed. The approach involves extracting the Variational Mode Decomposition (VMD) modes of EEG signals, from which the Short Time Fourier Transform (STFT) of all the modes are arranged to form EEG spectrum images. The EEG spectrum images generated are provided as input image to CNN. The two generic CNN architectures for MI classification (EEGNet and DeepConvNet) and the architectures for pattern recognition (AlexNet and LeNet) are used in this study. Among the four architectures, EEGNet provides average accuracies of 91.37%, 94.41%, 85.67% and 90.21% for the four datasets used to validate the proposed approach. Consistently better results in comparison with results in recent literature demonstrate that the EEG spectrum image generation using VMD-STFT is a promising method for the time frequency analysis of EEG signals.

Covariation Informed Graph Slepians for Motor Imagery Decoding

Graph signal processing (GSP) provides signal analytic tools for data defined in irregular domains, as is the case of non-invasive electroencephalography (EEG). In this work, the recently introduced technique of Graph Slepian functions is exploited for the robust decoding of motor imagery (MI) brain activity. The particular technique builds over the concept of graph Fourier transform (GFT) and provides additional flexibility in the subsequent data analysis by incorporating domain knowledge. Based on contrastive learning, we introduce an algorithmic pipeline that attains a data driven and subject specific design of Graph Slepian functions. These functions, by incorporating both the topology of the sensor array and the empirical evidence about the differential functional covariation, act as spatial filters that enhance the information conveyed by the multichannel signal and specifically relates to the participant's intention. The proposed technique for crafting Graph Slepians is incorporated in a MI-decoding scheme, in which the informed projections are fed to a support vector machine (SVM) that casts a prediction regarding the type of intended movement. The employed MI-decoder is evaluated based on two publicly available datasets and its superiority against popular alternatives in the field is established. Computational efficiency is listed among its main advantages, since it involves only simple matrix operations, allowing to consider its use in real-time implementations.

The CSP-Based New Features Plus Non-Convex Log Sparse Feature Selection for Motor Imagery EEG Classification

The common spatial pattern (CSP) is a very effective feature extraction method in motor imagery based brain computer interface (BCI), but its performance depends on the selection of the optimal frequency band. Although a lot of research works have been proposed to improve CSP, most of these works have the problems of large computation costs and long feature extraction time. To this end, three new feature extraction methods based on CSP and a new feature selection method based on non-convex log regularization are proposed in this paper. Firstly, EEG signals are spatially filtered by CSP, and then three new feature extraction methods are proposed. We called them CSP-wavelet, CSP-WPD and CSP-FB, respectively. For CSP-Wavelet and CSP-WPD, the discrete wavelet transform (DWT) or wavelet packet decomposition (WPD) is used to decompose the spatially filtered signals, and then the energy and standard deviation of the wavelet coefficients are extracted as features. For CSP-FB, the spatially filtered signals are filtered into multiple bands by a filter bank (FB), and then the logarithm of variances of each band are extracted as features. Secondly, a sparse optimization method regularized with a non-convex log function is proposed for the feature selection, which we called LOG, and an optimization algorithm for LOG is given. Finally, ensemble learning is used for secondary feature selection and classification model construction. Combing feature extraction and feature selection methods, a total of three new EEG decoding methods are obtained, namely CSP-Wavelet+LOG, CSP-WPD+LOG, and CSP-FB+LOG. Four public motor imagery datasets are used to verify the performance of the proposed methods. Compared to existing methods, the proposed methods achieved the highest average classification accuracy of 88.86, 83.40, 81.53, and 80.83 in datasets 1–4, respectively. The feature extraction time of CSP-FB is the shortest. The experimental results show that the proposed methods can effectively improve the classification accuracy and reduce the feature extraction time. With comprehensive consideration of classification accuracy and feature extraction time, CSP-FB+LOG has the best performance and can be used for the real-time BCI system.