Deep learning techniques for classification of electroencephalogram (EEG) motor imagery (MI) signals: a review

Closed-loop rehabilitation of upper-limb dyskinesia after stroke: from natural motion to neuronal microfluidics

This review proposes an innovative closed-loop rehabilitation strategy that integrates multiple subdomains of stroke science to address the global challenge of upper-limb dyskinesia post-stroke. Despite advancements in neural remodeling and rehabilitation research, the compartmentalization of subdomains has limited the effectiveness of current rehabilitation strategies. Our approach unites key areas-including the post-stroke brain, upper-limb rehabilitation robotics, motion sensing, metrics, neural microfluidics, and neuroelectronics-into a cohesive framework designed to enhance upper-limb motion rehabilitation outcomes. By leveraging cutting-edge technologies such as lightweight rehabilitation robotics, advanced motion sensing, and neural microfluidic models, this strategy enables real-time monitoring, adaptive interventions, and personalized rehabilitation plans. Furthermore, we explore the potential of closed-loop systems to drive neural plasticity and functional recovery, offering a transformative perspective on stroke rehabilitation. Finally, we discuss future directions, emphasizing the integration of emerging technologies and interdisciplinary collaboration to advance the field. This review highlights the promise of closed-loop strategies in achieving unprecedented integration of subdomains and improving post-stroke upper-limb rehabilitation outcomes.

Augmenting brain-computer interfaces with ART: An artifact removal transformer for reconstructing multichannel EEG signals

Artifact removal in electroencephalography (EEG) is a longstanding challenge that significantly impacts neuroscientific analysis and brain-computer interface (BCI) performance. Tackling this problem demands advanced algorithms, extensive noisy-clean training data, and thorough evaluation strategies. This study presents the Artifact Removal Transformer (ART), an innovative EEG denoising model employing transformer architecture to adeptly capture the transient millisecond-scale dynamics characteristic of EEG signals. Our approach offers a holistic, end-to-end denoising solution that simultaneously addresses multiple artifact types in multichannel EEG data. We enhanced the generation of noisy-clean EEG data pairs using an independent component analysis, thus fortifying the training scenarios critical for effective supervised learning. We performed comprehensive validations using a wide range of open datasets from various BCI applications, employing metrics like mean squared error and signal-to-noise ratio, as well as sophisticated techniques such as source localization and EEG component classification. Our evaluations confirm that ART surpasses other deep-learning-based artifact removal methods, setting a new benchmark in EEG signal processing. This advancement not only boosts the accuracy and reliability of artifact removal but also promises to catalyze further innovations in the field, facilitating the study of brain dynamics in naturalistic environments.

A deep learning framework leveraging spatiotemporal feature fusion for electrophysiological source imaging

BACKGROUND AND OBJECTIVES

Electrophysiological source imaging (ESI) is a challenging technique for noninvasively measuring brain activity, which involves solving a highly ill-posed inverse problem. Traditional methods attempt to address this challenge by imposing various priors, but considering the complexity and dynamic nature of the brain activity, these priors may not accurately reflect the true attributes of brain sources. In this study, we propose a novel deep learning-based framework, spatiotemporal source imaging network (SSINet), designed to provide accurate spatiotemporal estimates of brain activity using electroencephalography (EEG).

METHODS

SSINet integrates a residual network (ResBlock) for spatial feature extraction and a bidirectional LSTM for capturing temporal dynamics, fused through a Transformer module to capture global dependencies. A channel attention mechanism is employed to prioritize active brain regions, improving both the accuracy of the model and its interpretability. Additionally, a weighted loss function is introduced to address the spatial sparsity of the brain activity.

RESULTS

We evaluated the performance of SSINet through numerical simulations and found that it outperformed several state-of-the-art ESI methods across various conditions, such as varying numbers of sources, source range, and signal-to-noise ratio levels. Furthermore, SSINet demonstrated robust performance even with electrode position offsets and changes in conductivity. We also validated the model on three real EEG datasets: visual, auditory, and somatosensory stimuli. The results show that the source activity reconstructed by SSINet aligns closely with the established physiological basis of brain function.

CONCLUSIONS

SSINet provides accurate and stable source imaging results.

EEG Signal Prediction for Motor Imagery Classification in Brain-Computer Interfaces

Brain-computer interfaces (BCIs) based on motor imagery (MI) generally require EEG signals recorded from a large number of electrodes distributed across the cranial surface to achieve accurate MI classification. Not only does this entail long preparation times and high costs, but it also carries the risk of losing valuable information when an electrode is damaged, further limiting its practical applicability. In this study, a signal prediction-based method is proposed to achieve high accuracy in MI classification using EEG signals recorded from only a small number of electrodes. The signal prediction model was constructed using the elastic net regression technique, allowing for the estimation of EEG signals from 22 complete channels based on just 8 centrally located channels. The predicted EEG signals from the complete channels were used for feature extraction and MI classification. The results obtained indicate a notable efficacy of the proposed prediction method, showing an average performance of 78.16% in classification accuracy. The proposed method demonstrated superior performance compared to the traditional approach that used few-channel EEG and also achieved better results than the traditional method based on full-channel EEG. Although accuracy varies among subjects, from 62.30% to an impressive 95.24%, these data indicate the capability of the method to provide accurate estimates from a reduced set of electrodes. This performance highlights its potential to be implemented in practical MI-based BCI applications, thereby mitigating the time and cost constraints associated with systems that require a high density of electrodes.

EEGConvNeXt: A novel convolutional neural network model for automated detection of Alzheimer's Disease and Frontotemporal Dementia using EEG signals

BACKGROUND AND OBJECTIVE

Deep learning models have gained widespread adoption in healthcare for accurate diagnosis through the analysis of brain signals. Neurodegenerative disorders like Alzheimer's Disease (AD) and Frontotemporal Dementia (FD) are increasingly prevalent due to age-related brain volume reduction. Despite advances, existing models often lack comprehensive multi-class classification capabilities and are computationally expensive. This study addresses these gaps by proposing EEGConvNeXt, a novel convolutional neural network (CNN) model for detecting AD and FD using electroencephalogram (EEG) signals with high accuracy.

MATERIALS AND METHOD

In this research, we employ an open-access EEG signal public dataset containing three distinct classes: AD, FD, and control subjects. We then constructed a newly proposed EEGConvNeXt model comprised of a 2-dimensional CNN algorithm that firstly converts the EEG signals into power spectrogram-based images. Secondly, these images were used as input for the proposed EEGConvNeXt model for automated classification of AD, FD, and a control outcome. The proposed EEGConvNeXt model is therefore a lightweight model that contributes to a new image classification CNN structure based on the transformer model with four primary stages: a stem, a main model, downsampling, and an output stem.

RESULTS

The EEGConvNeXt model achieved a classification accuracy of ∼95.70% for three-class detection (AD, FD, and control), validated using a hold-out strategy. Binary classification cases, such as AD versus FD and FD versus control, achieved accuracies exceeding 98%, demonstrating the model's robustness across scenarios.

CONCLUSIONS

The proposed EEGConvNeXt model demonstrates high classification performance with a lightweight architecture suitable for deployment in resource-constrained settings. While the study establishes a novel framework for AD and FD detection, limitations include reliance on a relatively small dataset and the need for further validation on diverse populations. Future research should focus on expanding datasets, optimizing architecture, and exploring additional neurological disorders to enhance the model's utility in clinical applications.

A time segment adaptive optimization method based on separability criterion and correlation analysis for motor imagery BCIs

Motor imagery (MI) plays a crucial role in brain-computer interface (BCI), and the classification of MI tasks using electroencephalogram (EEG) is currently under extensive investigation. During MI classification, individual differences among subjects in terms of response and time latency need to be considered. Optimizing the time segment for different subjects can enhance subsequent classification performance. In view of the individual differences of subjects in motor imagery tasks, this article proposes a Time Segment Adaptive Optimization method based on Separability criterion and Correlation analysis (TSAOSC). The fundamental principle of this method involves applying the separability criterion to various sizes of time windows within the training data, identifying the optimal raw reference signal, and adaptively adjusting the time segment position for each trial's data by analyzing its relationship with the optimal reference signal. We evaluated our method on three BCI competition datasets, respectively. The utilization of the TSAOSC method in the experiments resulted in an enhancement of 4.90% in average classification accuracy compared to its absence. Additionally, building upon the TSAOSC approach, this study proposes a Nonlinear-TSAOSC method (N-TSAOSC) for analyzing EEG signals with nonlinearity, which shows improvements in the classification accuracy of certain subjects. The results of the experiments demonstrate that the proposed method is an effective time segment optimization method, and it can be integrated into other algorithms to further improve their accuracy.

Feature-aware domain invariant representation learning for EEG motor imagery decoding

Electroencephalography (EEG)-based motor imagery (MI) is extensively utilized in clinical rehabilitation and virtual reality-based movement control. Decoding EEG-based MI signals is challenging because of the inherent spatio-temporal variability of the original signal representation, coupled with a low signal-to-noise ratio (SNR), which impedes the extraction of clean and robust features. To address this issue, we propose a multi-scale spatio-temporal domain-invariant representation learning method, termed MSDI. By decomposing the original signal into spatial and temporal components, the proposed method extracts invariant features at multiple scales from both components. To further constrain the representation to invariant domains, we introduce a feature-aware shift operation that resamples the representation based on its feature statistics and feature measure, thereby projecting the features into a domain-invariant space. We evaluate our proposed method via two publicly available datasets, BNCI2014-001 and BNCI2014-004, demonstrating state-of-the-art performance on both datasets. Furthermore, our method exhibits superior time efficiency and noise resistance.

Recent applications of EEG-based brain-computer-interface in the medical field

Brain-computer interfaces (BCIs) represent an emerging technology that facilitates direct communication between the brain and external devices. In recent years, numerous review articles have explored various aspects of BCIs, including their fundamental principles, technical advancements, and applications in specific domains. However, these reviews often focus on signal processing, hardware development, or limited applications such as motor rehabilitation or communication. This paper aims to offer a comprehensive review of recent electroencephalogram (EEG)-based BCI applications in the medical field across 8 critical areas, encompassing rehabilitation, daily communication, epilepsy, cerebral resuscitation, sleep, neurodegenerative diseases, anesthesiology, and emotion recognition. Moreover, the current challenges and future trends of BCIs were also discussed, including personal privacy and ethical concerns, network security vulnerabilities, safety issues, and biocompatibility.

Recognition of brain activities via graph-based long short-term memory-convolutional neural network

Introduction

Human brain activities are always difficult to recognize due to its diversity and susceptibility to disturbance. With its unique capability of measuring brain activities, magnetoencephalography (MEG), as a high temporal and spatial resolution neuroimaging technique, has been used to identify multi-task brain activities. Accurately and robustly classifying motor imagery (MI) and cognitive imagery (CI) from MEG signals is a significant challenge in the field of brain-computer interface (BCI).

Methods

In this study, a graph-based long short-term memory-convolutional neural network (GLCNet) is proposed to classify the brain activities in MI and CI tasks. It was characterized by implementing three modules of graph convolutional network (GCN), spatial convolution and long short-term memory (LSTM) to effectively extract time-frequency-spatial features simultaneously. For performance evaluation, our method was compared with six benchmark algorithms of FBCSP, FBCNet, EEGNet, DeepConvNets, Shallow ConvNet and MEGNet on two public datasets of MEG-BCI and BCI competition IV dataset 3.

Results

The results demonstrated that the proposed GLCNet outperformed other models with the average accuracies of 78.65% and 65.8% for two classification and four classification on the MEG-BCI dataset, respectively.

Discussion

It was concluded that the GLCNet enhanced the model's adaptability in handling individual variability with robust performance. This would contribute to the exploration of brain activates in neuroscience.

The More, the Better? Evaluating the Role of EEG Preprocessing for Deep Learning Applications

The last decade has witnessed a notable surge in deep learning applications for electroencephalography (EEG) data analysis, showing promising improvements over conventional statistical techniques. However, deep learning models can underperform if trained with bad processed data. Preprocessing is crucial for EEG data analysis, yet there is no consensus on the optimal strategies in deep learning scenarios, leading to uncertainty about the extent of preprocessing required for optimal results. This study is the first to thoroughly investigate the effects of EEG preprocessing in deep learning applications, drafting guidelines for future research. It evaluates the effects of varying preprocessing levels, from raw and minimally filtered data to complex pipelines with automated artifact removal algorithms. Six classification tasks (eye blinking, motor imagery, Parkinson's, Alzheimer's disease, sleep deprivation, and first episode psychosis) and four established EEG architectures were considered for the evaluation. The analysis of 4800 trained models revealed statistical differences between preprocessing pipelines at the intra-task level for each model and at the inter-task level for the largest model. Models trained on raw data consistently performed poorly, always ranking last in average scores. In addition, models seem to benefit more from minimal pipelines without artifact handling methods. These findings suggest that EEG artifacts may affect the performance and generalizability of deep neural networks.

REI-Net: A Reference Electrode Standardization Interpolation Technique Based 3D CNN for Motor Imagery Classification

High-quality scalp EEG datasets are extremely valuable for motor imagery (MI) analysis. However, due to electrode size and montage, different datasets inevitably experience channel information loss, posing a significant challenge for MI decoding. A 2D representation that focuses on the time domain may loss the spatial information in EEG. In contrast, a 3D representation based on topography may suffer from channel loss and introduce noise through different padding methods. In this paper, we propose a framework called Reference Electrode Standardization Interpolation Network (REI-Net). Through an interpolation of 3D representation, REI-Net retains the temporal information in 2D scalp EEG while improving the spatial resolution within a certain montage. Additionally, to overcome the data variability caused by individual differences, transfer learning is employed to enhance the decoding robustness. Our approach achieves promising performance on two widely-recognized MI datasets, with an accuracy of 77.99% on BCI-C IV-2a and an accuracy of 63.94% on Kaya2018. The proposed algorithm outperforms the SOTAs leading to more accurate and robust results.

ESI-GAL: EEG source imaging-based trajectory estimation for grasp and lift task

BACKGROUND

Electroencephalogram (EEG) signals-based motor kinematics prediction (MKP) has been an active area of research to develop Brain-computer interface (BCI) systems such as exosuits, prostheses, and rehabilitation devices. However, EEG source imaging (ESI) based kinematics prediction is sparsely explored in the literature.

METHOD

In this study, pre-movement EEG features are utilized to predict three-dimensional (3D) hand kinematics for the grasp-and-lift motor task. A public dataset, WAY-EEG-GAL, is utilized for MKP analysis. In particular, sensor-domain (EEG data) and source-domain (ESI data) based features from the frontoparietal region are explored for MKP. Deep learning-based models are explored to achieve efficient kinematics decoding. Various time-lagged and window sizes are analyzed for hand kinematics prediction. Subsequently, intra-subject and inter-subject MKP analysis is performed to investigate the subject-specific and subject-independent motor-learning capabilities of the neural decoders. The Pearson correlation coefficient (PCC) is used as the performance metric for kinematics trajectory decoding.

RESULTS

The rEEGNet neural decoder achieved the best performance with sensor-domain and source-domain features with the time lag and window size of 100ms and 450ms, respectively. The highest mean PCC values of 0.790, 0.795, and 0.637 are achieved using sensor-domain features, while 0.769, 0.777, and 0.647 are achieved using source-domain features in x, y, and z-directions, respectively.

CONCLUSION

This study explores the feasibility of trajectory prediction using EEG sensor-domain and source-domain features for the grasp-and-lift task. Furthermore, inter-subject trajectory estimation is performed using the proposed deep learning decoder with EEG source domain features.

Electroencephalogram (EEG) Based Fuzzy Logic and Spiking Neural Networks (FLSNN) for Advanced Multiple Neurological Disorder Diagnosis

Neurological disorders are a major global health concern that have a substantial impact on death rates and quality of life. accurately identifying a number of diseases Due to inherent data uncertainties and Electroencephalogram (EEG) pattern overlap, conventional EEG diagnosis methods frequently encounter difficulties. This paper proposes a novel framework that integrates FLSNN to enhance the accuracy and robustness of multiple neurological disorder disease detection from EEG signals. In multiple neurological disorders, the primary motivation is to overcome the limitations of existing methods that are unable to handle the complex and overlapping nature of EEG signals. The key aim is to provide a unified, automated solution for detecting multiple neurological disorders such as epilepsy, Parkinson's, Alzheimer's, schizophrenia, and stroke in a single framework. In the Fuzzy Logic and Spiking Neural Networks (FLSNN) framework, EEG data is preprocessed to eliminate noise and artifacts, while a fuzzy logic model is applied to handling uncertainties prior to applying spike neural networking to analyze the temporal and dynamics of the signals. Processes EEG data three times faster than traditional techniques. This framework achieves 97.46% accuracy in binary classification and 98.87% accuracy in multi-class classification, indicating increased efficiency. This research provides a significant advancement in the diagnosis of multiple neurological disorders using EEG and enhances both the quality and speed of diagnostics from the EEG signal and the advancement of AI-based medical diagnostics. at https://github.com/jainshraddha12/FLSNN , the source code will be available to the public.

Adaptive GCN and Bi-GRU-Based Dual Branch for Motor Imagery EEG Decoding

Decoding motor imagery electroencephalography (MI-EEG) signals presents significant challenges due to the difficulty in capturing the complex functional connectivity between channels and the temporal dependencies of EEG signals across different periods. These challenges are exacerbated by the low spatial resolution and high signal redundancy inherent in EEG signals, which traditional linear models struggle to address. To overcome these issues, we propose a novel dual-branch framework that integrates an adaptive graph convolutional network (Adaptive GCN) and bidirectional gated recurrent units (Bi-GRUs) to enhance the decoding performance of MI-EEG signals by effectively modeling both channel correlations and temporal dependencies. The Chebyshev Type II filter decomposes the signal into multiple sub-bands giving the model frequency domain insights. The Adaptive GCN, specifically designed for the MI-EEG context, captures functional connectivity between channels more effectively than conventional GCN models, enabling accurate spatial-spectral feature extraction. Furthermore, combining Bi-GRU and Multi-Head Attention (MHA) captures the temporal dependencies across different time segments to extract deep time-spectral features. Finally, feature fusion is performed to generate the final prediction results. Experimental results demonstrate that our method achieves an average classification accuracy of 80.38% on the BCI-IV Dataset 2a and 87.49% on the BCI-I Dataset 3a, outperforming other state-of-the-art decoding approaches. This approach lays the foundation for future exploration of personalized and adaptive brain-computer interface (BCI) systems.

FLANet: A multiscale temporal convolution and spatial-spectral attention network for EEG artifact removal with adversarial training

Objective.Denoising artifacts, such as noise from muscle or cardiac activity, is a crucial and ubiquitous concern in neurophysiological signal processing, particularly for enhancing the signal-to-noise ratio in electroencephalograph (EEG) analysis. Novel methods based on deep learning demonstrate a notably prominent effect compared to traditional denoising approaches. However, those still suffer from certain limitations. Some methods often neglect the multi-domain characteristics of the artifact signal. Even among those that do consider these, there are deficiencies in terms of efficiency, effectiveness and computation cost.Approach.In this study, we propose a multiscale temporal convolution and spatial-spectral attention network with adversarial training for automatically filtering artifacts, named filter artifacts network (FLANet). The multiscale convolution module can extract sufficient temporal information and the spatial-spectral attention network can extract not only non-local similarity but also spectral dependencies. To make data denoising more efficient and accurate, we adopt adversarial training with novel loss functions to generate outputs that are closer to pure signals.Main results.The results show that the method proposed in this paper achieves great performance in artifact removal and valid information preservation on EEG signals contaminated by different types of artifacts. This approach enables a more optimal trade-off between denoising efficacy and computational overhead.Significance.The proposed artifact removal framework facilitates the implementation of an efficient denoising method, contributing to the advancement of neural analysis and neural engineering, and can be expected to be applied to clinical research and to realize novel human-computer interaction systems.

TFTL: A Task-Free Transfer Learning Strategy for EEG-Based Cross-Subject and Cross-Dataset Motor Imagery BCI

OBJECTIVE

Motor imagery-based brain-computer interfaces (MI-BCIs) have been playing an increasingly vital role in neural rehabilitation. However, the long-term task-based calibration required for enhanced model performance leads to an unfriendly user experience, while the inadequacy of EEG data hinders the performance of deep learning models. To address these challenges, a task-free transfer learning strategy (TFTL) for EEG-based cross-subject & cross-dataset MI-BCI is proposed for calibration time reduction and multi-center data co-modeling.

METHODS

TFTL strategy consists of data alignment, shared feature extractor, and specific classifiers, in which the label predictor for MI tasks classification, as well as domain and dataset discriminator for inter-subject variability reduction are concurrently optimized for knowledge transfer from subjects across different datasets to the target subject. Moreover, only resting data of the target subject is used for subject-specific model construction to achieve task-free.

RESULTS

We employed three deep learning methods (ShallowConvNet, EEGNet, and TCNet-Fusion) as baseline approaches to evaluate the effectiveness of the proposed strategy on five datasets (BCIC IV Dataset 2a, Dataset 1, Physionet MI, Dreyer 2023, and OpenBMI). The results demonstrate a significant improvement with the inclusion of the TFTL strategy compared to the baseline methods, reaching a maximum enhancement of 15.67% with a statistical significance (p = 2.4e-5 < 0.05). Moreover, task-free resulted in MI trials needed for calibration being 0 for all datasets, which significantly alleviated the calibration burden for patients before usage.

CONCLUSION/SIGNIFICANCE

The proposed TFTL strategy effectively addresses challenges posed by prolonged calibration periods and insufficient EEG data, thus promoting MI-BCI from laboratory to clinical application.

Sparse Bayesian correntropy learning for robust muscle activity reconstruction from noisy brain recordings

Sparse Bayesian learning has promoted many effective frameworks of brain activity decoding for the brain-computer interface, including the direct reconstruction of muscle activity using brain recordings. However, existing sparse Bayesian learning algorithms mainly use Gaussian distribution as error assumption in the reconstruction task, which is not necessarily the truth in the real-world application. On the other hand, brain recording is known to be highly noisy and contains many non-Gaussian noises, which could lead to large performance degradation for sparse Bayesian learning algorithms. The goal of this paper is to propose a novel robust implementation of sparse Bayesian learning so that robustness and sparseness can be realized simultaneously. Motivated by the exceptional robustness of maximum correntropy criterion (MCC), we proposed integrating MCC to the sparse Bayesian learning regime. To be specific, we derived the explicit error assumption inherent in the MCC, and then leveraged it for the likelihood function. Meanwhile, we utilized the automatic relevance determination technique as the sparse prior distribution. To fully evaluate the proposed method, a synthetic example and a real-world muscle activity reconstruction task with two different brain modalities were leveraged. Experimental results showed, our proposed sparse Bayesian correntropy learning framework significantly improves the robustness for the noisy regression tasks. Our proposed algorithm could realize higher correlation coefficients and lower root mean squared errors for the real-world muscle activity reconstruction scenario. Sparse Bayesian correntropy learning provides a powerful approach for brain activity decoding which will promote the development of brain-computer interface technology.

STIT-Net- A Wavelet based Convolutional Transformer Model for Motor Imagery EEG Signal Classification in the Sensorimotor Bands

Motor Imagery (MI) electroencephalographic (EEG) signal classification is a pioneer research branch essential for mobility rehabilitation. This paper proposes an end-to-end hybrid deep network "Spatio Temporal Inception Transformer Network (STIT-Net)" model for MI classification. Discrete Wavelet Transform (DWT) is used to derive the alpha (8-13) Hz and beta (13-30) Hz EEG sub bands which are dominant during motor tasks to enhance the performance of the proposed work. STIT-Net employs spatial and temporal convolutions to capture spatial dependencies and temporal information and an inception block with three parallel convolutions extracts multi-level features. Then the transformer encoder with self-attention mechanism highlights the similar task. The proposed model improves the classification of the Physionet EEG motor imagery dataset with an average accuracy of 93.52% and 95.70% for binary class in the alpha and beta bands respectively, and 85.26% and 87.34% for three class, for four class 81.95% and 82.66% were obtained in the alpha and beta band respective EEG based motor signals which is better compared to the results available in the literature. The proposed methodology is further evaluated on other motor imagery datasets, both for subject-independent and cross-subject conditions, to assess the performance of the model.

Enhancing Motor Imagery Classification with Residual Graph Convolutional Networks and Multi-Feature Fusion

Stroke, an abrupt cerebrovascular ailment resulting in brain tissue damage, has prompted the adoption of motor imagery (MI)-based brain-computer interface (BCI) systems in stroke rehabilitation. However, analyzing electroencephalogram (EEG) signals from stroke patients poses challenges. To address the issues of low accuracy and efficiency in EEG classification, particularly involving MI, the study proposes a residual graph convolutional network (M-ResGCN) framework based on the modified S-transform (MST), and introduces the self-attention mechanism into residual graph convolutional network (ResGCN). This study uses MST to extract EEG time-frequency domain features, derives spatial EEG features by calculating the absolute Pearson correlation coefficient (aPcc) between channels, and devises a method to construct the adjacency matrix of the brain network using aPcc to measure the strength of the connection between channels. Experimental results involving 16 stroke patients and 16 healthy subjects demonstrate significant improvements in classification quality and robustness across tests and subjects. The highest classification accuracy reached 94.91% and a Kappa coefficient of 0.8918. The average accuracy and F1 scores from 10 times 10-fold cross-validation are 94.38% and 94.36%, respectively. By validating the feasibility and applicability of brain networks constructed using the aPcc in EEG signal analysis and feature encoding, it was established that the aPcc effectively reflects overall brain activity. The proposed method presents a novel approach to exploring channel relationships in MI-EEG and improving classification performance. It holds promise for real-time applications in MI-based BCI systems.

Multiscale Spatial-Temporal Feature Fusion Neural Network for Motor Imagery Brain-Computer Interfaces

Motor imagery, one of the main brain-computer interface (BCI) paradigms, has been extensively utilized in numerous BCI applications, such as the interaction between disabled people and external devices. Precise decoding, one of the most significant aspects of realizing efficient and stable interaction, has received a great deal of intensive research. However, the current decoding methods based on deep learning are still dominated by single-scale serial convolution, which leads to insufficient extraction of abundant information from motor imagery signals. To overcome such challenges, we propose a new end-to-end convolutional neural network based on multiscale spatial-temporal feature fusion (MSTFNet) for EEG classification of motor imagery. The architecture of MSTFNet consists of four distinct modules: feature enhancement module, multiscale temporal feature extraction module, spatial feature extraction module and feature fusion module, with the latter being further divided into the depthwise separable convolution block and efficient channel attention block. Moreover, we implement a straightforward yet potent data augmentation strategy to bolster the performance of MSTFNet significantly. To validate the performance of MSTFNet, we conduct cross-session experiments and leave-one-subject-out experiments. The cross-session experiment is conducted across two public datasets and one laboratory dataset. On the public datasets of BCI Competition IV 2a and BCI Competition IV 2b, MSTFNet achieves classification accuracies of 83.62% and 89.26%, respectively. On the laboratory dataset, MSTFNet achieves 86.68% classification accuracy. Besides, the leave-one-subject-out experiment is performed on the BCI Competition IV 2a dataset, and MSTFNet achieves 66.31% classification accuracy. These experimental results outperform several state-of-the-art methodologies, indicate the proposed MSTFNet's robust capability in decoding EEG signals associated with motor imagery.

A Novel Recognition and Classification Approach for Motor Imagery Based on Spatio-Temporal Features

Motor imagery, as a paradigm of brain-computer interface, holds vast potential in the field of medical rehabilitation. Addressing the challenges posed by the non-stationarity and low signal-to-noise ratio of EEG signals, the effective extraction of features from motor imagery signals for accurate recognition stands as a key focus in motor imagery brain-computer interface technology. This paper proposes a motor imagery EEG signal classification model that combines functional brain networks with graph convolutional networks. First, functional brain networks are constructed using different brain functional connectivity metrics, and graph theory features are calculated to deeply analyze the characteristics of brain networks under different motor tasks. Then, the constructed functional brain networks are combined with graph convolutional networks for the classification and recognition of motor imagery tasks. The analysis based on brain functional connectivity reveals that the functional connectivity strength during the both fists task is significantly higher than that of other motor imagery tasks, and the functional connectivity strength during actual movement is generally superior to that of motor imagery tasks. In experiments conducted on the Physionet public dataset, the proposed model achieved a classification accuracy of 88.39% under multi-subject conditions, significantly outperforming traditional methods. Under single-subject conditions, the model effectively addressed the issue of individual variability, achieving an average classification accuracy of 99.31%. These results indicate that the proposed model not only exhibits excellent performance in the classification of motor imagery tasks but also provides new insights into the functional connectivity characteristics of different motor tasks and their corresponding brain regions.

Supervised Contrastive Learning-Based Domain Generalization Network for Cross-Subject Motor Decoding

Developing an electroencephalogram (EEG)-based motor imagery and motor execution (MI/ME) decoding system that is both highly accurate and calibration-free for cross-subject applications remains challenging due to domain shift problem inherent in such scenario. Recent research has increasingly embraced transfer learning strategies, especially domain adaptation techniques. However, domain adaptation becomes impractical when the target subject data is either difficult to obtain or unknown. To address this issue, we propose a supervised contrastive learning-based domain generalization network (SCLDGN) for cross-subject MI/ME decoding. Firstly, the feature encoder is purposefully designed to learn the EEG discriminative feature representations. Secondly, the domain alignment based on deep correlation alignment constrains the representations distance across various domains to learn domain-invariant features. In addition, the class regularization block is proposed, where the supervised contrastive learning with domain-agnostic mixup is established to learn the class-relevant features and achieve class-level alignment. Finally, in the latent space, clusters of domain-agnostic representations from the same class are mapped closer together. Consequently, SCLDGN is capable of learning domain-invariant and class-relevant discriminative representations, which are essential for effective cross-subject decoding. Extensive experiments conducted on six MI/ME datasets demonstrate the effectiveness of the proposed method in comparison with other state-of-the-art approaches. Furthermore, ablation study and visualization analyses explain the generalization mechanism of the proposed method and also show neurophysiologically meaningful patterns related to MI/ME.

A Bibliometric Review of Brain-Computer Interfaces in Motor Imagery and Steady-State Visually Evoked Potentials for Applications in Rehabilitation and Robotics

In this paper, a bibliometric review is conducted on brain-computer interfaces (BCI) in non-invasive paradigms like motor imagery (MI) and steady-state visually evoked potentials (SSVEP) for applications in rehabilitation and robotics. An exploratory and descriptive approach is used in the analysis. Computational tools such as the biblioshiny application for R-Bibliometrix and VOSViewer are employed to generate data on years, sources, authors, affiliation, country, documents, co-author, co-citation, and co-occurrence. This article allows for the identification of different bibliometric indicators such as the research process, evolution, visibility, volume, influence, impact, and production in the field of brain-computer interfaces for MI and SSVEP paradigms in rehabilitation and robotics applications from 2000 to August 2024.

A multi-feature fusion graph attention network for decoding motor imagery intention in spinal cord injury patients

Objective.Electroencephalogram (EEG) signals exhibit temporal-frequency-spatial multi-domain feature, and due to the nonplanar nature of the brain surface, the electrode distributions follow non-Euclidean topology. To fully resolve the EEG signals, this study proposes a temporal-frequency-spatial multi-domain feature fusion graph attention network (GAT) for motor imagery (MI) intention recognition in spinal cord injury (SCI) patients.Approach.The proposed model uses phase-locked value (PLV) to extract spatial phase connectivity information between EEG channels and continuous wavelet transform to extract valid EEG information in the time-frequency domain. It then models as a graph data structure containing multi-domain information. The gated recurrent unit and GAT learn EEG's dynamic temporal-spatial information. Finally, the fully connected layer outputs the MI intention recognition results.Main results.After 10 times 10-fold cross-validation, the proposed model can achieve an average accuracy of 95.82%. Furthermore, this study analyses the event-related desynchronization/event-related synchronization and PLV brain network to explore the brain activity of SCI patients during MI.Significance.This study confirms the potential of the proposed model in terms of EEG decoding performance and provides a reference for the mechanism of neural activity in SCI patients.

Motion Cognitive Decoding of Cross-Subject Motor Imagery Guided on Different Visual Stimulus Materials

BACKGROUND

Motor imagery (MI) plays an important role in brain-computer interfaces, especially in evoking event-related desynchronization and synchronization (ERD/S) rhythms in electroencephalogram (EEG) signals. However, the procedure for performing a MI task for a single subject is subjective, making it difficult to determine the actual situation of an individual's MI task and resulting in significant individual EEG response variations during motion cognitive decoding.

METHODS

To explore this issue, we designed three visual stimuli (arrow, human, and robot), each of which was used to present three MI tasks (left arm, right arm, and feet), and evaluated differences in brain response in terms of ERD/S rhythms. To compare subject-specific variations of different visual stimuli, a novel cross-subject MI-EEG classification method was proposed for the three visual stimuli. The proposed method employed a covariance matrix centroid alignment for preprocessing of EEG samples, followed by a model agnostic meta-learning method for cross-subject MI-EEG classification.

RESULTS AND CONCLUSION

The experimental results showed that robot stimulus materials were better than arrow or human stimulus materials, with an optimal cross-subject motion cognitive decoding accuracy of 79.04%. Moreover, the proposed method produced robust classification of cross-subject MI-EEG signal decoding, showing superior results to conventional methods on collected EEG signals.

Refining ADHD diagnosis with EEG: The impact of preprocessing and temporal segmentation on classification accuracy

BACKGROUND

EEG signals are commonly used in ADHD diagnosis, but they are often affected by noise and artifacts. Effective preprocessing and segmentation methods can significantly enhance the accuracy and reliability of ADHD classification.

METHODS

We applied filtering, ASR, and ICA preprocessing techniques to EEG data from children with ADHD and neurotypical controls. The EEG recordings were segmented, and features were extracted and selected based on statistical significance. Classification was performed using various EEG segments and channels with Machine Learning models (SVM, KNN, and XGBoost) to identify the most effective combinations for accurate ADHD diagnosis.

RESULTS

Our findings show that models trained on later EEG segments achieved significantly higher accuracy, indicating the potential role of cognitive fatigue in distinguishing ADHD. The highest classification accuracy (86.1%) was achieved using data from the P3, P4, and C3 channels, with key features such as Kurtosis, Katz fractal dimension, and power spectrums in the Delta, Theta, and Alpha bands contributing to the results.

CONCLUSION

This study highlights the importance of preprocessing and segmentation in improving the reliability of ADHD diagnosis through EEG. The results suggest that further research on cognitive fatigue and segmentation could enhance diagnostic accuracy in ADHD patients.

MACNet: A Multidimensional Attention-Based Convolutional Neural Network for Lower-Limb Motor Imagery Classification

Decoding lower-limb motor imagery (MI) is highly important in brain-computer interfaces (BCIs) and rehabilitation engineering. However, it is challenging to classify lower-limb MI from electroencephalogram (EEG) signals, because lower-limb motions (LLMs) including MI are excessively close to physiological representations in the human brain and generate low-quality EEG signals. To address this challenge, this paper proposes a multidimensional attention-based convolutional neural network (CNN), termed MACNet, which is specifically designed for lower-limb MI classification. MACNet integrates a temporal refining module and an attention-enhanced convolutional module by leveraging the local and global feature representation abilities of CNNs and attention mechanisms. The temporal refining module adaptively investigates critical information from each electrode channel to refine EEG signals along the temporal dimension. The attention-enhanced convolutional module extracts temporal and spatial features while refining the feature maps across the channel and spatial dimensions. Owing to the scarcity of public datasets available for lower-limb MI, a specified lower-limb MI dataset involving four routine LLMs is built, consisting of 10 subjects over 20 sessions. Comparison experiments and ablation studies are conducted on this dataset and a public BCI Competition IV 2a EEG dataset. The experimental results show that MACNet achieves state-of-the-art performance and outperforms alternative models for the subject-specific mode. Visualization analysis reveals the excellent feature learning capabilities of MACNet and the potential relationship between lower-limb MI and brain activity. The effectiveness and generalizability of MACNet are verified.

Decoding Imagined Speech from EEG Data: A Hybrid Deep Learning Approach to Capturing Spatial and Temporal Features

Neuroimaging is revolutionizing our ability to investigate the brain's structural and functional properties, enabling us to visualize brain activity during diverse mental processes and actions. One of the most widely used neuroimaging techniques is electroencephalography (EEG), which records electrical activity from the brain using electrodes positioned on the scalp. EEG signals capture both spatial (brain region) and temporal (time-based) data. While a high temporal resolution is achievable with EEG, spatial resolution is comparatively limited. Consequently, capturing both spatial and temporal information from EEG data to recognize mental activities remains challenging. In this paper, we represent spatial and temporal information obtained from EEG signals by transforming EEG data into sequential topographic brain maps. We then apply hybrid deep learning models to capture the spatiotemporal features of the EEG topographic images and classify imagined English words. The hybrid framework utilizes a sequential combination of three-dimensional convolutional neural networks (3DCNNs) and recurrent neural networks (RNNs). The experimental results reveal the effectiveness of the proposed approach, achieving an average accuracy of 77.8% in identifying imagined English speech.

Motor task-to-task transfer learning for motor imagery brain-computer interfaces

Motor imagery (MI) is one of the popular control paradigms in the non-invasive brain-computer interface (BCI) field. MI-BCI generally requires users to conduct the imagination of movement (e.g., left or right hand) to collect training data for generating a classification model during the calibration phase. However, this calibration phase is generally time-consuming and tedious, as users conduct the imagination of hand movement several times without being given feedback for an extended period. This obstacle makes MI-BCI non user-friendly and hinders its use. On the other hand, motor execution (ME) and motor observation (MO) are relatively easier tasks, yield lower fatigue than MI, and share similar neural mechanisms to MI. However, few studies have integrated these three tasks into BCIs. In this study, we propose a new task-to-task transfer learning approach of 3-motor tasks (ME, MO, and MI) for building a better user-friendly MI-BCI. For this study, 28 subjects participated in 3-motor tasks experiment, and electroencephalography (EEG) was acquired. User opinions regarding the 3-motor tasks were also collected through questionnaire survey. The 3-motor tasks showed a power decrease in the alpha rhythm, known as event-related desynchronization, but with slight differences in the temporal patterns. In the classification analysis, the cross-validated accuracy (within-task) was 67.05 % for ME, 65.93 % for MI, and 73.16 % for MO on average. Consistently with the results, the subjects scored MI (3.16) as the most difficult task compared with MO (1.42) and ME (1.41), with p < 0.05. In the analysis of task-to-task transfer learning, where training and testing are performed using different task datasets, the ME-trained model yielded an accuracy of 65.93 % (MI test), which is statistically similar to the within-task accuracy (p > 0.05). The MO-trained model achieved an accuracy of 60.82 % (MI test). On the other hand, combining two datasets yielded interesting results. ME and 50 % of the MI-trained model (50-shot) classified MI with a 69.21 % accuracy, which outperformed the within-task accuracy (p < 0.05), and MO and 50 % of the MI-trained model showed an accuracy of 66.75 %. Of the low performers with a within-task accuracy of 70 % or less, 90 % (n = 21) of the subjects improved in training with ME, and 76.2 % (n = 16) improved in training with MO on the MI test at 50-shot. These results demonstrate that task-to-task transfer learning is possible and could be a promising approach to building a user-friendly training protocol in MI-BCI.

IMH-Net: a convolutional neural network for end-to-end EEG motor imagery classification

As the main component of Brain-computer interface (BCI) technology, the classification algorithm based on EEG has developed rapidly. The previous algorithms were often based on subject-dependent settings, resulting in BCI needing to be calibrated for new users. In this work, we propose IMH-Net, an end-to-end subject-independent model. The model first uses Inception blocks extracts the frequency domain features of the data, then further compresses the feature vectors to extract the spatial domain features, and finally learns the global information and classification through Multi-Head Attention mechanism. On the OpenBMI dataset, IMH-Net obtained 73.90 ± 13.10% accuracy and 73.09 ± 14.99% F1-score in subject-independent manner, which improved the accuracy by 1.96% compared with the comparison model. On the BCI competition IV dataset 2a, this model also achieved the highest accuracy and F1-score in subject-dependent manner. The IMH-Net model we proposed can improve the accuracy of subject-independent Motor Imagery (MI), and the robustness of the algorithm is high, which has strong practical value in the field of BCI.

Leveraging temporal dependency for cross-subject-MI BCIs by contrastive learning and self-attention

Brain-computer interfaces (BCIs) built based on motor imagery paradigm have found extensive utilization in motor rehabilitation and the control of assistive applications. However, traditional MI-BCI systems often exhibit suboptimal classification performance and require significant time for new users to collect subject-specific training data. This limitation diminishes the user-friendliness of BCIs and presents significant challenges in developing effective subject-independent models. In response to these challenges, we propose a novel subject-independent framework for learning temporal dependency for motor imagery BCIs by Contrastive Learning and Self-attention (CLS). In CLS model, we incorporate self-attention mechanism and supervised contrastive learning into a deep neural network to extract important information from electroencephalography (EEG) signals as features. We evaluate the CLS model using two large public datasets encompassing numerous subjects in a subject-independent experiment condition. The results demonstrate that CLS outperforms six baseline algorithms, achieving a mean classification accuracy improvement of 1.3 % and 4.71 % than the best algorithm on the Giga dataset and OpenBMI dataset, respectively. Our findings demonstrate that CLS can effectively learn invariant discriminative features from training data obtained from non-target subjects, thus showcasing its potential for building models for new users without the need for calibration.

Attention-based cross-frequency graph convolutional network for driver fatigue estimation

Fatigue driving significantly contributes to global vehicle accidents and fatalities, making driver fatigue level estimation crucial. Electroencephalography (EEG) is a proven reliable predictor of brain states. With Deep Learning (DL) advancements, brain state estimation algorithms have improved significantly. Nonetheless, EEG's multi-domain nature and the intricate spatial-temporal-frequency correlations among EEG channels present challenges in developing precise DL models. In this work, we introduce an innovative Attention-based Cross-Frequency Graph Convolutional Network (ACF-GCN) for estimating drivers' reaction times using EEG signals from theta, alpha, and beta bands. This method utilizes a multi-head attention mechanism to detect long-range dependencies between EEG channels across frequencies. Concurrently, the transformer's encoder module learns node-level feature maps from the attention-score matrix. Subsequently, the Graph Convolutional Network (GCN) integrates this matrix with feature maps to estimate driver reaction time. Our validation on a publicly available dataset shows that ACF-GCN outperforms several state-of-the-art methods. We also explore the brain dynamics within the cross-frequency attention-score matrix, identifying theta and alpha bands as key influencers in fatigue estimating performance. The ACF-GCN method advances brain state estimation and provides insights into the brain dynamics underlying multi-channel EEG signals.

A combination of deep learning models and type-2 fuzzy for EEG motor imagery classification through spatiotemporal-frequency features

Developing a robust and effective technique is crucial for interpreting a user's brainwave signals accurately in the realm of biomedical signal processing. The variability and uncertainty present in EEG patterns over time, compounded by noise, pose notable challenges, particularly in mental tasks like motor imagery. Introducing fuzzy components can enhance the system's ability to withstand noisy environments. The emergence of deep learning has significantly impacted artificial intelligence and data analysis, prompting extensive exploration into assessing and understanding brain signals. This work introduces a hybrid series architecture called FCLNET, which combines Compact-CNN to extract frequency and spatial features alongside the LSTM network for temporal feature extraction. The activation functions in the CNN architecture were implemented using type-2 fuzzy functions to tackle uncertainties. Hyperparameters of the FCLNET model are tuned by the Bayesian optimisation algorithm. The efficacy of this approach is assessed through the BCI Competition IV-2a database and the BCI Competition IV-1 database. By incorporating type-2 fuzzy activation functions and employing Bayesian optimisation for tuning, the proposed architecture indicates good classification accuracy compared to the literature. Outcomes showcase the exceptional achievements of the FCLNET model, suggesting that integrating fuzzy units into other classifiers could lead to advancements in motor imagery-based BCI systems.

An effective classification approach for EEG-based motor imagery tasks combined with attention mechanisms

Currently, electroencephalogram (EEG)-based motor imagery (MI) signals have been received extensive attention, which can assist disabled subjects to control wheelchair, automatic driving and other activities. However, EEG signals are easily affected by some factors, such as muscle movements, wireless devices, power line, etc., resulting in the low signal-to-noise ratios and the worse recognition results on EEG decoding. Therefore, it is crucial to develop a stable model for decoding MI-EEG signals. To address this issue and further improve the decoding performance for MI tasks, a hybrid structure combining convolutional neural networks and bidirectional long short-term memory (BLSTM) model, namely CBLSTM, is developed in this study to handle the various EEG-based MI tasks. Besides, the attention mechanism (AM) model is further adopted to adaptively assign the weight of EEG vital features and enhance the expression which beneficial to classification for MI tasks. First of all, the spatial features and the time series features are extracted by CBLSTM from preprocessed MI-EEG data, respectively. Meanwhile, more effective features information can be mined by the AM model, and the softmax function is utilized to recognize intention categories. Ultimately, the numerical results illustrate that the model presented achieves an average accuracy of 98.40% on the public physioNet dataset and faster training process for decoding MI tasks, which is superior to some other advanced models. Ablation experiment performed also verifies the effectiveness and feasibility of the developed model. Moreover, the established network model provides a good basis for the application of brain-computer interface in rehabilitation medicine.

A Comprehensive Review of Hardware Acceleration Techniques and Convolutional Neural Networks for EEG Signals

This paper comprehensively reviews hardware acceleration techniques and the deployment of convolutional neural networks (CNNs) for analyzing electroencephalogram (EEG) signals across various application areas, including emotion classification, motor imagery, epilepsy detection, and sleep monitoring. Previous reviews on EEG have mainly focused on software solutions. However, these reviews often overlook key challenges associated with hardware implementation, such as scenarios that require a small size, low power, high security, and high accuracy. This paper discusses the challenges and opportunities of hardware acceleration for wearable EEG devices by focusing on these aspects. Specifically, this review classifies EEG signal features into five groups and discusses hardware implementation solutions for each category in detail, providing insights into the most suitable hardware acceleration strategies for various application scenarios. In addition, it explores the complexity of efficient CNN architectures for EEG signals, including techniques such as pruning, quantization, tensor decomposition, knowledge distillation, and neural architecture search. To the best of our knowledge, this is the first systematic review that combines CNN hardware solutions with EEG signal processing. By providing a comprehensive analysis of current challenges and a roadmap for future research, this paper provides a new perspective on the ongoing development of hardware-accelerated EEG systems.

EEG-based motor imagery channel selection and classification using hybrid optimization and two-tier deep learning

Brain-computer interface (BCI) technology holds promise for individuals with profound motor impairments, offering the potential for communication and control. Motor imagery (MI)-based BCI systems are particularly relevant in this context. Despite their potential, achieving accurate and robust classification of MI tasks using electroencephalography (EEG) data remains a significant challenge. In this paper, we employed the Minimum Redundancy Maximum Relevance (MRMR) algorithm to optimize channel selection. Furthermore, we introduced a hybrid optimization approach that combines the War Strategy Optimization (WSO) and Chimp Optimization Algorithm (ChOA). This hybridization significantly enhances the classification model's overall performance and adaptability. A two-tier deep learning architecture is proposed for classification, consisting of a Convolutional Neural Network (CNN) and a modified Deep Neural Network (M-DNN). The CNN focuses on capturing temporal correlations within EEG data, while the M-DNN is designed to extract high-level spatial characteristics from selected EEG channels. Integrating optimal channel selection, hybrid optimization, and the two-tier deep learning methodology in our BCI framework presents an enhanced approach for precise and effective BCI control. Our model got 95.06% accuracy with high precision. This advancement has the potential to significantly impact neurorehabilitation and assistive technology applications, facilitating improved communication and control for individuals with motor impairments.

CTNet: a convolutional transformer network for EEG-based motor imagery classification

Brain-computer interface (BCI) technology bridges the direct communication between the brain and machines, unlocking new possibilities for human interaction and rehabilitation. EEG-based motor imagery (MI) plays a pivotal role in BCI, enabling the translation of thought into actionable commands for interactive and assistive technologies. However, the constrained decoding performance of brain signals poses a limitation to the broader application and development of BCI systems. In this study, we introduce a convolutional Transformer network (CTNet) designed for EEG-based MI classification. Firstly, CTNet employs a convolutional module analogous to EEGNet, dedicated to extracting local and spatial features from EEG time series. Subsequently, it incorporates a Transformer encoder module, leveraging a multi-head attention mechanism to discern the global dependencies of EEG's high-level features. Finally, a straightforward classifier module comprising fully connected layers is followed to categorize EEG signals. In subject-specific evaluations, CTNet achieved remarkable decoding accuracies of 82.52% and 88.49% on the BCI IV-2a and IV-2b datasets, respectively. Furthermore, in the challenging cross-subject assessments, CTNet achieved recognition accuracies of 58.64% on the BCI IV-2a dataset and 76.27% on the BCI IV-2b dataset. In both subject-specific and cross-subject evaluations, CTNet holds a leading position when compared to some of the state-of-the-art methods. This underscores the exceptional efficacy of our approach and its potential to set a new benchmark in EEG decoding.

Evaluation of perceived urgency from single-trial EEG data elicited by upper-body vibration feedback using deep learning

Notification systems that convey urgency without adding cognitive burden are crucial in human-computer interaction. Haptic feedback systems, particularly those utilizing vibration feedback, have emerged as a compelling solution, capable of providing desirable levels of urgency depending on the application. High-risk applications require an evaluation of the urgency level elicited during critical notifications. Traditional evaluations of perceived urgency rely on subjective self-reporting and performance metrics, which, while useful, are not real-time and can be distracting from the task at hand. In contrast, EEG technology offers a direct, non-intrusive method of assessing the user's cognitive state. Leveraging deep learning, this study introduces a novel approach to evaluate perceived urgency from single-trial EEG data, induced by vibration stimuli on the upper body, utilizing our newly collected urgency-via-vibration dataset. The proposed model combines a 2D convolutional neural network with a temporal convolutional network to capture spatial and temporal EEG features, outperforming several established EEG models. The proposed model achieves an average classification accuracy of 83% through leave-one-subject-out cross-validation across three urgency classes (not urgent, urgent, and very urgent) from a single trial of EEG data. Furthermore, explainability analysis showed that the prefrontal brain region, followed by the central brain region, are the most influential in predicting the urgency level. A follow-up neural statistical analysis revealed an increase in event-related synchronization (ERS) in the theta frequency band (4-7 Hz) with the increased level of urgency, which is associated with high arousal and attention in the neuroscience literature. A limitation of this study is that the proposed model's performance was tested only the urgency-via-vibration dataset, which may affect the generalizability of the findings.

Riemannian Locality Preserving Method for Transfer Learning With Applications on Brain-Computer Interface

Brain-computer interfaces (BCIs) have been widely focused and extensively studied in recent years for their huge prospect of medical rehabilitation and commercial applications. Transfer learning exploits the information in the source domain and applies in another different but related domain (target domain), and is therefore introduced into the BCIs to figure out the inter-subject variances of electroencephalography (EEG) signals. In this article, a novel transfer learning method is proposed to preserve the Riemannian locality of data structure in both the source and target domains and simultaneously realize the joint distribution adaptation of both domains to enhance the effectiveness of transfer learning. Specifically, a Riemannian graph is first defined and constructed based on the Riemannian distance to represent the Riemannian geometry information. To simultaneously align the marginal and conditional distribution of source and target domains and preserve the Riemannian locality of data structure in both domains, the Riemannian graph is embedded in the joint distribution adaptation (JDA) framework and forms the proposed Riemannian locality preserving-based transfer learning (RLPTL). To validate the effect of the proposed method, it is compared with several existing methods on two open motor imagery datasets, and both multi-source domains (MSD) and single-source domains (SSD) experiments are considered. Experimental results show that the proposed method achieves the highest accuracies in MSD and SSD experiments on three datasets and outperforms eight baseline methods, which demonstrates that the proposed method creates a feasible and efficient way to realize transfer learning.

Suppression of negative transfer in motor imagery brain-computer interface based on mutual information and Pearson correlation coefficient

The focus of this paper is on the main challenges in brain-computer interface transfer learning: how to address data characteristic length and the source domain sample selection problems caused by individual differences. To overcome the negative migration that results from feature length, we propose a migration algorithm based on mutual information transfer (MIT), which selects effective features by calculating the entropy value of the probability distribution and conditional distribution, thereby reducing negative migration and improving learning efficiency. Source domain participants who differ too much from the target domain distribution can affect the overall classification performance. On the basis of MIT, we propose the Pearson correlation coefficient source domain automatic selection algorithm (PDAS algorithm). The PDAS algorithm can automatically select the appropriate source domain participants according to the target domain distribution, which reduces the negative migration of participant data among the source domain participants, improves experimental accuracy, and greatly reduces training time. The two proposed algorithms were tested offline and online on two public datasets, and the results were compared with those from existing advanced algorithms. The experimental results showed that the MIT algorithm and the MIT + PDAS algorithm had obvious advantages.

A review on evaluating mental stress by deep learning using EEG signals

Mental stress is a common problem that affects individuals all over the world. Stress reduces human functionality during routine work and may lead to severe health defects. Early detection of stress is important for preventing diseases and other negative health-related consequences of stress. Several neuroimaging techniques have been utilized to assess mental stress, however, due to its ease of use, robustness, and non-invasiveness, electroencephalography (EEG) is commonly used. This paper aims to fill a knowledge gap by reviewing the different EEG-related deep learning algorithms with a focus on Convolutional Neural Networks (CNNs) and Long Short-Term Memory networks (LSTMs) for the evaluation of mental stress. The review focuses on data representation, individual deep neural network model architectures, hybrid models, and results amongst others. The contributions of the paper address important issues such as data representation and model architectures. Out of all reviewed papers, 67% used CNN, 9% LSTM, and 24% hybrid models. Based on the reviewed literature, we found that dataset size and different representations contributed to the performance of the proposed networks. Raw EEG data produced classification accuracy around 62% while using spectral and topographical representation produced up to 88%. Nevertheless, the roles of generalizability across different deep learning models and individual differences remain key areas of inquiry. The review encourages the exploration of innovative avenues, such as EEG data image representations concurrently with graph convolutional neural networks (GCN), to mitigate the impact of inter-subject variability. This novel approach not only allows us to harmonize structural nuances within the data but also facilitates the integration of temporal dynamics, thereby enabling a more comprehensive assessment of mental stress levels.

MSE-VGG: A Novel Deep Learning Approach Based on EEG for Rapid Ischemic Stroke Detection

Ischemic stroke is a type of brain dysfunction caused by pathological changes in the blood vessels of the brain which leads to brain tissue ischemia and hypoxia and ultimately results in cell necrosis. Without timely and effective treatment in the early time window, ischemic stroke can lead to long-term disability and even death. Therefore, rapid detection is crucial in patients with ischemic stroke. In this study, we developed a deep learning model based on fusion features extracted from electroencephalography (EEG) signals for the fast detection of ischemic stroke. Specifically, we recruited 20 ischemic stroke patients who underwent EEG examination during the acute phase of stroke and collected EEG signals from 19 adults with no history of stroke as a control group. Afterwards, we constructed correlation-weighted Phase Lag Index (cwPLI), a novel feature, to explore the synchronization information and functional connectivity between EEG channels. Moreover, the spatio-temporal information from functional connectivity and the nonlinear information from complexity were fused by combining the cwPLI matrix and Sample Entropy (SaEn) together to further improve the discriminative ability of the model. Finally, the novel MSE-VGG network was employed as a classifier to distinguish ischemic stroke from non-ischemic stroke data. Five-fold cross-validation experiments demonstrated that the proposed model possesses excellent performance, with accuracy, sensitivity, and specificity reaching 90.17%, 89.86%, and 90.44%, respectively. Experiments on time consumption verified that the proposed method is superior to other state-of-the-art examinations. This study contributes to the advancement of the rapid detection of ischemic stroke, shedding light on the untapped potential of EEG and demonstrating the efficacy of deep learning in ischemic stroke identification.

A deep neural network and transfer learning combined method for cross-task classification of error-related potentials

Background

Error-related potentials (ErrPs) are electrophysiological responses that naturally occur when humans perceive wrongdoing or encounter unexpected events. It offers a distinctive means of comprehending the error-processing mechanisms within the brain. A method for detecting ErrPs with high accuracy holds significant importance for various ErrPs-based applications, such as human-in-the-loop Brain-Computer Interface (BCI) systems. Nevertheless, current methods fail to fulfill the generalization requirements for detecting such ErrPs due to the high non-stationarity of EEG signals across different tasks and the limited availability of ErrPs datasets.

Methods

This study introduces a deep learning-based model that integrates convolutional layers and transformer encoders for the classification of ErrPs. Subsequently, a model training strategy, grounded in transfer learning, is proposed for the effective training of the model. The datasets utilized in this study are available for download from the publicly accessible databases.

Results

In cross-task classification, an average accuracy of about 78% was achieved, exceeding the baseline. Furthermore, in the leave-one-subject-out, within-session, and cross-session classification scenarios, the proposed model outperformed the existing techniques with an average accuracy of 71.81, 78.74, and 77.01%, respectively.

Conclusions

Our approach contributes to mitigating the challenge posed by limited datasets in the ErrPs field, achieving this by reducing the requirement for extensive training data for specific target tasks. This may serve as inspiration for future studies that concentrate on ErrPs and their applications.

Comprehensive evaluation methods for translating BCI into practical applications: usability, user satisfaction and usage of online BCI systems

Although brain-computer interface (BCI) is considered a revolutionary advancement in human-computer interaction and has achieved significant progress, a considerable gap remains between the current technological capabilities and their practical applications. To promote the translation of BCI into practical applications, the gold standard for online evaluation for classification algorithms of BCI has been proposed in some studies. However, few studies have proposed a more comprehensive evaluation method for the entire online BCI system, and it has not yet received sufficient attention from the BCI research and development community. Therefore, the qualitative leap from analyzing and modeling for offline BCI data to the construction of online BCI systems and optimizing their performance is elaborated, and then user-centred is emphasized, and then the comprehensive evaluation methods for translating BCI into practical applications are detailed and reviewed in the article, including the evaluation of the usability (including effectiveness and efficiency of systems), the evaluation of the user satisfaction (including BCI-related aspects, etc.), and the evaluation of the usage (including the match between the system and user, etc.) of online BCI systems. Finally, the challenges faced in the evaluation of the usability and user satisfaction of online BCI systems, the efficacy of online BCI systems, and the integration of BCI and artificial intelligence (AI) and/or virtual reality (VR) and other technologies to enhance the intelligence and user experience of the system are discussed. It is expected that the evaluation methods for online BCI systems elaborated in this review will promote the translation of BCI into practical applications.

EEG-based emotion recognition systems; comprehensive study

Emotion recognition technology through EEG signal analysis is currently a fundamental concept in artificial intelligence. This recognition has major practical implications in emotional health care, human-computer interaction, and so on. This paper provides a comprehensive study of different methods for extracting electroencephalography (EEG) features for emotion recognition from four different perspectives, including time domain features, frequency domain features, time-frequency features, and nonlinear features. We summarize the current pattern recognition methods adopted in most related works, and with the rapid development of deep learning (DL) attracting the attention of researchers in this field, we pay more attention to deep learning-based studies and analyse the characteristics, advantages, disadvantages, and applicable scenarios. Finally, the current challenges and future development directions in this field were summarized. This paper can help novice researchers in this field gain a systematic understanding of the current status of emotion recognition research based on EEG signals and provide ideas for subsequent related research.

Optimal Channel Selection of Multiclass Motor Imagery Classification Based on Fusion Convolutional Neural Network with Attention Blocks

The widely adopted paradigm in brain-computer interfaces (BCIs) involves motor imagery (MI), enabling improved communication between humans and machines. EEG signals derived from MI present several challenges due to their inherent characteristics, which lead to a complex process of classifying and finding the potential tasks of a specific participant. Another issue is that BCI systems can result in noisy data and redundant channels, which in turn can lead to increased equipment and computational costs. To address these problems, the optimal channel selection of a multiclass MI classification based on a Fusion convolutional neural network with Attention blocks (FCNNA) is proposed. In this study, we developed a CNN model consisting of layers of convolutional blocks with multiple spatial and temporal filters. These filters are designed specifically to capture the distribution and relationships of signal features across different electrode locations, as well as to analyze the evolution of these features over time. Following these layers, a Convolutional Block Attention Module (CBAM) is used to, further, enhance EEG signal feature extraction. In the process of channel selection, the genetic algorithm is used to select the optimal set of channels using a new technique to deliver fixed as well as variable channels for all participants. The proposed methodology is validated showing 6.41% improvement in multiclass classification compared to most baseline models. Notably, we achieved the highest results of 93.09% for binary classes involving left-hand and right-hand movements. In addition, the cross-subject strategy for multiclass classification yielded an impressive accuracy of 68.87%. Following channel selection, multiclass classification accuracy was enhanced, reaching 84.53%. Overall, our experiments illustrated the efficiency of the proposed EEG MI model in both channel selection and classification, showing superior results with either a full channel set or a reduced number of channels.

A Data Augmentation Method for Motor Imagery EEG Signals Based on DCGAN-GP Network

Motor imagery electroencephalography (EEG) signals have garnered attention in brain-computer interface (BCI) research due to their potential in promoting motor rehabilitation and control. However, the limited availability of labeled data poses challenges for training robust classifiers. In this study, we propose a novel data augmentation method utilizing an improved Deep Convolutional Generative Adversarial Network with Gradient Penalty (DCGAN-GP) to address this issue. We transformed raw EEG signals into two-dimensional time-frequency maps and employed a DCGAN-GP network to generate synthetic time-frequency representations resembling real data. Validation experiments were conducted on the BCI IV 2b dataset, comparing the performance of classifiers trained with augmented and unaugmented data. Results demonstrated that classifiers trained with synthetic data exhibit enhanced robustness across multiple subjects and achieve higher classification accuracy. Our findings highlight the effectiveness of utilizing a DCGAN-GP-generated synthetic EEG data to improve classifier performance in distinguishing different motor imagery tasks. Thus, the proposed data augmentation method based on a DCGAN-GP offers a promising avenue for enhancing BCI system performance, overcoming data scarcity challenges, and bolstering classifier robustness, thereby providing substantial support for the broader adoption of BCI technology in real-world applications.

EISATC-Fusion: Inception Self-Attention Temporal Convolutional Network Fusion for Motor Imagery EEG Decoding

The motor imagery brain-computer interface (MI-BCI) based on electroencephalography (EEG) is a widely used human-machine interface paradigm. However, due to the non-stationarity and individual differences among subjects in EEG signals, the decoding accuracy is limited, affecting the application of the MI-BCI. In this paper, we propose the EISATC-Fusion model for MI EEG decoding, consisting of inception block, multi-head self-attention (MSA), temporal convolutional network (TCN), and layer fusion. Specifically, we design a DS Inception block to extract multi-scale frequency band information. And design a new cnnCosMSA module based on CNN and cos attention to solve the attention collapse and improve the interpretability of the model. The TCN module is improved by the depthwise separable convolution to reduces the parameters of the model. The layer fusion consists of feature fusion and decision fusion, fully utilizing the features output by the model and enhances the robustness of the model. We improve the two-stage training strategy for model training. Early stopping is used to prevent model overfitting, and the accuracy and loss of the validation set are used as indicators for early stopping. The proposed model achieves within-subject classification accuracies of 84.57% and 87.58% on BCI Competition IV Datasets 2a and 2b, respectively. And the model achieves cross-subject classification accuracies of 67.42% and 71.23% (by transfer learning) when training the model with two sessions and one session of Dataset 2a, respectively. The interpretability of the model is demonstrated through weight visualization method.

Prefrontal Cortex Activity During Gait in People With Persistent Symptoms After Concussion

BACKGROUND

Concussions result in transient symptoms stemming from a cortical metabolic energy crisis. Though this metabolic energy crisis typically resolves in a month, symptoms can persist for years. The symptomatic period is associated with gait dysfunction, the cortical underpinnings of which are poorly understood. Quantifying prefrontal cortex (PFC) activity during gait may provide insight into post-concussion gait dysfunction. The purpose of this study was to explore the effects of persisting concussion symptoms on PFC activity during gait. We hypothesized that adults with persisting concussion symptoms would have greater PFC activity during gait than controls. Within the concussed group, we hypothesized that worse symptoms would relate to increased PFC activity during gait, and that increased PFC activity would relate to worse gait characteristics.

METHODS

The Neurobehavior Symptom Inventory (NSI) characterized concussion symptoms. Functional near-infrared spectroscopy quantified PFC activity (relative concentration changes of oxygenated hemoglobin [HbO2]) in 14 people with a concussion and 25 controls. Gait was assessed using six inertial sensors in the concussion group.

RESULTS

Average NSI total score was 26.4 (13.2). HbO2 was significantly higher (P = .007) for the concussed group (0.058 [0.108]) compared to the control group (-0.016 [0.057]). Within the concussion group, HbO2 correlated with NSI total symptom score (ρ = .62; P = .02), sagittal range of motion (r = .79; P = .001), and stride time variability (r = -.54; P = .046).

CONCLUSION

These data suggest PFC activity relates to symptom severity and some gait characteristics in people with persistent concussion symptoms. Identifying the neurophysiological underpinnings to gait deficits post-concussion expands our knowledge of motor behavior deficits in people with persistent concussion symptoms.

Multiclass classification of motor imagery tasks based on multi-branch convolutional neural network and temporal convolutional network model

Motor imagery (MI) is a cognitive process wherein an individual mentally rehearses a specific movement without physically executing it. Recently, MI-based brain-computer interface (BCI) has attracted widespread attention. However, accurate decoding of MI and understanding of neural mechanisms still face huge challenges. These seriously hinder the clinical application and development of BCI systems based on MI. Thus, it is very necessary to develop new methods to decode MI tasks. In this work, we propose a multi-branch convolutional neural network (MBCNN) with a temporal convolutional network (TCN), an end-to-end deep learning framework to decode multi-class MI tasks. We first used MBCNN to capture the MI electroencephalography signals information on temporal and spectral domains through different convolutional kernels. Then, we introduce TCN to extract more discriminative features. The within-subject cross-session strategy is used to validate the classification performance on the dataset of BCI Competition IV-2a. The results showed that we achieved 75.08% average accuracy for 4-class MI task classification, outperforming several state-of-the-art approaches. The proposed MBCNN-TCN-Net framework successfully captures discriminative features and decodes MI tasks effectively, improving the performance of MI-BCIs. Our findings could provide significant potential for improving the clinical application and development of MI-based BCI systems.