Recognition of single upper limb motor imagery tasks from EEG using multi-branch fusion convolutional neural network

A Synergy of Convolutional Neural Networks for Sensor-Based EEG Brain-Computer Interfaces to Enhance Motor Imagery Classification

Enhancing motor disability assessment and its imagery classification is a significant concern in contemporary medical practice, necessitating reliable solutions to improve patient outcomes. One promising avenue is the use of brain-computer interfaces (BCIs), which establish a direct communication pathway between users and machines. This technology holds the potential to revolutionize human-machine interaction, especially for individuals diagnosed with motor disabilities. Despite this promise, extracting reliable control signals from noisy brain data remains a critical challenge. In this paper, we introduce a novel approach leveraging the collaborative synergy of five convolutional neural network (CNN) models to improve the classification accuracy of motor imagery tasks, which are essential components of BCI systems. Our method demonstrates exceptional performance, achieving an accuracy of 79.44% on the BCI Competition IV 2a dataset, surpassing existing state-of-the-art techniques in using multiple CNN models. This advancement offers significant promise for enhancing the efficacy and versatility of BCIs in a wide range of real-world applications, from assistive technologies to neurorehabilitation, thereby providing robust solutions for individuals with motor disabilities.

Identification and analysis of reference-independent movement event-related desynchronization

Characterization of the electroencephalography (EEG) signals related to motor activity, such as alpha- and beta-band motor event-related desynchronizations (ERDs), is essential for Brain Computer Interface (BCI) development. Determining the best electrode combination to detect the ERD is crucial for the success of the BCI. Considering that the EEG signals are bipolar, this involves the choice of the main and reference electrodes. So far, no strategy to guarantee signals free of the activity from the reference electrode has achieved consensus among the scientific community. Therefore, mapping the ERD in terms of the spatial distribution of the main and reference electrodes can provide additional perspectives for the BCI field. The goal of this work is to identify subject-specific channels where ERD is temporally coupled to the initiation of an upper-limb motor task. We defined a criterion to determine the presence of the ERD linked to the movement onset and searched, separately for each subject, for the single channel with the most prominent ERD. The search was conducted over all available channels composed by a pair of electrodes, and the selected signals were analyzed according to their temporal and spatial characteristics. We found that alpha- and beta-band ERD temporarily linked to movement onset can be detected in atypical channels (pairs of electrodes) across the scalp. The selected channels were different across subjects. Four ERD temporal patterns were observed in terms of the initiation instant of the ERD. These patterns revealed that the M1 cortex seems to be related to later ERDs. Moreover, they were also associated to different cortical processes related to the motor task. To the best of our knowledge, this is the first time these findings are reported. Aiming at BCI development, further experiments with more subjects and with motor-imagery tasks are desirable for more robustness and applicability of these findings.

[Three-dimensional convolutional neural network based on spatial-spectral feature pictures learning for decoding motor imagery electroencephalography signal]

The brain-computer interface (BCI) based on motor imagery electroencephalography (EEG) shows great potential in neurorehabilitation due to its non-invasive nature and ease of use. However, motor imagery EEG signals have low signal-to-noise ratios and spatiotemporal resolutions, leading to low decoding recognition rates with traditional neural networks. To address this, this paper proposed a three-dimensional (3D) convolutional neural network (CNN) method that learns spatial-frequency feature maps, using Welch method to calculate the power spectrum of EEG frequency bands, converted time-series EEG into a brain topographical map with spatial-frequency information. A 3D network with one-dimensional and two-dimensional convolutional layers was designed to effectively learn these features. Comparative experiments demonstrated that the average decoding recognition rate reached 86.89%, outperforming traditional methods and validating the effectiveness of this approach in motor imagery EEG decoding.

3D convolutional neural network based on spatial-spectral feature pictures learning for decoding motor imagery EEG signal

Non-invasive brain-computer interfaces (BCI) hold great promise in the field of neurorehabilitation. They are easy to use and do not require surgery, particularly in the area of motor imagery electroencephalography (EEG). However, motor imagery EEG signals often have a low signal-to-noise ratio and limited spatial and temporal resolution. Traditional deep neural networks typically only focus on the spatial and temporal features of EEG, resulting in relatively low decoding and accuracy rates for motor imagery tasks. To address these challenges, this paper proposes a 3D Convolutional Neural Network (P-3DCNN) decoding method that jointly learns spatial-frequency feature maps from the frequency and spatial domains of the EEG signals. First, the Welch method is used to calculate the frequency band power spectrum of the EEG, and a 2D matrix representing the spatial topology distribution of the electrodes is constructed. These spatial-frequency representations are then generated through cubic interpolation of the temporal EEG data. Next, the paper designs a 3DCNN network with 1D and 2D convolutional layers in series to optimize the convolutional kernel parameters and effectively learn the spatial-frequency features of the EEG. Batch normalization and dropout are also applied to improve the training speed and classification performance of the network. Finally, through experiments, the proposed method is compared to various classic machine learning and deep learning techniques. The results show an average decoding accuracy rate of 86.69%, surpassing other advanced networks. This demonstrates the effectiveness of our approach in decoding motor imagery EEG and offers valuable insights for the development of BCI.

Graph neural network based on brain inspired forward-forward mechanism for motor imagery classification in brain-computer interfaces

Introduction

Within the development of brain-computer interface (BCI) systems, it is crucial to consider the impact of brain network dynamics and neural signal transmission mechanisms on electroencephalogram-based motor imagery (MI-EEG) tasks. However, conventional deep learning (DL) methods cannot reflect the topological relationship among electrodes, thereby hindering the effective decoding of brain activity.

Methods

Inspired by the concept of brain neuronal forward-forward (F-F) mechanism, a novel DL framework based on Graph Neural Network combined forward-forward mechanism (F-FGCN) is presented. F-FGCN framework aims to enhance EEG signal decoding performance by applying functional topological relationships and signal propagation mechanism. The fusion process involves converting the multi-channel EEG into a sequence of signals and constructing a network grounded on the Pearson correlation coeffcient, effectively representing the associations between channels. Our model initially pre-trains the Graph Convolutional Network (GCN), and fine-tunes the output layer to obtain the feature vector. Moreover, the F-F model is used for advanced feature extraction and classification.

Results and discussion

Achievement of F-FGCN is assessed on the PhysioNet dataset for a four-class categorization, compared with various classical and state-of-the-art models. The learned features of the F-FGCN substantially amplify the performance of downstream classifiers, achieving the highest accuracy of 96.11% and 82.37% at the subject and group levels, respectively. Experimental results affirm the potency of FFGCN in enhancing EEG decoding performance, thus paving the way for BCI applications.

Multiclass classification of motor imagery tasks based on multi-branch convolutional neural network and temporal convolutional network model

Motor imagery (MI) is a cognitive process wherein an individual mentally rehearses a specific movement without physically executing it. Recently, MI-based brain-computer interface (BCI) has attracted widespread attention. However, accurate decoding of MI and understanding of neural mechanisms still face huge challenges. These seriously hinder the clinical application and development of BCI systems based on MI. Thus, it is very necessary to develop new methods to decode MI tasks. In this work, we propose a multi-branch convolutional neural network (MBCNN) with a temporal convolutional network (TCN), an end-to-end deep learning framework to decode multi-class MI tasks. We first used MBCNN to capture the MI electroencephalography signals information on temporal and spectral domains through different convolutional kernels. Then, we introduce TCN to extract more discriminative features. The within-subject cross-session strategy is used to validate the classification performance on the dataset of BCI Competition IV-2a. The results showed that we achieved 75.08% average accuracy for 4-class MI task classification, outperforming several state-of-the-art approaches. The proposed MBCNN-TCN-Net framework successfully captures discriminative features and decodes MI tasks effectively, improving the performance of MI-BCIs. Our findings could provide significant potential for improving the clinical application and development of MI-based BCI systems.

Brain-Computer Interfaces for Upper Limb Motor Recovery after Stroke: Current Status and Development Prospects (Review)

Brain-computer interfaces (BCIs) are a group of technologies that allow mental training with feedback for post-stroke motor recovery. Varieties of these technologies have been studied in numerous clinical trials for more than 10 years, and their construct and software are constantly being improved. Despite the positive treatment results and the availability of registered medical devices, there are currently a number of problems for the wide clinical application of BCI technologies. This review provides information on the most studied types of BCIs and its training protocols and describes the evidence base for the effectiveness of BCIs for upper limb motor recovery after stroke. The main problems of scaling this technology and ways to solve them are also described.

TSPNet: a time-spatial parallel network for classification of EEG-based multiclass upper limb motor imagery BCI

The classification of electroencephalogram (EEG) motor imagery signals has emerged as a prominent research focus within the realm of brain-computer interfaces. Nevertheless, the conventional, limited categories (typically just two or four) offered by brain-computer interfaces fail to provide an extensive array of control modes. To address this challenge, we propose the Time-Spatial Parallel Network (TSPNet) for recognizing six distinct categories of upper limb motor imagery. Within TSPNet, temporal and spatial features are extracted separately, with the time dimension feature extractor and spatial dimension feature extractor performing their respective functions. Following this, the Time-Spatial Parallel Feature Extractor is employed to decouple the connection between temporal and spatial features, thus diminishing feature redundancy. The Time-Spatial Parallel Feature Extractor deploys a gating mechanism to optimize weight distribution and parallelize time-spatial features. Additionally, we introduce a feature visualization algorithm based on signal occlusion frequency to facilitate a qualitative analysis of TSPNet. In a six-category scenario, TSPNet achieved an accuracy of 49.1% ± 0.043 on our dataset and 49.7% ± 0.029 on a public dataset. Experimental results conclusively establish that TSPNet outperforms other deep learning methods in classifying data from these two datasets. Moreover, visualization results vividly illustrate that our proposed framework can generate distinctive classifier patterns for multiple categories of upper limb motor imagery, discerned through signals of varying frequencies. These findings underscore that, in comparison to other deep learning methods, TSPNet excels in intention recognition, which bears immense significance for non-invasive brain-computer interfaces.

Coupling Effects of Cross-Corticomuscular Association during Object Manipulation Tasks on Different Haptic Sensations

The effects of corticomuscular connectivity during object manipulation tasks with different haptic sensations have not been quantitatively investigated. Connectivity analyses enable the study of cortical effects and muscle responses during movements, revealing communication pathways between the brain and muscles. This study aims to examine the corticomuscular connectivity of three Electroencephalography (EEG) channels and five muscles during object manipulation tasks involving contact surfaces of Sandpaper, Suede, and Silk. The analyses included 12 healthy subjects performing tasks with their right hand. Power-Based Connectivity (PBC) and Mutual Information (MI) measures were utilized to evaluate significant differences in connectivity between contact surfaces, EEG channels, muscles, and frequency bands. The research yielded the following findings: Suede contact surface exhibited higher connectivity; Mu and Gamma frequency bands exerted greater influence; significant connectivity was observed between the three EEG channels (C 3 ,C z ,C 4 ) and the Anterior Deltoid (AD) and Brachioradialis (B) muscles; and connectivity was primarily involved during active movement in the AD muscle compared to the resting state. These findings suggest potential implementation in motor rehabilitation for more complex movements using novel alternative training systems with high effectiveness.