Motor imagery EEG classification algorithm based on CNN-LSTM feature fusion network

Self-supervised motor imagery EEG recognition model based on 1-D MTCNN-LSTM network

Objective.Aiming for the research on the brain-computer interface (BCI), it is crucial to design a MI-EEG recognition model, possessing a high classification accuracy and strong generalization ability, and not relying on a large number of labeled training samples.Approach.In this paper, we propose a self-supervised MI-EEG recognition method based on self-supervised learning with one-dimensional multi-task convolutional neural networks and long short-term memory (1-D MTCNN-LSTM). The model is divided into two stages: signal transform identification stage and pattern recognition stage. In the signal transform recognition phase, the signal transform dataset is recognized by the upstream 1-D MTCNN-LSTM network model. Subsequently, the backbone network from the signal transform identification phase is transferred to the pattern recognition phase. Then, it is fine-tuned using a trace amount of labeled data to finally obtain the motion recognition model.Main results.The upstream stage of this study achieves more than 95% recognition accuracy for EEG signal transforms, up to 100%. For MI-EEG pattern recognition, the model obtained recognition accuracies of 82.04% and 87.14% with F1 scores of 0.7856 and 0.839 on the datasets of BCIC-IV-2b and BCIC-IV-2a.Significance.The improved accuracy proves the superiority of the proposed method. It is prospected to be a method for accurate classification of MI-EEG in the BCI system.

Enhancing Classification Accuracy with Integrated Contextual Gate Network: Deep Learning Approach for Functional Near-Infrared Spectroscopy Brain-Computer Interface Application

Brain-computer interface (BCI) systems include signal acquisition, preprocessing, feature extraction, classification, and an application phase. In fNIRS-BCI systems, deep learning (DL) algorithms play a crucial role in enhancing accuracy. Unlike traditional machine learning (ML) classifiers, DL algorithms eliminate the need for manual feature extraction. DL neural networks automatically extract hidden patterns/features within a dataset to classify the data. In this study, a hand-gripping (closing and opening) two-class motor activity dataset from twenty healthy participants is acquired, and an integrated contextual gate network (ICGN) algorithm (proposed) is applied to that dataset to enhance the classification accuracy. The proposed algorithm extracts the features from the filtered data and generates the patterns based on the information from the previous cells within the network. Accordingly, classification is performed based on the similar generated patterns within the dataset. The accuracy of the proposed algorithm is compared with the long short-term memory (LSTM) and bidirectional long short-term memory (Bi-LSTM). The proposed ICGN algorithm yielded a classification accuracy of 91.23 ± 1.60%, which is significantly (p < 0.025) higher than the 84.89 ± 3.91 and 88.82 ± 1.96 achieved by LSTM and Bi-LSTM, respectively. An open access, three-class (right- and left-hand finger tapping and dominant foot tapping) dataset of 30 subjects is used to validate the proposed algorithm. The results show that ICGN can be efficiently used for the classification of two- and three-class problems in fNIRS-based BCI applications.

Multilevel Laser-Induced Pain Measurement with Wasserstein Generative Adversarial Network - Gradient Penalty Model

Pain is an experience of unpleasant sensations and emotions associated with actual or potential tissue damage. In the global context, billions of people are affected by pain disorders. There are particular challenges in the measurement and assessment of pain, and the commonly used pain measuring tools include traditional subjective scoring methods and biomarker-based measures. The main tools for biomarker-based analysis are electroencephalography (EEG), electrocardiography and functional magnetic resonance. The EEG-based quantitative pain measurements are of immense value in clinical pain management and can provide objective assessments of pain intensity. The assessment of pain is now primarily limited to the identification of the presence or absence of pain, with less research on multilevel pain. High power laser stimulation pain experimental paradigm and five pain level classification methods based on EEG data augmentation are presented. First, the EEG features are extracted using modified S-transform, and the time-frequency information of the features is retained. Based on the pain recognition effect, the 20-40[Formula: see text]Hz frequency band features are optimized. Afterwards the Wasserstein generative adversarial network with gradient penalty is used for feature data augmentation. It can be inferred from the good classification performance of features in the parietal region of the brain that the sensory function of the parietal lobe region is effectively activated during the occurrence of pain. By comparing the latest data augmentation methods and classification algorithms, the proposed method has significant advantages for the five-level pain dataset. This research provides new ways of thinking and research methods related to pain recognition, which is essential for the study of neural mechanisms and regulatory mechanisms of pain.

Fog Computing for Control of Cyber-Physical Systems in Industry Using BCI

Brain-computer interfaces use signals from the brain, such as EEG, to determine brain states, which in turn can be used to issue commands, for example, to control industrial machinery. While Cloud computing can aid in the creation and operation of industrial multi-user BCI systems, the vast amount of data generated from EEG signals can lead to slow response time and bandwidth problems. Fog computing reduces latency in high-demand computation networks. Hence, this paper introduces a fog computing solution for BCI processing. The solution consists in using fog nodes that incorporate machine learning algorithms to convert EEG signals into commands to control a cyber-physical system. The machine learning module uses a deep learning encoder to generate feature images from EEG signals that are subsequently classified into commands by a random forest. The classification scheme is compared using various classifiers, being the random forest the one that obtained the best performance. Additionally, a comparison was made between the fog computing approach and using only cloud computing through the use of a fog computing simulator. The results indicate that the fog computing method resulted in less latency compared to the solely cloud computing approach.

Automatic epileptic seizure detection based on EEG using a moth-flame optimization of one-dimensional convolutional neural networks

Introduction

Frequent epileptic seizures can cause irreversible damage to the brains of patients. A potential therapeutic approach is to detect epileptic seizures early and provide artificial intervention to the patient. Currently, extracting electroencephalogram (EEG) features to detect epileptic seizures often requires tedious methods or the repeated adjustment of neural network hyperparameters, which can be time- consuming and demanding for researchers.

Methods

This study proposes an automatic detection model for an EEG based on moth-flame optimization (MFO) optimized one-dimensional convolutional neural networks (1D-CNN). First, according to the characteristics and need for early epileptic seizure detection, a data augmentation method for dividing an EEG into small samples is proposed. Second, the hyperparameters are tuned based on MFO and trained for an EEG. Finally, the softmax classifier is used to output EEG classification from a small-sample and single channel.

Results

The proposed model is evaluated with the Bonn EEG dataset, which verifies the feasibility of EEG classification problems that involve up to five classes, including healthy, preictal, and ictal EEG from various brain regions and individuals.

Discussion

Compared with existing advanced optimization algorithms, such as particle swarm optimization, genetic algorithm, and grey wolf optimizer, the superiority of the proposed model is further verified. The proposed model can be implemented into an automatic epileptic seizure detection system to detect seizures in clinical applications.

Deep temporal networks for EEG-based motor imagery recognition

The electroencephalogram (EEG) based motor imagery (MI) signal classification, also known as motion recognition, is a highly popular area of research due to its applications in robotics, gaming, and medical fields. However, the problem is ill-posed as these signals are non-stationary and noisy. Recently, a lot of efforts have been made to improve MI signal classification using a combination of signal decomposition and machine learning techniques but they fail to perform adequately on large multi-class datasets. Previously, researchers have implemented long short-term memory (LSTM), which is capable of learning the time-series information, on the MI-EEG dataset for motion recognition. However, it can not model very long-term dependencies present in the motion recognition data. With the advent of transformer networks in natural language processing (NLP), the long-term dependency issue has been widely addressed. Motivated by the success of transformer algorithms, in this article, we propose a transformer-based deep learning neural network architecture that performs motion recognition on the raw BCI competition III IVa and IV 2a datasets. The validation results show that the proposed method achieves superior performance than the existing state-of-the-art methods. The proposed method produces classification accuracy of 99.7% and 84% on the binary class and the multi-class datasets, respectively. Further, the performance of the proposed transformer-based model is also compared with LSTM.

Motor Imagery EEG Classification Based on a Weighted Multi-Branch Structure Suitable for Multisubject Data

OBJECTIVE

Electroencephalogram (EEG) signal recognition based on deep learning technology requires the support of sufficient data. However, training data scarcity usually occurs in subject-specific motor imagery tasks unless multisubject data can be used to enlarge training data. Unfortunately, because of the large discrepancies between data distributions from different subjects, model performance could only be improved marginally or even worsened by simply training on multisubject data.

METHOD

This article proposes a novel weighted multi-branch (WMB) structure for handling multisubject data to solve the problem, in which each branch is responsible for fitting a pair of source-target subject data and adaptive weights are used to integrate all branches or select branches with the largest weights to make the final decision. The proposed WMB structure was applied to six well-known deep learning models (EEGNet, Shallow ConvNet, Deep ConvNet, ResNet, MSFBCNN, and EEG_TCNet) and comprehensive experiments were conducted on EEG datasets BCICIV-2a, BCICIV-2b, high gamma dataset (HGD) and two supplementary datasets.

RESULT

Superior results against the state-of-the-art models have demonstrated the efficacy of the proposed method in subject-specific motor imagery EEG classification. For example, the proposed WMB_EEGNet achieved classification accuracies of 84.14%, 90.23%, and 97.81% on BCICIV-2a, BCICIV-2b and HGD, respectively.

CONCLUSION

It is clear that the proposed WMB structure is capable to make good use of multisubject data with large distribution discrepancies for subject-specific EEG classification.

Application and Development of EEG Acquisition and Feedback Technology: A Review

This review focuses on electroencephalogram (EEG) acquisition and feedback technology and its core elements, including the composition and principles of the acquisition devices, a wide range of applications, and commonly used EEG signal classification algorithms. First, we describe the construction of EEG acquisition and feedback devices encompassing EEG electrodes, signal processing, and control and feedback systems, which collaborate to measure faint EEG signals from the scalp, convert them into interpretable data, and accomplish practical applications using control feedback systems. Subsequently, we examine the diverse applications of EEG acquisition and feedback across various domains. In the medical field, EEG signals are employed for epilepsy diagnosis, brain injury monitoring, and sleep disorder research. EEG acquisition has revealed associations between brain functionality, cognition, and emotions, providing essential insights for psychologists and neuroscientists. Brain-computer interface technology utilizes EEG signals for human-computer interaction, driving innovation in the medical, engineering, and rehabilitation domains. Finally, we introduce commonly used EEG signal classification algorithms. These classification tasks can identify different cognitive states, emotional states, brain disorders, and brain-computer interface control and promote further development and application of EEG technology. In conclusion, EEG acquisition technology can deepen the understanding of EEG signals while simultaneously promoting developments across multiple domains, such as medicine, science, and engineering.

Classification Algorithm for Electroencephalogram-based Motor Imagery Using Hybrid Neural Network with Spatio-temporal Convolution and Multi-head Attention Mechanism

Motor imagery (MI) is a brain-computer interface (BCI) technique in which specific brain regions are activated when people imagine their limbs (or muscles) moving, even without actual movement. The technology converts electroencephalogram (EEG) signals generated by the brain into computer-readable commands by measuring neural activity. Classification of motor imagery is one of the tasks in BCI. Researchers have done a lot of work on motor imagery classification, and the existing literature has relatively mature decoding methods for two-class motor tasks. However, as the categories of EEG-based motor imagery tasks increase, further exploration is needed for decoding research on four-class motor imagery tasks. In this study, we designed a hybrid neural network that combines spatiotemporal convolution and attention mechanisms. Specifically, the data is first processed by spatiotemporal convolution to extract features and then processed by a Multi-branch Convolution block. Finally, the processed data is input into the encoder layer of the Transformer for a self-attention calculation to obtain the classification results. Our approach was tested on the well-known MI datasets BCI Competition IV 2a and 2b, and the results show that the 2a dataset has a global average classification accuracy of 83.3% and a kappa value of 0.78. Experimental results show that the proposed method outperforms most of the existing methods.

Prediction of response to repetitive transcranial magnetic stimulation for major depressive disorder using hybrid Convolutional recurrent neural networks and raw Electroencephalogram Signal

Major Depressive Disorder (MDD) is a high prevalence disease that needs an effective and timely treatment to prevent its progress and additional costs. Repetitive Transcranial Magnetic Stimulation (rTMS) is an effective treatment option for MDD patients which uses strong magnetic pulses to stimulate specific regions of the brain. However, some patients do not respond to this treatment which causes the waste of multiple weeks as treatment time and clinical resources. Therefore developing an effective way for the prediction of response to the rTMS treatment of depression is necessary. In this work, we proposed a hybrid model created by pre-trained Convolutional Neural Networks (CNN) models and Bidirectional Long Short-Term Memory (BLSTM) cells to predict response to rTMS treatment from raw EEG signal. Three pre-trained CNN models named VGG16, InceptionResNetV2, and EffecientNetB0 were utilized as Transfer Learning (TL) models to construct hybrid TL-BLSTM models. Then an ensemble of these models was created using weighted majority voting which the weights were optimized by Differential Evolution (DE) optimization algorithm. Evaluation of these models shows the superior performance of the ensemble model by the accuracy of 98.51%, sensitivity of 98.64%, specificity of 98.36%, F1-score of 98.6%, and AUC of 98.5%. Therefore, the ensemble of the proposed hybrid convolutional recurrent networks can efficiently predict the treatment outcome of rTMS using raw EEG data.

An Evaluation of Hybrid Deep Learning Models for Classifying Multiple Lower Limb Actions

Brain-computer Interfaces (BCIs) interpret electroencephalography (EEG) signals and translate them into control commands for operating external devices. The motor imagery (MI) paradigm is popular in this context. Recent research has demonstrated that deep learning models, such as convolutional neural network (CNN) and long short-term memory (LSTM), are successful in a wide range of classification applications. This is because CNN has the property of spatial invariance, and LSTM can capture temporal associations among features. A combination of CNN and LSTM could enhance the classification performance of EEG signals due to the complementation of their strengths. Such a combination has been applied to MI classification based on EEG. However, most studies focused on either the upper limbs or treated both lower limbs as a single class, with only limited research performed on separate lower limbs. We, therefore, explored hybrid models (different combinations of CNN and LSTM) and evaluated them in the case of individual lower limbs. In addition, we classified multiple actions: MI, real movements and movement observations using four typical hybrid models and aimed to identify which model was the most suitable. The comparison results demonstrated that no model was significantly better than the others in terms of classification accuracy, but all of them were better than the chance level. Our study informs the possibility of the use of multiple actions in BCI systems and provides useful information for further research into the classification of separate lower limb actions.

EEG motor imagery classification using deep learning approaches in naïve BCI users

Motor Imagery (MI)-Brain Computer-Interfaces (BCI) illiteracy defines that not all subjects can achieve a good performance in MI-BCI systems due to different factors related to the fatigue, substance consumption, concentration, and experience in the use. To reduce the effects of lack of experience in the use of BCI systems (naïve users), this paper presents the implementation of three Deep Learning (DL) methods with the hypothesis that the performance of BCI systems could be improved compared with baseline methods in the evaluation of naïve BCI users. The methods proposed here are based on Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM)/Bidirectional Long Short-Term Memory (BiLSTM), and a combination of CNN and LSTM used for upper limb MI signal discrimination on a dataset of 25 naïve BCI users. The results were compared with three widely used baseline methods based on the Common Spatial Pattern (CSP), Filter Bank Common Spatial Pattern (FBCSP), and Filter Bank Common Spatial-Spectral Pattern (FBCSSP), in different temporal window configurations. As results, the LSTM-BiLSTM-based approach presented the best performance, according to the evaluation metrics of Accuracy, F-score, Recall, Specificity, Precision, and ITR, with a mean performance of 80% (maximum 95%) and ITR of 10 bits/min using a temporal window of 1.5 s. The DL Methods represent a significant increase of 32% compared with the baseline methods (p< 0.05). Thus, with the outcomes of this study, it is expected to increase the controllability, usability, and reliability of the use of robotic devices in naïve BCI users.

Recognize enhanced temporal-spatial-spectral features with a parallel multi-branch CNN and GRU

Deep learning has been applied to the recognition of motor imagery electroencephalograms (MI-EEG) in brain-computer interface, and the performance results depend on data representation as well as neural network structure. Especially, MI-EEG is so complex with the characteristics of non-stationarity, specific rhythms, and uneven distribution; however, its multidimensional feature information is difficult to be fused and enhanced simultaneously in the existing recognition methods. In this paper, a novel channel importance (NCI) based on time-frequency analysis is proposed to develop an image sequence generation method (NCI-ISG) for enhancing the integrity of data representation and highlighting the contribution inequalities of different channels as well. Each electrode of MI-EEG is converted to a time-frequency spectrum by utilizing short-time Fourier transform; the corresponding part to 8-30 Hz is combined with random forest algorithm for computing NCI; and it is further divided into three sub-images covered by α (8-13 Hz), β₁ (13-21 Hz), and β₂ (21-30 Hz) bands; their spectral powers are further weighted by NCI and interpolated to 2-dimensional electrode coordinates, producing three main sub-band image sequences. Then, a parallel multi-branch convolutional neural network and gate recurrent unit (PMBCG) is designed to successively extract and identify the spatial-spectral and temporal features from the image sequences. Two public four-class MI-EEG datasets are adopted; the proposed classification method respectively achieves the average accuracies of 98.26% and 80.62% by 10-fold cross-validation experiment; and its statistical performance is also evaluated by multi-indexes, such as Kappa value, confusion matrix, and ROC curve. Extensive experiment results show that NCI-ISG + PMBCG can yield great performance on MI-EEG classification compared to state-of-the-art methods. The proposed NCI-ISG can enhance the feature representation of time-frequency-space domains and match well with PMBCG, which improves the recognition accuracies of MI tasks and demonstrates the preferable reliability and distinguishable ability. This paper proposes a novel channel importance (NCI) based on time-frequency analysis to develop an image sequences generation method (NCI-ISG) for enhancing the integrity of data representation and highlighting the contribution inequalities of different channels as well. Then, a parallel multi-branch convolutional neural network and gate recurrent unit (PMBCG) is designed to successively extract and identify the spatial-spectral and temporal features from the image sequences.

EEG Signal Complexity Measurements to Enhance BCI-Based Stroke Patients' Rehabilitation

The second leading cause of death and one of the most common causes of disability in the world is stroke. Researchers have found that brain-computer interface (BCI) techniques can result in better stroke patient rehabilitation. This study used the proposed motor imagery (MI) framework to analyze the electroencephalogram (EEG) dataset from eight subjects in order to enhance the MI-based BCI systems for stroke patients. The preprocessing portion of the framework comprises the use of conventional filters and the independent component analysis (ICA) denoising approach. Fractal dimension (FD) and Hurst exponent (Hur) were then calculated as complexity features, and Tsallis entropy (TsEn) and dispersion entropy (DispEn) were assessed as irregularity parameters. The MI-based BCI features were then statistically retrieved from each participant using two-way analysis of variance (ANOVA) to demonstrate the individuals' performances from four classes (left hand, right hand, foot, and tongue). The dimensionality reduction algorithm, Laplacian Eigenmap (LE), was used to enhance the MI-based BCI classification performance. Utilizing k-nearest neighbors (KNN), support vector machine (SVM), and random forest (RF) classifiers, the groups of post-stroke patients were ultimately determined. The findings show that LE with RF and KNN obtained 74.48% and 73.20% accuracy, respectively; therefore, the integrated set of the proposed features along with ICA denoising technique can exactly describe the proposed MI framework, which may be used to explore the four classes of MI-based BCI rehabilitation. This study will help clinicians, doctors, and technicians make a good rehabilitation program for people who have had a stroke.

Enhancement of motor imagery training efficiency by an online adaptive training paradigm integrated with error related potential

Objective. Motor imagery (MI) is a process of autonomously modulating the motor area to rehearse action mentally without actual execution. Based on the neuroplasticity of the cerebral cortex, MI can promote the functional rehabilitation of the injured cerebral cortex motor area. However, it usually takes several days to a few months to train individuals to acquire the necessary MI ability to control rehabilitation equipment in current studies, which greatly limits the clinical application of rehabilitation training systems based on the MI brain-computer interface (BCI).Approach. A novel MI training paradigm combined with the error related potential (ErrP) is proposed, and online adaptive training of the MI classifier was performed using ErrP. ErrP is used to correct the output of the MI classification to obtain a higher accuracy of kinesthetic feedback based on the imagination intention of subjects while generating simulated labels for MI online adaptive training. In this way, we improved the MI training efficiency. Thirteen subjects were randomly divided into an experimental group using the proposed paradigm and a control group using the traditional MI training paradigm to participate in six MI training experiments.Main results. The proposed paradigm enabled the experimental group to obtain a higher event-related desynchronization modulation level in the contralateral brain region compared with the control group and 69.76% online classification accuracy of MI after three MI training experiments. The online classification accuracy reached 72.76% and the whole system recognized the MI intention of the subjects with an online accuracy of 82.61% after six experiments.Significance. Compared with the conventional unimodal MI training strategy, the proposed approach enables subjects to use the MI-BCI based system directly and achieve a better performance after only three training experiments with training left and right hands simultaneously. This greatly improves the usability of the MI-BCI-based rehabilitation system and makes it more convenient for clinical use.