A Graph-Based Hierarchical Attention Model for Movement Intention Detection from EEG Signals

An epilepsy detection method based on multi-dimensional feature extraction and dual-branch hypergraph convolutional network

Epilepsy is a disease caused by abnormal neural discharge, which severely harms the health of patients. Its pathogenesis is complex and variable with various forms of seizures, leading to significant differences in epilepsy manifestations among different patients. The changes of brain network are strongly correlated with related pathologies. Therefore, it is crucial to effectively and deeply explore the intrinsic features of epilepsy signals to reveal the rules of epilepsy occurrence and achieve accurate detection. Existing methods have faced the following issues: 1) single approach for feature extraction, resulting in insufficient classification information due to the lack of rich dimensions in captured features; 2) inability to deeply analyze the essential commonality of epilepsy signal after feature extraction, making the model susceptible to data distribution and noise interference. Thus, we proposed a high-precision and robust model for epileptic seizure detection, which, for the first time, applies hypergraph convolution to the field of epilepsy detection. Through a hypergraph network structure constructed based on relationships between channels in electroencephalogram (EEG) signals, the model explores higher-order characteristics of epilepsy EEG data. Specifically, we use the Conv-LSTM module and Power spectral density (PSD), a two-branch parallel method, to extract channel features from space-time and frequency domains to solve the problem of insufficient feature extraction, and can adequately describe the data structure and distribution from multiple perspectives through double-branch parallel feature extraction. In addition, we construct a hypergraph on the captured features to explore the intrinsic features in the high-dimensional space in an attempt to reveal the essential commonality of epileptic signal feature extraction. Finally, using the ensemble learning concept, we accomplished epilepsy detection on the dual-branch hypergraph convolution. The model underwent leave-one-out cross-validation on the TUH dataset, achieving an average accuracy of 96.9%, F1 score of 97.3%, Pre of 98.2% and Re of 96.7%. In addition, the model was generalized performance tested on CHB-MIT scalp EEG dataset with leave-one-out cross-validation, and the average ACC, F1 score, Pre and Re were 94.4%, 95.1%, 95.8%, and 93.9% respectively. Experimental results indicate that the model outperforms related literature, providing valuable reference for the clinical application of epilepsy detection.

MuLHiTA: A Novel Multiclass Classification Framework With Multibranch LSTM and Hierarchical Temporal Attention for Early Detection of Mental Stress

Mental stress is an increasingly common psychological issue leading to diseases such as depression, addiction, and heart attack. In this study, an early detection framework based on electroencephalogram (EEG) data is developed for reducing the risk of these diseases. In existing frameworks, signals are often segmented into smaller sections prior to being input to a deep neural network. However, this approach ignores the fundamental nature of EEG signals as a carrier of valuable information (e.g., the integrity of frequency and phase, and temporal fluctuations of EEG components). As such, this type of segmenting may lead to information loss and a failure to effectively identify mental stress levels. Thus, we propose a novel multiclass classification framework termed multibranch LSTM and hierarchical temporal attention (MuLHiTA) for the early identification of mental stress levels. It specifically focuses on not only intraslice (within each slice) but also interslice (between different slices) samples in parallel. This was achieved by including two complementary branches, each of which integrated a specifically designed attention module into a bidirectional long short-term memory (BLSTM) network, enabling extraction of the most discriminative features from interslice and intraslice EEG signals simultaneously. The outputs of attention modules were then summed to obtain a feature representation that contributes to reduce overfitting and more effective multiclass classification. In addition, electrode positions were optimized using neural activity areas under high-stress conditions, thereby reducing computational costs by minimizing the number of critical electrodes. MuLHiTA was evaluated across one private [Montreal imaging stress task (MIST)] and two publicly available EEG datasets [EEG during mental arithmetic tasks (DMAT) and Simultaneous task EEG workload (STEW)]. These were divided into training and test sets using an 8:2 ratio, and the training data were further divided into training and validation sets using a fivefold cross-validation (CV) method, in which the model with the highest accuracy among the five was selected. The model was trained once more with the full training set, and the test data were then used to evaluate its performance. This approach achieved average classification accuracies of 93.58%, 91.80%, and 99.71% for the MIST, STEW, and DMAT datasets, respectively. Experimental results showed MuLHiTA was superior to state-of-the-art algorithms, including EEGNet, BLSTM, EEGLearn, convolutional neural network (CNN)-long short-term memory (LSTM), and convolutional recurrent attention model (CRAM), for multiclass classification. This demonstrates the viability of MuLHiTA for the early detection of mental stress.

Dynamic Multi-Graph Convolution-Based Channel-Weighted Transformer Feature Fusion Network for Epileptic Seizure Prediction

Electroencephalogram (EEG) based seizure prediction plays an important role in the closed-loop neuromodulation system. However, most existing seizure prediction methods based on graph convolution network only focused on constructing the static graph, ignoring multi-domain dynamic changes in deep graph structure. Moreover, the existing feature fusion strategies generally concatenated coarse-grained epileptic EEG features directly, leading to the suboptimal seizure prediction performance. To address these issues, we propose a novel multi-branch dynamic multi-graph convolution based channel-weighted transformer feature fusion network (MB-dMGC-CWTFFNet) for the patient-specific seizure prediction with the superior performance. Specifically, a multi-branch (MB) feature extractor is first applied to capture the temporal, spatial and spectral representations fromthe epileptic EEG jointly. Then, we design a point-wise dynamic multi-graph convolution network (dMGCN) to dynamically learn deep graph structures, which can effectively extract high-level features from the multi-domain graph. Finally, by integrating the local and global channel-weighted strategies with the multi-head self-attention mechanism, a channel-weighted transformer feature fusion network (CWTFFNet) is adopted to efficiently fuse the multi-domain graph features. The proposed MB-dMGC-CWTFFNet is evaluated on the public CHB-MIT EEG and a private intracranial sEEG datasets, and the experimental results demonstrate that our proposed method achieves outstanding prediction performance compared with the state-of-the-art methods, indicating an effective tool for patient-specific seizure warning. Our code will be available at: https://github.com/Rockingsnow/MB-dMGC-CWTFFNet.

CLEP: Contrastive Learning for Epileptic Seizure Prediction Using a Spatio-Temporal-Spectral Network

Seizure prediction of epileptic preictal period through electroencephalogram (EEG) signals is important for clinical epilepsy diagnosis. However, recent deep learning-based methods commonly employ intra-subject training strategy and need sufficient data, which are laborious and time-consuming for a practical system and pose a great challenge for seizure predicting. Besides, multi-domain characterizations, including spatio-temporal-spectral dependencies in an epileptic brain are generally neglected or not considered simultaneously in current approaches, and this insufficiency commonly leads to suboptimal seizure prediction performance. To tackle the above issues, in this paper, we propose Contrastive Learning for Epileptic seizure Prediction (CLEP) using a Spatio-Temporal-Spectral Network (STS-Net). Specifically, the CLEP learns intrinsic epileptic EEG patterns across subjects by contrastive learning. The STS-Net extracts multi-scale temporal and spectral representations under different rhythms from raw EEG signals. Then, a novel triple attention layer (TAL) is employed to construct inter-dimensional interaction among multi-domain features. Moreover, a spatio dynamic graph convolution network (sdGCN) is proposed to dynamically model the spatial relationships between electrodes and aggregate spatial information. The proposed CLEP-STS-Net achieves a sensitivity of 96.7% and a false prediction rate of 0.072/h on the CHB-MIT scalp EEG database. We also validate the proposed method on clinical intracranial EEG (iEEG) database from our Xuanwu Hospital of Capital Medical University, and the predicting system yielded a sensitivity of 95%, a false prediction rate of 0.087/h. The experimental results outperform the state-of-the-art studies which validate the efficacy of our method. Our code is available at https://github.com/LianghuiGuo/CLEP-STS-Net.

Feature hypergraph representation learning on spatial-temporal correlations for EEG emotion recognition

Electroencephalogram(EEG) becomes popular in emotion recognition for its capability of selectively reflecting the real emotional states. Existing graph-based methods have made primary progress in representing pairwise spatial relationships, but leaving higher-order relationships among EEG channels and higher-order relationships inside EEG series. Constructing a hypergraph is a general way of representing higher-order relations. In this paper, we propose a spatial-temporal hypergraph convolutional network(STHGCN) to capture higher-order relationships that existed in EEG recordings. STHGCN is a two-block hypergraph convolutional network, in which feature hypergraphs are constructed over the spectrum, space, and time domains, to explore spatial and temporal correlations under specific emotional states, namely the correlations of EEG channels and the dynamic relationships of temporal stamps. What's more, a self-attention mechanism is combined with the hypergraph convolutional network to initialize and update the relationships of EEG series. The experimental results demonstrate that constructed feature hypergraphs can effectively capture the correlations among valuable EEG channels and the correlations inside valuable EEG series, leading to the best emotion recognition accuracy among the graph methods. In addition, compared with other competitive methods, the proposed method achieves state-of-art results on SEED and SEED-IV datasets.

Improved Skip-Gram Based on Graph Structure Information

Applying the Skip-gram to graph representation learning has become a widely researched topic in recent years. Prior works usually focus on the migration application of the Skip-gram model, while Skip-gram in graph representation learning, initially applied to word embedding, is left insufficiently explored. To compensate for the shortcoming, we analyze the difference between word embedding and graph embedding and reveal the principle of graph representation learning through a case study to explain the essential idea of graph embedding intuitively. Through the case study and in-depth understanding of graph embeddings, we propose Graph Skip-gram, an extension of the Skip-gram model using graph structure information. Graph Skip-gram can be combined with a variety of algorithms for excellent adaptability. Inspired by word embeddings in natural language processing, we design a novel feature fusion algorithm to fuse node vectors based on node vector similarity. We fully articulate the ideas of our approach on a small network and provide extensive experimental comparisons, including multiple classification tasks and link prediction tasks, demonstrating that our proposed approach is more applicable to graph representation learning.

Explainable Deep Learning Model for EMG-Based Finger Angle Estimation Using Attention

Electromyography (EMG) is one of the most common methods to detect muscle activities and intentions. However, it has been difficult to estimate accurate hand motions represented by the finger joint angles using EMG signals. We propose an encoder-decoder network with an attention mechanism, an explainable deep learning model that estimates 14 finger joint angles from forearm EMG signals. This study demonstrates that the model trained by the single-finger motion data can be generalized to estimate complex motions of random fingers. The color map result of the after-training attention matrix shows that the proposed attention algorithm enables the model to learn the nonlinear relationship between the EMG signals and the finger joint angles, which is explainable. The highly activated entries in the color map of the attention matrix derived from model training are consistent with the experimental observations in which certain EMG sensors are highly activated when a particular finger moves. In summary, this study proposes an explainable deep learning model that estimates finger joint angles based on EMG signals of the forearm using the attention mechanism.

Time-Distributed Attention Network for EEG-based Motor Imagery Decoding from the Same Limb

A brain-computer interface (BCI) based on motor imagery (MI) from the same limb can provide an intuitive control pathway but has received limited attention. It is still a challenge to classify multiple MI tasks from the same limb. The goal of this study is to propose a novel decoding method to classify the MI tasks of four joints of the same upper limb and the resting state. EEG signals were collected from 20 participants. A time-distributed attention network (TD-Atten) was proposed to adaptively assign different weights to different classes and frequency bands of the input multiband Common Spatial Pattern (CSP) features. The long short-term memory (LSTM) and dense layers were then used to learn sequential information from the reweight features and perform the classification. Our proposed method outperformed other baseline and deep learning-based methods and obtained the accuracies of 46.8% in the 5-class scenario and 53.4% in the 4-class scenario. The visualization results of attention weights indicated that the proposed framework can adaptively pay attention to alpha-band related features in MI tasks, which was consistent with the analysis of brain activation patterns. These results demonstrated the feasibility and interpretability of the attention mechanism in MI decoding and the potential of this fine MI paradigm to be applied for the control of a robotic arm or a neural prosthesis.

FB-CGANet: filter bank Channel Group Attention network for multi-class motor imagery classification

OBJECTIVE

Motor imagery-based brain computer interface (MI-BCI) is one of the most important BCI paradigms and can identify the target limb of subjects from the feature of MI-based Electroencephalography (EEG) signals. Deep learning methods, especially lightweight neural networks, provide an efficient technique for MI decoding, but the performance of lightweight neural networks is still limited and need further improving. This paper aimed to design a novel lightweight neural network for improving the performance of multi-class MI decoding.

APPROACH

A hybrid filter bank structure that can extract information in both time and frequency domain was proposed and combined with a novel channel attention method Channel Group Attention (CGA) to build a lightweight neural network Filter Bank Channel Group Attention Network (FB-CGANet). Accompanied with FB-CGANet, the Band Exchange data augmentation method was proposed to generate training data for networks with filter bank structure.

MAIN RESULTS

The proposed method can achieve higher 4-class average accuracy (79.4%) than compared methods on the BCI Competition IV IIa dataset in the experiment on the unseen evaluation data. Also, higher average accuracy (93.5%) than compared methods can be obtained in the cross-validation experiment.

SIGNIFICANCE

This work implies the effectiveness of channel attention and filter bank structure in lightweight neural networks and provides a novel option for multi-class motor imagery classification.

Persistent Homology-Based Topological Analysis on the Gestalt Patterns during Human Brain Cognition Process

The neuropsychological characteristics inside the brain are still not sufficiently understood in previous Gestalt psychological analyses. In particular, the extraction and analysis of human brain consciousness information itself have not received enough attention for the time being. In this paper, we aim to investigate the features of EEG signals from different conscious thoughts. Specifically, we try to extract the physiologically meaningful features of the brain responding to different contours and shapes in images in Gestalt cognitive tests by combining persistent homology analysis with electroencephalogram (EEG). The experimental results show that more brain regions in the frontal lobe are involved when the subject perceives the random and disordered combination of images compared to the ordered Gestalt images. Meanwhile, the persistence entropy of EEG data evoked by random sequence diagram (RSD) is significantly different from that evoked by the ordered Gestalt (GST) images in several frequency bands, which indicate that the human cognition of the shape and contour of images can be separated to some extent through topological analysis. This implies the feasibility to digitize the neural signals while preserving the whole and local features of the original signals, which are further verified by our extensive experiments. In general, this paper evaluates and quantifies cognitively related neural correlates by persistent homology features of EEG signals, which provides an approach to realizing the digitization of neural signals. Preliminary verification of the analyzability of human consciousness signals provides reliable research ideas and directions for the realization of feature extraction and analysis of human brain consciousness cognition.

Gated Transformer for Decoding Human Brain EEG Signals

In this work, we propose to use a deep learning framework for decoding the electroencephalogram (EEG) signals of human brain activities. More specifically, we learn an end-to-end model that recognizes natural images or motor imagery by the EEG data that is collected from the corresponding human neural activities. In order to capture the temporal information encoded in the long EEG sequences, we first employ an enhanced version of Transformer, i.e., gated Transformer, on EEG signals to learn the feature representation along a sequence of embeddings. Then a fully-connected Softmax layer is used to predict the classification results of the decoded representations. To demonstrate the effectiveness of the gated Transformer approach, we conduct experiments on the image classification task for a human brain-visual dataset and the classification task for a motor imagery dataset. The experimental results show that our method achieves new state-of-the-art performance compared to multiple existing methods that are widely used for EEG classification.

A Temporal-Spectral-Based Squeeze-and- Excitation Feature Fusion Network for Motor Imagery EEG Decoding

Motor imagery (MI) electroencephalography (EEG) decoding plays an important role in brain-computer interface (BCI), which enables motor-disabled patients to communicate with the outside world via external devices. Recent deep learning methods, which fail to fully explore both deep-temporal characterizations in EEGs itself and multi-spectral information in different rhythms, generally ignore the temporal or spectral dependencies in MI-EEG. Also, the lack of effective feature fusion probably leads to redundant or irrelative information and thus fails to achieve the most discriminative features, resulting in the limited MI-EEG decoding performance. To address these issues, in this paper, a MI-EEG decoding framework is proposed, which uses a novel temporal-spectral-based squeeze-and-excitation feature fusion network (TS-SEFFNet). First, the deep-temporal convolution block (DT-Conv block) implements convolutions in a cascade architecture, which extracts high-dimension temporal representations from raw EEG signals. Second, the multi-spectral convolution block (MS-Conv block) is then conducted in parallel using multi-level wavelet convolutions to capture discriminative spectral features from corresponding clinical subbands. Finally, the proposed squeeze-and-excitation feature fusion block (SE-Feature-Fusion block) maps the deep-temporal and multi-spectral features into comprehensive fused feature maps, which highlights channel-wise feature responses by constructing interdependencies among different domain features. Competitive experimental results on two public datasets demonstrate that our method is able to achieve promising decoding performance compared with the state-of-the-art methods.

Geometric Deep Learning for Subject-Independent Epileptic Seizure Prediction using Scalp EEG Signals

Recently, researchers in the biomedical community have introduced deep learning-based epileptic seizure prediction models using electroencephalograms (EEGs) that can anticipate an epileptic seizure by differentiating between the pre-ictal and interictal stages of the subjects brain. Despite having the appearance of a typical anomaly detection task, this problem is complicated by subject-specific characteristics in EEG data. Therefore, studies that investigate seizure prediction widely employ subject-specific models. However, this approach is not suitable in situations where a target subject has limited (or no) data for training. Subject-independent models can address this issue by learning to predict seizures from multiple subjects, and therefore are of greater value in practice. In this study, we propose a subject-independent seizure predictor using Geometric Deep Learning (GDL). In the first stage of our GDL-based method we use graphs derived from physical connections in the EEG grid. We subsequently seek to synthesize subject-specific graphs using deep learning. The models proposed in both stages achieve state-of-the-art performance using a one-hour early seizure prediction window on two benchmark datasets (CHB-MIT-EEG: 95.38% with 23 subjects and Siena-EEG: 96.05% with 15 subjects). To the best of our knowledge, this is the first study that proposes synthesizing subject-specific graphs for seizure prediction. Furthermore, through model interpretation we outline how this method can potentially contribute towards Scalp EEG-based seizure localization.

A Survey on Deep Learning-Based Short/Zero-Calibration Approaches for EEG-Based Brain-Computer Interfaces

Brain-computer interfaces (BCIs) utilizing machine learning techniques are an emerging technology that enables a communication pathway between a user and an external system, such as a computer. Owing to its practicality, electroencephalography (EEG) is one of the most widely used measurements for BCI. However, EEG has complex patterns and EEG-based BCIs mostly involve a cost/time-consuming calibration phase; thus, acquiring sufficient EEG data is rarely possible. Recently, deep learning (DL) has had a theoretical/practical impact on BCI research because of its use in learning representations of complex patterns inherent in EEG. Moreover, algorithmic advances in DL facilitate short/zero-calibration in BCI, thereby suppressing the data acquisition phase. Those advancements include data augmentation (DA), increasing the number of training samples without acquiring additional data, and transfer learning (TL), taking advantage of representative knowledge obtained from one dataset to address the so-called data insufficiency problem in other datasets. In this study, we review DL-based short/zero-calibration methods for BCI. Further, we elaborate methodological/algorithmic trends, highlight intriguing approaches in the literature, and discuss directions for further research. In particular, we search for generative model-based and geometric manipulation-based DA methods. Additionally, we categorize TL techniques in DL-based BCIs into explicit and implicit methods. Our systematization reveals advances in the DA and TL methods. Among the studies reviewed herein, ~45% of DA studies used generative model-based techniques, whereas ~45% of TL studies used explicit knowledge transferring strategy. Moreover, based on our literature review, we recommend an appropriate DA strategy for DL-based BCIs and discuss trends of TLs used in DL-based BCIs.

Recognition of Consumer Preference by Analysis and Classification EEG Signals

Neuromarketing has gained attention to bridge the gap between conventional marketing studies and electroencephalography (EEG)-based brain-computer interface (BCI) research. It determines what customers actually want through preference prediction. The performance of EEG-based preference detection systems depends on a suitable selection of feature extraction techniques and machine learning algorithms. In this study, We examined preference detection of neuromarketing dataset using different feature combinations of EEG indices and different algorithms for feature extraction and classification. For EEG feature extraction, we employed discrete wavelet transform (DWT) and power spectral density (PSD), which were utilized to measure the EEG-based preference indices that enhance the accuracy of preference detection. Moreover, we compared deep learning with other traditional classifiers, such as k-nearest neighbor (KNN), support vector machine (SVM), and random forest (RF). We also studied the effect of preference indicators on the performance of classification algorithms. Through rigorous offline analysis, we investigated the computational intelligence for preference detection and classification. The performance of the proposed deep neural network (DNN) outperforms KNN and SVM in accuracy, precision, and recall; however, RF achieved results similar to those of the DNN for the same dataset.

MI3DNet: A Compact CNN for Motor Imagery EEG Classification with Visualizable Dense Layer Parameters

Electroencephalography (EEG) based Brain Computer Interface (BCI) attracts more and more attention. Motor Imagery (MI) is a popular one among all the EEG paradigms. Building a subject-independent MI EEG classification procedure is a main challenge in practical applications. Recently, Convolutional Neural Network (CNN) has been introduced and achieved state-of-the-art performance in related areas. To extract subject-independent features in MI EEG classification, we propose the MI3DNet, using a remapped signal cubic as the input. Experiments show that MI3DNet has a higher performance with fewer parameters and layers. We also give a method to plot the parameters of the dense layer, and explain its effect.

Motor Imagery Classification via Temporal Attention Cues of Graph Embedded EEG Signals

Motor imagery classification from EEG signals is essential for motor rehabilitation with a Brain-Computer Interface (BCI). Most current works on this issue require a subject-specific adaptation step before applied to a new user. Thus the research of directly extending a pre-trained model to new users is particularly desired and indispensable. As brain dynamics fluctuate considerably across different subjects, it is challenging to design practical hand-crafted features based on prior knowledge. Regarding this gap, this paper proposes a Graph-based Convolutional Recurrent Attention Model (G-CRAM) to explore EEG features across different subjects for motor imagery classification. A graph structure is first developed to represent the positioning information of EEG nodes. Then a convolutional recurrent attention model learns EEG features from both spatial and temporal dimensions and emphasizes on the most distinguishable temporal periods. We evaluate the proposed approach on two benchmark EEG datasets of motor imagery classification on the subject-independent testing. The results show that the G-CRAM achieves superior performance to state-of-the-art methods regarding recognition accuracy and ROC-AUC. Furthermore, model interpretation studies reveal the learning process of different neural network components and demonstrate that the proposed model can extract detailed features efficiently.