Dual-CNN based multi-modal sleep scoring with temporal correlation driven fine-tuning

Self-Supervised Learning for Electroencephalography

Decades of research have shown machine learning superiority in discovering highly nonlinear patterns embedded in electroencephalography (EEG) records compared with conventional statistical techniques. However, even the most advanced machine learning techniques require relatively large, labeled EEG repositories. EEG data collection and labeling are costly. Moreover, combining available datasets to achieve a large data volume is usually infeasible due to inconsistent experimental paradigms across trials. Self-supervised learning (SSL) solves these challenges because it enables learning from EEG records across trials with variable experimental paradigms, even when the trials explore different phenomena. It aggregates multiple EEG repositories to increase accuracy, reduce bias, and mitigate overfitting in machine learning training. In addition, SSL could be employed in situations where there is limited labeled training data, and manual labeling is costly. This article: 1) provides a brief introduction to SSL; 2) describes some SSL techniques employed in recent studies, including EEG; 3) proposes current and potential SSL techniques for future investigations in EEG studies; 4) discusses the cons and pros of different SSL techniques; and 5) proposes holistic implementation tips and potential future directions for EEG SSL practices.

GAC-SleepNet: A dual-structured sleep staging method based on graph structure and Euclidean structure

Sleep staging is a precondition for the diagnosis and treatment of sleep disorders. However, how to fully exploit the relationship between spatial features of the brain and sleep stages is an important task. Many current classical algorithms only extract the characteristic information of the brain in the Euclidean space without considering other spatial structures. In this study, a sleep staging network named GAC-SleepNet is designed. GAC-SleepNet uses the characteristic information in the dual structure of the graph structure and the Euclidean structure for the classification of sleep stages. In the graph structure, this study uses a graph convolutional neural network to learn the deep features of each sleep stage and converts the features in the topological structure into feature vectors by a multilayer perceptron. In the Euclidean structure, this study uses convolutional neural networks to learn the temporal features of sleep information and combine attention mechanism to portray the connection between different sleep periods and EEG signals, while enhancing the description of global features to avoid local optima. In this study, the performance of the proposed network is evaluated on two public datasets. The experimental results show that the dual spatial structure captures more adequate and comprehensive information about sleep features and shows advancement in terms of different evaluation metrics.

EEG Reconstruction With a Dual-Scale CNN-LSTM Model for Deep Artifact Removal

Artifact removal has been an open critical issue for decades in tasks centering on EEG analysis. Recent deep learning methods mark a leap forward from the conventional signal processing routines; however, those in general still suffer from insufficient capabilities 1) to capture potential temporal dependencies embedded in EEG and 2) to adapt to scenarios without a priori knowledge of artifacts. This study proposes an approach (namely DuoCL) to deep artifact removal with a dual-scale CNN (Convolutional Neural Network)-LSTM (Long Short-Term Memory) model, operating on the raw EEG in three phases: 1) Morphological Feature Extraction, a dual-branch CNN utilizes convolution kernels of two different scales to learn morphological features (individual sample); 2) Feature Reinforcement, the dual-scale features are then reinforced with temporal dependencies (inter-sample) captured by LSTM; and 3) EEG Reconstruction, the resulting feature vectors are finally aggregated to reconstruct the artifact-free EEG via a terminal fully connected layer. Extensive experiments have been performed to compare DuoCL to six state-of-the-art counterparts (e.g., 1D-ResCNN and NovelCNN). DuoCL can reconstruct more accurate waveforms and achieve the highest ${\mathsf{SNR}}$ & correlation (${\mathsf{CC}}$) as well as the lowest error (${\mathsf{RRMSE}}_{\mathsf{t}}$ & ${\mathsf{RRMSE}}_{\mathsf{f}}$). In particular, DuoCL holds potentials in providing a high-quality removal of unknown and hybrid artifacts.

A Multilevel Temporal Context Network for Sleep Stage Classification

Sleep stage classification is essential in diagnosing and treating sleep disorders. Many deep learning models have been proposed to classify sleep stages by automatic learning features and temporal context information. These temporal context features come from the intra-epoch temporal features, which represent the overall morphology of an epoch, and temporal features of adjacent epochs and long epochs, which represent the influence between epochs. However, most existing methods do not fully use the complementarity of the three-level temporal features, resulting in incomplete extracted temporal features. To solve this problem, we propose a multilevel temporal context network (MLTCN) to learn the temporal features from intra-epoch, adjacent epochs, and long epochs, which utilizes the complete temporal features to improve classification accuracy. We evaluate the performance of the proposed model on the Sleep-EDF datasets published in 2013 and 2018. The experimental results show that our MLTCN can achieve an overall accuracy of 84.2% and a kappa coefficient of 0.78 on the Sleep-EDF-2013 dataset. On the larger Sleep-EDF-2018 dataset, the overall accuracy is 81.0%, and a kappa coefficient is 0.74. Our model can better assist sleep experts in diagnosing sleep disorders.

Hybrid Sleep Stage Classification for Clinical Practices across Different Polysomnography Systems Using Frontal EEG

Automatic bio-signal processing and scoring have been a popular topic in recent years. This includes sleep stage classification, which is time-consuming when carried out by hand. Multiple sleep stage classification has been proposed in recent years. While effective, most of these processes are trained and validated against a singular set of data in uniformed pre-processing, whilst in a clinical environment, polysomnography (PSG) may come from different PSG systems that use different signal processing methods. In this study, we present a generalized sleep stage classification method that uses power spectra and entropy. To test its generality, we first trained our system using a uniform dataset and then validated it against another dataset with PSGs from different PSG systems. We found that the system achieved an accuracy of 0.80 and that it is highly consistent across most PSG records. A few samples of NREM3 sleep were classified poorly, and further inspection showed that these samples lost crucial NREM3 features due to aggressive filtering. This implies that the system’s effectiveness can be evaluated by human knowledge. Overall, our classification system shows consistent performance against PSG records that have been collected from different PSG systems, which gives it high potential in a clinical environment.

Nonlinear Phase Synchronization Analysis of EEG Signals in Amnesic Mild Cognitive Impairment with Type 2 Diabetes Mellitus

Studying the nonlinear synchronization of electroencephalogram (EEG) in type 2 diabetic mellitus (T2DM) to find the EEG characteristics related to cognitive impairment is beneficial to the early prevention and diagnosis of mild cognitive impairment. Correlation between probabilities of recurrence (CPR) is a nonlinear phase synchronization method based on recurrence and recurrence probability, which had shown its superiority in detecting epilepsy. In this study, CPR method was used for the first time to analyze the synchronization of eye-closed resting EEG signals with T2DM. The 27 participants were divided into amnesic mild cognitive impairment (aMCI) group (17 case) and control group (10 cases with age and education matched). The CPR values in two groups were statistically analyzed by Mann-Whitney U test, and the correlation between EEG synchronization and cognitive function was studied by Spearman's correlation. The results showed that aMCI group had lower CPR values at each electrode pair than control group, and two groups had decreased CPR values with the increase of the spatial distance of the electrode pair in inter hemispheric. The CPR values were significantly different in frontal, parietal and temporal regions in intra hemispheric between two groups. The CPR values of C3-F7, F4-C4 and FP2-T6 were significantly positively correlated with the MOCA values. This study showed that the synchronization values of EEG signals obtained by the CPR method were significantly different between aMCI and control group, and they were the EEG characteristics associated with cognitive impairment in T2DM.

Children ASD Evaluation Through Joint Analysis of EEG and Eye-Tracking Recordings With Graph Convolution Network

Recent advances in neuroscience indicate that analysis of bio-signals such as rest state electroencephalogram (EEG) and eye-tracking data can provide more reliable evaluation of children autism spectrum disorder (ASD) than traditional methods of behavior measurement relying on scales do. However, the effectiveness of the new approaches still lags behind the increasing requirement in clinical or educational practices as the “bio-marker” information carried by the bio-signal of a single-modality is likely insufficient or distorted. This study proposes an approach to joint analysis of EEG and eye-tracking for children ASD evaluation. The approach focuses on deep fusion of the features in two modalities as no explicit correlations between the original bio-signals are available, which also limits the performance of existing methods along this direction. First, the synchronization measures, information entropy, and time-frequency features of the multi-channel EEG are derived. Then a random forest applies to the eye-tracking recordings of the same subjects to single out the most significant features. A graph convolutional network (GCN) model then naturally fuses the two group of features to differentiate the children with ASD from the typically developed (TD) subjects. Experiments have been carried out on the two types of the bio-signals collected from 42 children (21 ASD and 21 TD subjects, 3–6 years old). The results indicate that (1) the proposed approach can achieve an accuracy of 95% in ASD detection, and (2) strong correlations exist between the two bio-signals collected even asynchronously, in particular the EEG synchronization against the face related/joint attentions in terms of covariance.

A CNN identified by reinforcement learning-based optimization framework for EEG-based state evaluation

Objective.Electroencephalogram (EEG) data, as a kind of complex time-series, is one of the most widely-used information measurements for evaluating human psychophysiological states. Recently, numerous works applied deep learning techniques, especially the convolutional neural network (CNN), into EEG-based research. The design of the hyper-parameters of the CNN model has a great influence on the performance of the model. Therefore, automatically designing these hyper-parameters can save the time and labor of experts. This leads to the appearance of the neural architecture search technique. In this paper, we propose a reinforcement learning (RL)-based step-by-step framework to efficiently search for CNN models.Approach.Specifically, the deep Q network in RL is first used to determine the depth of convolutional layers and the connection modes among layers. Then particle swarm optimization algorithm is used to fine-tune the number and size of convolution kernels. Through this step-by-step strategy, the search space can be narrowed in each step for saving the overall time cost. This framework is employed for both EEG-based sleep stage classification and driver drowsiness evaluation tasks.Main results.The results show that compared with state-of-the-art methods, the high-performance CNN models identified by the proposed optimization framework, can achieve high overall accuracy and better root mean squared error in the two tasks.Significance.Therefore, the proposed optimization framework has a great potential to provide high-performance results for other kinds of classification and prediction tasks. In this way, it can greatly save researchers' time cost and promote broader applications of CNNs.