Brain analysis to approach human muscles synergy using deep learning

Brain signals and muscle movements have been analyzed using electroencephalogram (EEG) data in several studies. EEG signals contain a lot of noise, such as electromyographic (EMG) waves. Further studies have been done to improve the quality of the results, though it is thought that the combination of these two signals can lead to a significant improvement in the synergistic analysis of muscle movements and muscle connections. Using graph theory, this study examined the interaction of EMG and EEG signals during hand movement and estimated the synergy between muscle and brain signals. Mapping of the brain diagram was also developed to reconstruct the muscle signals from the muscle connections in the brain diagram. The proposed method included noise removal from EEG and EMG signals, graph feature analysis from EEG, and synergy calculation from EMG. Two methods were used to estimate synergy. In the first method, after calculating the brain connections, the features of the communication graph were extracted and then synergy estimating was made with neural networks. In the second method, a convolutional network created a transition from the matrix of brain connections to the synergistic EMG signal. This study reached the high correlation values of 99.8% and maximum MSE error of 0.0084. Compared to other graph-based methods, this method based on regression analysis had a very significant performance. This research can lead to the improvement of rehabilitation methods and brain-computer interfaces.

EEG-based fatigue state evaluation by combining complex network and frequency-spatial features

BACKGROUND

The proportion of traffic accidents caused by fatigue driving is increasing year by year, which has aroused wide concerns for researchers. In order to rapidly and accurately detect drivers' fatigue, this paper proposed an electroencephalogram (EEG)-based fatigue state evaluation method by combining complex network and frequency-spatial features.

NEW METHOD

First, this paper constructed a complex network model based on the relative wavelet entropy to characterize the correlation strength information between channels. Then, the differential entropy and symmetry quotient were respectively calculated to extract frequency and spatial features. Then, the brain heat map combined the complex network and frequency-spatial features with different dimensions together as the fusion features. Finally, a convolutional neural network-long short-term memory (CNN-LSTM) neural network was used to evaluate the three-class fatigue states of the EEG data in the Shanghai Jiao Tong University (SJTU) Emotion EEG Dataset (SEED)-VIG dataset, and it was validated on the dataset on the Mendeley Data website.

RESULTS

The experimental results of SEED-VIG dataset show that the average classification accuracy of three-class fatigue states, namely, awake, tired and drowsy, reaches 96.57 %. The average classification accuracy on the dataset on the Mendeley Data website reaches 99.23 %.

COMPARISON WITH EXISTING METHODS

This method has a best evaluation performance compared with the state-of-the-art methods for the three-class fatigue states recognition.

CONCLUSIONS

The experiment results validated the feasibility of the fatigue state evaluation method based on the correlations between channels and the frequency-spatial features, which is of great significance for developing a driver fatigue detection system.

Neuroimaging features for cognitive fatigue and its recovery with VR intervention: An EEG microstates analysis

INTRODUCTION

Cognitive fatigue is mainly caused by enduring mental stress or monotonous work, impairing cognitive and physical performance. Natural scene exposure is a promising intervention for relieving cognitive fatigue, but the efficacy of virtual reality (VR) simulated natural scene exposure is unclear. We aimed to investigate the effect of VR natural scene on cognitive fatigue and further explored its underlying neurophysiological alterations with electroencephalogram (EEG) microstates analysis.

METHODS

Ten participants performed a 20-minute 1-back task before and after VR intervention while EEG was recorded (pre-task, post-task). Performance was measured with mean accuracy rate (MAR) and mean reaction time (MRT) of the continuous 1-back task. VR simulation of the Canal Town scene was utilized to alleviate cognitive fatigue caused by 1-back tasks. Four resting-state phases were identified: beginning, pre, post, and end phases.

RESULTS

Post-task had a higher MAR and a lower MRT than pre-task. For pre-task, MAR was negatively correlated with trials, while MRT was positively correlated with trials. Four EEG microstates classes (A-D) were identified, and their temporal parameters (mean duration, time coverage and occurrence) and transition probabilities were calculated. After intervention, mean duration and time coverage of class B decreased, all parameters of class C increased, while all parameters of class D decreased. Transition probabilities between classes B and D decreased but increased between classes A and C.

CONCLUSION

VR simulation of Canal Town scene is a potentially effective method to alleviate cognitive fatigue. Microstate is an electrophysiological trait characteristic of cognitive fatigue and might be used to indicate the effect of VR intervention.

A hybrid critical channels and optimal feature subset selection framework for EEG fatigue recognition

Fatigue driving is one of the potential factors threatening road safety, and monitoring drivers' mental state through electroencephalography (EEG) can effectively prevent such risks. In this paper, a new model, DE-GFRJMCMC, is proposed for selecting critical channels and optimal feature subsets from EEG data to improve the accuracy of fatigue driving recognition. The model is validated on the SEED-VIG dataset. The model first selects critical EEG channels using the Differential Evolution (DE) algorithm, extracting important electrode channel information to enhance recognition accuracy. These electrode channels are used to construct a Functional Brain Network (FBN), from which the topological feature set is extracted. Empirical Mode Decomposition (EMD) is then applied to extract the intrinsic mode components as network nodes, thereby reducing the influence of the number of electrode channels on the brain functional network. The topological features extracted from these components form the suboptimal feature set. To minimize redundant information, we propose an improved Reversible Jump Markov Chain Monte Carlo (RJMCMC) algorithm for selecting the optimal feature subset, ensuring both the efficiency and accuracy of fatigue recognition. The optimal feature subsets were input into various classifiers, and the results showed that the K-Nearest Neighbor (KNN)-based classifier achieved the highest recognition accuracy of 96.11% ± 0.43%, demonstrating the method's stability and robustness. Compared to similar studies, this model shows superior performance in fatigue driving recognition, which is of significant value for research on fatigue driving detection and prevention.

A portable EEG signal acquisition system and a limited-electrode channel classification network for SSVEP

Brain-computer interfaces (BCIs) have garnered significant research attention, yet their complexity has hindered widespread adoption in daily life. Most current electroencephalography (EEG) systems rely on wet electrodes and numerous electrodes to enhance signal quality, making them impractical for everyday use. Portable and wearable devices offer a promising solution, but the limited number of electrodes in specific regions can lead to missing channels and reduced BCI performance. To overcome these challenges and enable better integration of BCI systems with external devices, this study developed an EEG signal acquisition platform (Gaitech BCI) based on the Robot Operating System (ROS) using a 10-channel dry electrode EEG device. Additionally, a multi-scale channel attention selection network based on the Squeeze-and-Excitation (SE) module (SEMSCS) is proposed to improve the classification performance of portable BCI devices with limited channels. Steady-state visual evoked potential (SSVEP) data were collected using the developed BCI system to evaluate both the system and network performance. Offline data from ten subjects were analyzed using within-subject and cross-subject experiments, along with ablation studies. The results demonstrated that the SEMSCS model achieved better classification performance than the comparative reference model, even with a limited number of channels. Additionally, the implementation of online experiments offers a rational solution for controlling external devices via BCI.

Causal brain network analysis of driving fatigue based on generalized orthogonalized partially directed coherence

Driving fatigue is a serious threat to driving safety. Therefore, it is of great significance to accurately detect driving fatigue. In this study, the generalized orthogonal partial directed coherence (gOPDC) algorithm, which measures the time-frequency domain interaction of electroencephalogram (EEG) signals, was used to accurately estimate the connectivity between cortical channels. The causal brain network of driver continuous driving is constructed. The results show that the clustering coefficient and global efficiency tend to decrease with the increase in driving time. Causal information flow in the left prefrontal, parietal, occipital regions and the right posterior frontal region increased significantly when subjects transitioned from awake to fatigued, while causal information flow in the right prefrontal, parietal, occipital regions and the left posterior frontal region decreased mutually significantly. Compared with the traditional driving fatigue algorithm, the accuracy of the method used in this paper is higher than the traditional methods.

Clustering for mitigating subject variability in driving fatigue classification using electroencephalography source-space functional connectivity features

Objective.While Electroencephalography (EEG)-based driver fatigue state classification models have demonstrated effectiveness, their real-world application remains uncertain. The substantial variability in EEG signals among individuals poses a challenge in developing a universal model, often necessitating retraining with the introduction of new subjects. However, obtaining sufficient data for retraining, especially fatigue data for new subjects, is impractical in real-world settings.Approach.In response to these challenges, this paper introduces a hybrid solution for fatigue detection that combines clustering with classification. Unsupervised clustering groups subjects based on their EEG functional connectivity (FC) in an alert state, and classification models are subsequently applied to each cluster for predicting alert and fatigue states.Main results. Results indicate that classification on clusters achieves higher accuracy than scenarios without clustering, suggesting successful grouping of subjects with similar FC characteristics through clustering, thereby enhancing the classification process.Significance.Furthermore, the proposed hybrid method ensures a practical and realistic retraining process, improving the adaptability and effectiveness of the fatigue detection system in real-world applications.

Real-time EEG-based detection of driving fatigue using a novel semi-dry electrode with self-replenishment of conductive fluid

A novel semi-dry electrode that can realize self-replenishment of conductive liquid is proposed in this study. Driving fatigue is detected by extracting the refined composite multiscale fluctuation dispersion entropy (RCMFDE) features in electroencephalogram (EEG) signals collected by this electrode. The results show that the new semi-dry electrode can automatically complete the conductive fluid supplement according to its own humidity conditions, which not only notably improves the effective working time, but also significantly reduces the skin impedance. By comparing with the classical entropy algorithms, the computational speed and the stability of the RCMFDE method are Substantially enhanced.

EEG spatial inter-channel connectivity analysis: A GCN-based dual stream approach to distinguish mental fatigue status

Mental fatigue is defined as a decline in the ability and efficiency of mental activities. A lot of research suggests that the transition from alertness to fatigue is accompanied by alterations in correlation patterns among various brain regions. However, conventional methods for detecting mental fatigue seldom emphases inter-channel connectivity in the spatial domain. To fill this gap, this paper explores the spatial inter-channel connectivity in alertness and fatigue, employing spectral graph convolutional networks (GCN) for mental fatigue detection. We utilized Pearson correlation coefficients (PCC) to establish temporal connections and magnitude-squared coherence (MSC) for spectral connections. Topological features of the brain network were then analysed. To enhance the learning of spatial inter-channel connectivity, a dual-graph strategy transforms edge features into node features, serving as inputs to the spectral GCN. By simultaneously learning PCC and MSC features, the model results indicate significant differences in some brain network characteristics between alert and fatigue states. It confirms that the synchronicity of brain operations differs in the alert state compared to mental fatigue, and indicates that fatigue states can influence correlation patterns among different brain regions. Our approach is evaluated on a self-designed experimental dataset containing 7 subjects, demonstrating a classification accuracy of 89.59 % in group-level experiments and 95.24 % at the subject level. Additionally, on the public dataset SEED-VIG containing 23 subjects, our method achieves an accuracy of 86.58 %. In summary, this paper proposes a neural network approach based on a dynamic functional connectivity network. The network integrates both temporal and spectral connections with the goal of simultaneously learning spatial inter-channel connectivity in time and frequency domains. This effectively accomplishes fatigue state detection, highlighting that fatigue significantly influences correlations among different brain regions.

Objective assessment of cognitive fatigue: a bibliometric analysis

Aim

The objective of this study was to gain insight into the nature of cognitive fatigue and to identify future trends of objective assessment techniques in this field.

Methods

One thousand and eighty-five articles were retrieved from the Web of Science Core Collection database. R version 4.3.1, VOSviewer 1.6.20, CiteSpace 6.2.R4, and Microsoft Excel 2019 were used to perform the analysis.

Results

A total of 704 institutes from 56 countries participated in the relevant research, while the People's Republic of China contributed 126 articles and was the leading country. The most productive institute was the University of Gothenburg. Johansson Birgitta from the University of Gothenburg has posted the most articles (n = 13). The PLOS ONE published most papers (n = 38). The Neurosciences covered the most citations (n = 1,094). A total of 3,116 keywords were extracted and those with high frequency were mental fatigue, performance, quality-of-life, etc. Keywords mapping analysis indicated that cognitive fatigue caused by continuous work and traumatic brain injury, as well as its rehabilitation, have become the current research trend. The most co-cited literature was published in Sports Medicine. The strongest citation burst was related to electroencephalogram (EEG) event-related potential and spectral power analysis.

Conclusion

Publication information of related literature on the objective assessment of cognitive fatigue from 2007 to 2024 was summarized, including country and institute of origin, authors, and published journal, offering the current hotspots and novel directions in this field.

Learning Interpretable Brain Functional Connectivity via Self-Supervised Triplet Network With Depth-Wise Attention

Brain functional connectivity has been widely explored to reveal the functional interaction dynamics between the brain regions. However, conventional connectivity measures rely on deterministic models demanding application-specific empirical analysis, while deep learning approaches focus on finding discriminative features for state classification, having limited capability to capture the interpretable connectivity characteristics. To address the challenges, this study proposes a self-supervised triplet network with depth-wise attention (TripletNet-DA) to generate the functional connectivity: 1) TripletNet-DA firstly utilizes channel-wise transformations for temporal data augmentation, where the correlated & uncorrelated sample pairs are constructed for self-supervised training, 2) Channel encoder is designed with a convolution network to extract the deep features, while similarity estimator is employed to generate the similarity pairs and the functional connectivity representations, 3) TripletNet-DA applies Triplet loss with anchor-negative similarity penalty for model training, where the similarities of uncorrelated sample pairs are minimized to enhance model's learning capability. Experimental results on pathological EEG datasets (Autism Spectrum Disorder, Major Depressive Disorder) indicate that 1) TripletNet-DA demonstrates superiority in both ASD discrimination and MDD classification than the state-of-the-art counterparts, where the connectivity features in beta & gamma bands have respectively achieved the accuracy of 97.05%, 98.32% for ASD discrimination, 89.88%, 91.80% for MDD classification in the eyes-closed condition and 90.90%, 92.26% in the eyes-open condition, 2) TripletNet-DA enables to uncover significant differences of functional connectivity between ASD EEG and TD ones, and the prominent connectivity links are in accordance with the empirical findings, thus providing potential biomarkers for clinical ASD analysis.

Driving Fatigue Detection Based on Hybrid Electroencephalography and Eye Tracking

EEG-based unimodal method has demonstrated significant success in the detection of driving fatigue. Nonetheless, data from a single modality might be not sufficient to optimize fatigue detection due to incomplete information. To address this limitation and enhance the performance of driving fatigue detection, a novel multimodal architecture combining hybrid electroencephalograph (EEG) and eye tracking data was proposed in this work. Specifically, the EEG and eye tracking data were separately input into encoders, generating two one-dimensional (1D) features. Subsequently, these 1D features were fed into a cross-modal predictive alignment module to improve fusion efficiency and two 1D attention modules to enhance feature representation. Furthermore, the fused features were recognized by a linear classifier. To evaluate the effectiveness of the proposed multimodal method, comprehensive validation tasks were conducted, including intra-session, cross-session, and cross-subject evaluations. In the intra-session task, the proposed architecture achieves an exceptional average accuracy of 99.93%. Moreover, in the cross-session task, our method demonstrates an average accuracy of 88.67%, surpassing the performance of EEG-only approach by 8.52%, eye tracking-only method by 5.92%, multimodal deep canonical correlation analysis (DCCA) technique by 0.42%, and multimodal deep generalized canonical correlation analysis (DGCCA) approach by 0.84%. Similarly, in the cross-subject task, the proposed approach achieves an average accuracy of 78.19%, outperforming EEG-only method by 5.87%, eye tracking-only approach by 4.21%, DCCA method by 0.55%, and DGCCA approach by 0.44%. The experimental results conclusively illustrate the superior effectiveness of the proposed method compared to both single modality approaches and canonical correlation analysis-based multimodal methods.

Attention-based cross-frequency graph convolutional network for driver fatigue estimation

Fatigue driving significantly contributes to global vehicle accidents and fatalities, making driver fatigue level estimation crucial. Electroencephalography (EEG) is a proven reliable predictor of brain states. With Deep Learning (DL) advancements, brain state estimation algorithms have improved significantly. Nonetheless, EEG's multi-domain nature and the intricate spatial-temporal-frequency correlations among EEG channels present challenges in developing precise DL models. In this work, we introduce an innovative Attention-based Cross-Frequency Graph Convolutional Network (ACF-GCN) for estimating drivers' reaction times using EEG signals from theta, alpha, and beta bands. This method utilizes a multi-head attention mechanism to detect long-range dependencies between EEG channels across frequencies. Concurrently, the transformer's encoder module learns node-level feature maps from the attention-score matrix. Subsequently, the Graph Convolutional Network (GCN) integrates this matrix with feature maps to estimate driver reaction time. Our validation on a publicly available dataset shows that ACF-GCN outperforms several state-of-the-art methods. We also explore the brain dynamics within the cross-frequency attention-score matrix, identifying theta and alpha bands as key influencers in fatigue estimating performance. The ACF-GCN method advances brain state estimation and provides insights into the brain dynamics underlying multi-channel EEG signals.

Designing a practical fatigue detection system: A review on recent developments and challenges

INTRODUCTION

Fatigue is considered to have a life-threatening effect on human health and it has been an active field of research in different sectors. Deploying wearable physiological sensors helps to detect the level of fatigue objectively without any concern of bias in subjective assessment and interfering with work.

METHODS

This paper provides an in-depth review of fatigue detection approaches using physiological signals to pinpoint their main achievements, identify research gaps, and recommend avenues for future research. The review results are presented under three headings, including: signal modality, experimental environments, and fatigue detection models. Fatigue detection studies are first divided based on signal modality into uni-modal and multi-modal approaches. Then, the experimental environments utilized for fatigue data collection are critically analyzed. At the end, the machine learning models used for the classification of fatigue state are reviewed.

PRACTICAL APPLICATIONS

The directions for future research are provided based on critical analysis of past studies. Finally, the challenges of objective fatigue detection in the real-world scenario are discussed.

The architecture of functional brain network modulated by driving under train running noise exposure

A noisy environment can considerably impact drivers' attention and fatigue, endangering driving safety. Consequently, this study designed a simulated driving experimental scenario to analyse the effects of noise generated during urban rail transit train operation on drivers' functional brain networks. The experiment recruited 16 participants, and the simulated driving scenario was conducted at noise levels of 50, 60, 70, and 80 dB. Functional connectivity between all electrode pairs across various frequency bands was evaluated using the weighted phase lag index (WPLI), and a brain network based on this was constructed. Graph theoretic analysis employed network global efficiency, degree, and clustering coefficient as metrics. Significant increases in the WPLI values of theta and alpha frequency bands were observed in high noise environments (70 dB, 80 dB), as well as enhanced brain synchronisation. Furthermore, concerning the topological metrics of brain networks, it was observed that the global efficiency of brain networks in theta and alpha frequency ranges, as well as the node degree and clustering coefficients, experienced substantial growth in high noise environments (70 dB, 80 dB) as opposed to 50 dB and 60 dB. This finding indicates that high-noise environments impact the reorganisation of functional brain networks, leading to a preference for network structures with improved global efficiency. Such findings may improve our understanding of the neural mechanisms of driving under noise exposure, and thus potentially reduce road accidents to some extent.

Augmented Recognition of Distracted Driving State Based on Electrophysiological Analysis of Brain Network

In this study, we propose an electrophysiological analysis-based brain network method for the augmented recognition of different types of distractions during driving. Driver distractions, such as cognitive processing and visual disruptions during driving, lead to distinct alterations in the electroencephalogram (EEG) signals and the extracted brain networks. We designed and conducted a simulated experiment comprising 4 distracted driving subtasks. Three connectivity indices, including both linear and nonlinear synchronization measures, were chosen to construct the brain network. By computing connectivity strengths and topological features, we explored the potential relationship between brain network configurations and states of driver distraction. Statistical analysis of network features indicates substantial differences between normal and distracted states, suggesting a reconfiguration of the brain network under distracted conditions. Different brain network features and their combinations are fed into varied machine learning classifiers to recognize the distracted driving states. The results indicate that XGBoost demonstrates superior adaptability, outperforming other classifiers across all selected network features. For individual networks, features constructed using synchronization likelihood (SL) achieved the highest accuracy in distinguishing between cognitive and visual distraction. The optimal feature set from 3 network combinations achieves an accuracy of 95.1% for binary classification and 88.3% for ternary classification of normal, cognitively distracted, and visually distracted driving states. The proposed method could accomplish the augmented recognition of distracted driving states and may serve as a valuable tool for further optimizing driver assistance systems with distraction control strategies, as well as a reference for future research on the brain-computer interface in autonomous driving.

Integrating EEG and Ensemble Learning for Accurate Grading and Quantification of Generalized Anxiety Disorder: A Novel Diagnostic Approach

Current assessments for generalized anxiety disorder (GAD) are often subjective and do not rely on a standardized measure to evaluate the GAD across its severity levels. The lack of objective and multi-level quantitative diagnostic criteria poses as a significant challenge for individualized treatment strategies. To address this need, this study aims to establish a GAD grading and quantification diagnostic model by integrating an electroencephalogram (EEG) and ensemble learning. In this context, a total of 39 normal subjects and 80 GAD patients were recruited and divided into four groups: normal control, mild GAD, moderate GAD, and severe GAD. Ten minutes resting state EEG data were collected for every subject. Functional connectivity features were extracted from each EEG segment with different time windows. Then, ensemble learning was employed for GAD classification studies and brain mechanism analysis. Hence, the results showed that the Catboost model with a 10 s time window achieved an impressive 98.1% accuracy for four-level classification. Particularly, it was found that those functional connections situated between the frontal and temporal lobes were significantly more abundant than in other regions, with the beta rhythm being the most prominent. The analysis framework and findings of this study provide substantial evidence for the applications of artificial intelligence in the clinical diagnosis of GAD.

Driving fatigue detection based on brain source activity and ARMA model

Fatigue among drivers is a significant issue in society, and according to organizational reports, it substantially contributes to accidents. So accurate fatigue detection in drivers plays a crucial role in reducing the number of people fatalities or injured resulting from accidents. Several methods are proposed for fatigue driver recognition among which electroencephalography (EEG) is one. This paper proposed a method for fatigue recognition by EEG signals with extracted features from source and sensor spaces. The proposed method starts with preprocessing by applying filtering and artifact rejection. Then source localization methods are applied to EEG signals for active source extraction. A multivariate autoregressive (MVAR) model is fitted to selected sources, and a dual Kalman filter is applied to estimate the source activity and their relationships. Then multivariate autoregressive moving average (ARMA) is fitted between EEG and source activity signals. Features are extracted from model parameters, source relationship matrix, and wavelet transform of EEG and source activity signals. The novelty of this approach is the use of ARMA model between source activities (as input) and EEG signals (as output) and feature extraction from source relations. Relevant features are selected using a combination of RelifF and neighborhood component analysis (NCA) methods. Three classifiers, namely k-nearest neighbor (KNN), support vector machine (SVM), and naive Bayesian (NB) classifiers, are employed to classify drivers. To improve performance, the final label for fatigue detection is calculated by combining these classifiers using the voting method. The results demonstrate that the proposed method accurately recognizes and classifies fatigued drivers with the ensemble classifiers in comparison with other methods.

Resting-state EEG recorded with gel-based vs. consumer dry electrodes: spectral characteristics and across-device correlations

Introduction

Recordings of electroencephalographic (EEG) rhythms and their analyses have been instrumental in basic neuroscience, clinical diagnostics, and the field of brain-computer interfaces (BCIs). While in the past such measurements have been conducted mostly in laboratory settings, recent advancements in dry electrode technology pave way to a broader range of consumer and medical application because of their greater convenience compared to gel-based electrodes.

Methods

Here we conducted resting-state EEG recordings in two groups of healthy participants using three dry-electrode devices, the PSBD Headband, the PSBD Headphones and the Muse Headband, and one standard gel electrode-based system, the NVX. We examined signal quality for various spatial and spectral ranges which are essential for cognitive monitoring and consumer applications.

Results

Distinctive characteristics of signal quality were found, with the PSBD Headband showing sensitivity in low-frequency ranges and replicating the modulations of delta, theta and alpha power corresponding to the eyes-open and eyes-closed conditions, and the NVX system performing well in capturing high-frequency oscillations. The PSBD Headphones were more prone to low-frequency artifacts compared to the PSBD Headband, yet recorded modulations in the alpha power and had a strong alignment with the NVX at the higher EEG frequencies. The Muse Headband had several limitations in signal quality.

Discussion

We suggest that while dry-electrode technology appears to be appropriate for the EEG rhythm-based applications, the potential benefits of these technologies in terms of ease of use and accessibility should be carefully weighed against the capacity of each given system.

ST-SCGNN: A Spatio-Temporal Self-Constructing Graph Neural Network for Cross-Subject EEG-Based Emotion Recognition and Consciousness Detection

In this paper, a novel spatio-temporal self-constructing graph neural network (ST-SCGNN) is proposed for cross-subject emotion recognition and consciousness detection. For spatio-temporal feature generation, activation and connection pattern features are first extracted and then combined to leverage their complementary emotion-related information. Next, a self-constructing graph neural network with a spatio-temporal model is presented. Specifically, the graph structure of the neural network is dynamically updated by the self-constructing module of the input signal. Experiments based on the SEED and SEED-IV datasets showed that the model achieved average accuracies of 85.90% and 76.37%, respectively. Both values exceed the state-of-the-art metrics with the same protocol. In clinical besides, patients with disorders of consciousness (DOC) suffer severe brain injuries, and sufficient training data for EEG-based emotion recognition cannot be collected. Our proposed ST-SCGNN method for cross-subject emotion recognition was first attempted in training in ten healthy subjects and testing in eight patients with DOC. We found that two patients obtained accuracies significantly higher than chance level and showed similar neural patterns with healthy subjects. Covert consciousness and emotion-related abilities were thus demonstrated in these two patients. Our proposed ST-SCGNN for cross-subject emotion recognition could be a promising tool for consciousness detection in DOC patients.

A Connectivity-Aware Graph Neural Network for Real-Time Drowsiness Classification

Drowsy driving is one of the primary causes of driving fatalities. Electroencephalography (EEG), a method for detecting drowsiness directly from brain activity, has been widely used for detecting driver drowsiness in real-time. Recent studies have revealed the great potential of using brain connectivity graphs constructed based on EEG data for drowsy state predictions. However, traditional brain connectivity networks are irrelevant to the downstream prediction tasks. This article proposes a connectivity-aware graph neural network (CAGNN) using a self-attention mechanism that can generate task-relevant connectivity networks via end-to-end training. Our method achieved an accuracy of 72.6% and outperformed other convolutional neural networks (CNNs) and graph generation methods based on a drowsy driving dataset. In addition, we introduced a squeeze-and-excitation (SE) block to capture important features and demonstrated that the SE attention score can reveal the most important feature band. We compared our generated connectivity graphs in the drowsy and alert states and found drowsiness connectivity patterns, including significantly reduced occipital connectivity and interregional connectivity. Additionally, we performed a post hoc interpretability analysis and found that our method could identify drowsiness features such as alpha spindles. Our code is available online at https://github.com/ALEX95GOGO/CAGNN.

Activation network improves spatiotemporal modelling of human brain communication processes

Dynamic functional networks (DFN) have considerably advanced modelling of the brain communication processes. The prevailing implementation capitalizes on the system and network-level correlations between time series. However, this approach does not account for the continuous impact of non-dynamic dependencies within the statistical correlation, resulting in relatively stable connectivity patterns of DFN over time with limited sensitivity for communication dynamic between brain regions. Here, we propose an activation network framework based on the activity of functional connectivity (AFC) to extract new types of connectivity patterns during brain communication process. The AFC captures potential time-specific fluctuations associated with the brain communication processes by eliminating the non-dynamic dependency of the statistical correlation. In a simulation study, the positive correlation (r=0.966,p<0.001) between the extracted dynamic dependencies and the simulated "ground truth" validates the method's dynamic detection capability. Applying to autism spectrum disorders (ASD) and COVID-19 datasets, the proposed activation network extracts richer topological reorganization information, which is largely invisible to the DFN. Detailed, the activation network exhibits significant inter-regional connections between function-specific subnetworks and reconfigures more efficiently in the temporal dimension. Furthermore, the DFN fails to distinguish between patients and healthy controls. However, the proposed method reveals a significant decrease (p<0.05) in brain information processing abilities in patients. Finally, combining two types of networks successfully classifies ASD (83.636 % ± 11.969 %,mean±std) and COVID-19 (67.333 % ± 5.398 %). These findings suggest the proposed method could be a potential analytic framework for elucidating the neural mechanism of brain dynamics.

Self-Regulation Phenomenon Emerged During Prolonged Fatigue Driving: An EEG Connectivity Study

Driving fatigue is a common experience for most drivers and can reduce human cognition and judgment abilities. Previous studies have exhibited a phenomenon of the non-monotonically varying indicators (both behavioral and neurophysiological) for driving fatigue evaluation but paid little attention to this phenomenon. Herein, we propose a hypothesis that the non-monotonically varying phenomenon is caused by the self-regulation of brain activity, which is defined as the fatigue self-regulation (FSR) phenomenon. In this study, a 90-min simulated driving task was performed on 26 healthy university students. EEG data and reaction time (RT) were synchronously recorded during the whole task. To identify the FSR phenomenon, a data-driven criterion was proposed based on clustering analysis of individual behavioral data and the FSR group was determined as having non-monotonic increase trend of RT and the drops of RT during prolonged driving were more than two levels among the total five levels. The subjects were then divided into two groups: the FSR group and the non-FSR group. Quantitative comparative analysis showed significant differences in behavioral performance, functional connectivity, network characteristics, and classification performance between the FSR and non-FSR groups. Specifically, the behavioral performance exhibited apparent non-monotonic development trend: increasing-decreasing-increasing. Moreover, network characteristics presented similar self-regulated development trends. Our study provides a new insight for revealing the complex neural mechanisms of driving fatigue, which may promote the development of practical techniques for automatic detection method and mitigation strategy.

Vigilance estimation using truncated l1 distance kernel-based sparse representation regression with physiological signals

BACKGROUND

With a large number of accidents caused by the decline in the vigilance of operators, finding effective automatic vigilance monitoring methods is a work of great significance in recent years. Based on physiological signals and machine learning algorithms, researchers have opened up a path for objective vigilance estimation.

METHODS

Sparse representation (SR)-based recognition algorithms with excellent performance and simple models are very promising approaches in this field. This paper aims to study the adaptability and performance improvement of truncated l1 distance (TL1) kernel on SR-based algorithm in the context of physiological signal vigilance estimation. Compared with the traditional radial basis function (RBF), the TL1 kernel has good adaptiveness to nonlinearity and is suitable for the discrimination of complex physiological signals. A recognition framework based on TL1 and SR theory is proposed. Firstly, the inseparable physiological features are mapped to the reproducing kernel Kreĭn space through the infinite-dimensional projection of the TL1 kernel. Then the obtained kernel matrix is converted into the symmetric positive definite matrix according to the eigenspectrum approaches. Finally, the final prediction result is obtained through the sparse representation regression process.

RESULTS

We verified the performance of the proposed framework on the popular SEED-VIG dataset containing physiological signals (electroencephalogram and electrooculogram) associated with vigilance. In the experimental results, the TL1 kernel is superior to the RBF kernel in both performance and kernel parameter stability.

CONCLUSIONS

This demonstrates the effectiveness of the TL1 kernel in distinguishing physiological signals and the excellent vigilance estimation capability of the proposed framework. Moreover, the contribution of our research motivates the development of physiological signal recognition based on kernel methods.

EEG feature selection method based on maximum information coefficient and quantum particle swarm

To reduce the dimensionality of EEG features and improve classification accuracy, we propose an improved hybrid feature selection method for EEG feature selection. First, MIC is used to remove irrelevant features and redundant features to reduce the search space of the second stage. QPSO is then used to optimize the feature in the second stage to obtain the optimal feature subset. Considering that both dimensionality and classification accuracy affect the performance of feature subsets, we design a new fitness function. Moreover, we optimize the parameters of the classifier while optimizing the feature subset to improve the classification accuracy and reduce the running time of the algorithm. Finally, experiments were performed on EEG and UCI datasets and compared with five existing feature selection methods. The results show that the feature subsets obtained by the proposed method have low dimensionality, high classification accuracy, and low computational complexity, which validates the effectiveness of the proposed method.

Self-Attentive Channel-Connectivity Capsule Network for EEG-Based Driving Fatigue Detection

Deep neural networks have recently been successfully extended to EEG-based driving fatigue detection. Nevertheless, most existing models fail to reveal the intrinsic inter-channel relations that are known to be beneficial for EEG-based classification. Additionally, these models require substantial data for training, which is often impractical due to the high cost of data collection. To simultaneously address these two issues, we propose a Self-Attentive Channel-Connectivity Capsule Network (SACC-CapsNet) for EEG-based driving fatigue detection in this paper. SACC-CapsNet starts with a temporal-channel attention module to investigate the critical temporal information and important channels for driving fatigue detection, refining the input EEG signals. Subsequently, the refined EEG data are transformed into a channel covariance matrix to capture the inter-channel relations, followed by selective kernel attention to extract the highly discriminative channel-connectivity features. Finally, a capsule neural network is employed to effectively learn the relationships between connectivity features, which is more suitable for limited data. To confirm the effectiveness of SACC-CapsNet, we collected 24-channel EEG data from 31 subjects (mean age=23.13±2.68 years, male/female=18/13) in a simulated fatigue driving environment. Extensive experiments were conducted with the acquired data, and the comparison results show that our proposed model outperforms state-of-the-art methods. Additionally, the channel covariance matrix learned from SACC-CapsNet reveals that the frontal pole is most informative for detecting driving fatigue, followed by the parietal and central regions. Intriguingly, the temporal-channel attention module can enhance the significance of these critical regions, and the reconstructed channel covariance matrix generated by the decoder network of SACC-CapsNet can effectively preserve valuable information about them.

Evaluation on induction of driver fatigue in driving different traffic condition: An EEG study. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2023;2023:1-4. [PMID: 38082603 DOI: 10.1109/embc40787.2023.10340407] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

Fatigue is the one of major causes of traffic accidents. Causes of driver fatigue can be divided into physical factors such as lack of sleep, long duration of driving, and a large amount of driving maneuvers, and mental factors such as recognition of traffic scenes to need to drive. Among those factors, a lot of studies on fatigue of drivers focused on the lack of sleep, and long duration of driving and induction of fatigue. However, little attention has been paid to the amount of driving maneuvers and recognition of traffic scenes on the induction of fatigue of drivers even both mental and physical factor related to driving. In this study, we created different traffic scenarios in terms of tuning traffic density and/or driving area to control the amount of driving maneuvers and recognition of traffic environment, and we evaluate the theta and alpha band EEG responses which are known to the one of biological index reflecting fatigue to investigate drivers' fatigue by driving in such different traffic condition.

Source-Space Brain Functional Connectivity Features in Electroencephalogram-Based Driver Fatigue Classification

This study examined the brain source space functional connectivity from the electroencephalogram (EEG) activity of 48 participants during a driving simulation experiment where they drove until fatigue developed. Source-space functional connectivity (FC) analysis is a state-of-the-art method for understanding connections between brain regions that may indicate psychological differences. Multi-band FC in the brain source space was constructed using the phased lag index (PLI) method and used as features to train an SVM classification model to classify driver fatigue and alert conditions. With a subset of critical connections in the beta band, a classification accuracy of 93% was achieved. Additionally, the source-space FC feature extractor demonstrated superiority over other methods, such as PSD and sensor-space FC, in classifying fatigue. The results suggested that source-space FC is a discriminative biomarker for detecting driving fatigue.

Granger-Causality-Based Multi-Frequency Band EEG Graph Feature Extraction and Fusion for Emotion Recognition

Graph convolutional neural networks (GCN) have attracted much attention in the task of electroencephalogram (EEG) emotion recognition. However, most features of current GCNs do not take full advantage of the causal connection between the EEG signals in different frequency bands during the process of constructing the adjacency matrix. Based on the causal connectivity between the EEG channels obtained by Granger causality (GC) analysis, this paper proposes a multi-frequency band EEG graph feature extraction and fusion method for EEG emotion recognition. First, the original GC matrices between the EEG signals at each frequency band are calculated via GC analysis, and then they are adaptively converted to asymmetric binary GC matrices through an optimal threshold. Then, a kind of novel GC-based GCN feature (GC-GCN) is constructed by using differential entropy features and the binary GC matrices as the node values and adjacency matrices, respectively. Finally, on the basis of the GC-GCN features, a new multi-frequency band feature fusion method (GC-F-GCN) is proposed, which integrates the graph information of the EEG signals at different frequency bands for the same node. The experimental results demonstrate that the proposed GC-F-GCN method achieves better recognition performance than the state-of-the-art GCN methods, for which average accuracies of 97.91%, 98.46%, and 98.15% were achieved for the arousal, valence, and arousal-valence classifications, respectively.

Driving Fatigue Detection with Three Non-Hair-Bearing EEG Channels and Modified Transformer Model

Driving fatigue is the main cause of traffic accidents, which seriously affects people's life and property safety. Many researchers have applied electroencephalogram (EEG) signals for driving fatigue detection to reduce negative effects. The main challenges are the practicality and accuracy of the EEG-based driving fatigue detection method when it is applied on the real road. In our previous study, we attempted to improve the practicality of fatigue detection based on the proposed non-hair-bearing (NHB) montage with fewer EEG channels, but the recognition accuracy was only 76.47% with the random forest (RF) model. In order to improve the accuracy with NHB montage, this study proposed an improved transformer architecture for one-dimensional feature vector classification based on introducing the Gated Linear Unit (GLU) in the Attention sub-block and Feed-Forward Networks (FFN) sub-block of a transformer, called GLU-Oneformer. Moreover, we constructed an NHB-EEG-based feature set, including the same EEG features (power ratio, approximate entropy, and mutual information (MI)) in our previous study, and the lateralization features of the power ratio and approximate entropy based on the strategy of brain lateralization. The results indicated that our GLU-Oneformer method significantly improved the recognition performance and achieved an accuracy of 86.97%. Our framework demonstrated that the combination of the NHB montage and the proposed GLU-Oneformer model could well support driving fatigue detection.

Asymmetry of Regional Phase Synchrony Cortical Networks Under Cognitive Alertness and Vigilance Decrement States

This study investigates intra-regional connectivity and regional hemispheric asymmetry under two vigilance states: alertness and vigilance decrement. The vigilance states were induced on nine healthy subjects while performing 30 min in-congruent Stroop color-word task (I-SCWT). We measured brain activity using Electroencephalography (EEG) signals with 64-channels. We quantified the regional network connectivity using the phase-locking value (PLV) with graph theory analysis (GTA) and Support Vector Machines (SVM). Results showed that the vigilance decrement state was associated with impaired information processing within the frontal and central regions in delta and theta frequency bands. Meanwhile, the hemispheric asymmetry results showed that the laterality shifted to the right-temporal in delta, right-central, parietal, and left frontal in theta, right-frontal and left-central, temporal and parietal in alpha, and right-parietal and left temporal in beta frequency bands. These findings represent the first demonstration of intra-regional connectivity and hemispheric asymmetry changes as a function of cognitive vigilance states. The overall results showed that vigilance decrement is region and frequency band-specific. Our SVM model achieved the highest classification accuracy of 99.73% in differentiating between the two vigilance states based on the frontal and central connectivity networks measures.

Phase lag index-based graph attention networks for detecting driving fatigue

It is important to understand the changes in the characteristics of the brain network in the state of driving fatigue and to reveal the pattern of functional connectivity between brain regions when fatigue occurs. This paper proposes a method for the detection of driving fatigue based on electroencephalogram (EEG) signals using a phase lag index graph attention network (PLI-GAT). Phase synchronization between EEG signals is a key attribute for establishing communication links among different regions of the brain, and so, the PLI is used to construct a functional brain network reflecting the relationship between EEG signals from different channels. Multi-channel EEG time-frequency features are then modeled as graph data, and the driving fatigue monitoring model is trained using a GAT. Compared with traditional graph neural networks, the GAT applies an aggregation operation to adjacent EEG channel features through the attention mechanism. This enables the adaptive assignment of different neighbor weights, which greatly improves the expressiveness of the graph neural network model. The proposed method is validated on the publicly available SEED-VIG dataset, and the accuracy of fatigue state recognition is found to reach 85.53%. The results show that the functional connectivity among different channels is significantly enhanced in the fatigue state.

Toward practical driving fatigue detection using three frontal EEG channels: a proof-of-concept study

Objective. Although various driving fatigue detection strategies have been introduced, the limited practicability is still an obstacle for the real application of these technologies. This study is based on the newly proposed non-hair-bearing (NHB) method to achieve practical driving fatigue detection with fewer channels from NHB areas and more efficient electroencephalogram (EEG) features.Approach. EEG data were recorded from 20 healthy subjects (15 males, age = 22.2 ± 3.2 years) in a 90 min simulated driving task using a remote wireless cap. Behaviorally, subjects demonstrated a salient fatigue effect, as reflected by a monotonic increase in reaction time. Using a sliding-window approach, we determined the vigilant and fatigued states at individual level to reduce the inter-subject differences in behavioral impairment and brain activity. Multiple EEG features, including power-spectrum density (PSD), functional connectivity (FC), and entropy, were estimated in a pairwise manner, which were set as input for fatigue classification.Main results. Intriguingly, this data-driven approach showed that the best classification performance was achieved using three EEG channel pairs located in the NHB area. The mixed features of the frontal NHB area lead to the high within-subject detection rate of driving fatigue (92.7% ± 0.92%) with satisfactory generalizability for fatigue classification across different subjects (77.13% ± 0.85%). Moreover, we found the most prominent contributing features were PSD of different frequency bands within the frontal NHB area and FC within the frontal NHB area and between frontal and parietal areas.Significance. In summary, the current work provided objective evidence to support the effectiveness of the NHB method and further improved the performance, thereby moving a step forward towards practical driving fatigue detection in real-world scenarios.