Self-Attentive Channel-Connectivity Capsule Network for EEG-Based Driving Fatigue Detection

Feature-aware domain invariant representation learning for EEG motor imagery decoding

Electroencephalography (EEG)-based motor imagery (MI) is extensively utilized in clinical rehabilitation and virtual reality-based movement control. Decoding EEG-based MI signals is challenging because of the inherent spatio-temporal variability of the original signal representation, coupled with a low signal-to-noise ratio (SNR), which impedes the extraction of clean and robust features. To address this issue, we propose a multi-scale spatio-temporal domain-invariant representation learning method, termed MSDI. By decomposing the original signal into spatial and temporal components, the proposed method extracts invariant features at multiple scales from both components. To further constrain the representation to invariant domains, we introduce a feature-aware shift operation that resamples the representation based on its feature statistics and feature measure, thereby projecting the features into a domain-invariant space. We evaluate our proposed method via two publicly available datasets, BNCI2014-001 and BNCI2014-004, demonstrating state-of-the-art performance on both datasets. Furthermore, our method exhibits superior time efficiency and noise resistance.

Cognitive load recognition in simulated flight missions: an EEG study

Cognitive load recognition (CLR) utilizing EEG signals has experienced significant advancement in recent years. However, current load-eliciting paradigms often rely on simplistic cognitive tasks such as arithmetic calculations, failing to adequately replicate real-world scenarios and lacking applicability. This study explores simulated flight missions over time to better reflect operational environments and investigate temporal dynamics of multiple load states. Thirty-six participants were recruited to perform simulated flight tasks with varying cognitive load levels of low, medium, and high. Throughout the experiments, we collected EEG load data from three sessions, pre- and post-task resting-state EEG data, subjective ratings, and objective performance metrics. Then, we employed several deep convolutional neural network (CNN) models, utilizing raw EEG data as model input, to assess cognitive load levels with six classification designs. Key findings from the study include (1) a notable distinction between resting-state and post-fatigue EEG data; (2) superior performance of shallow CNN models compared to more complex ones; and (3) temporal dynamics decline in CLR as the missions progressed. This paper establishes a potential foundation for assessing cognitive states during intricate simulated tasks across different individuals.

A hybrid critical channels and optimal feature subset selection framework for EEG fatigue recognition

Fatigue driving is one of the potential factors threatening road safety, and monitoring drivers' mental state through electroencephalography (EEG) can effectively prevent such risks. In this paper, a new model, DE-GFRJMCMC, is proposed for selecting critical channels and optimal feature subsets from EEG data to improve the accuracy of fatigue driving recognition. The model is validated on the SEED-VIG dataset. The model first selects critical EEG channels using the Differential Evolution (DE) algorithm, extracting important electrode channel information to enhance recognition accuracy. These electrode channels are used to construct a Functional Brain Network (FBN), from which the topological feature set is extracted. Empirical Mode Decomposition (EMD) is then applied to extract the intrinsic mode components as network nodes, thereby reducing the influence of the number of electrode channels on the brain functional network. The topological features extracted from these components form the suboptimal feature set. To minimize redundant information, we propose an improved Reversible Jump Markov Chain Monte Carlo (RJMCMC) algorithm for selecting the optimal feature subset, ensuring both the efficiency and accuracy of fatigue recognition. The optimal feature subsets were input into various classifiers, and the results showed that the K-Nearest Neighbor (KNN)-based classifier achieved the highest recognition accuracy of 96.11% ± 0.43%, demonstrating the method's stability and robustness. Compared to similar studies, this model shows superior performance in fatigue driving recognition, which is of significant value for research on fatigue driving detection and prevention.

A portable EEG signal acquisition system and a limited-electrode channel classification network for SSVEP

Brain-computer interfaces (BCIs) have garnered significant research attention, yet their complexity has hindered widespread adoption in daily life. Most current electroencephalography (EEG) systems rely on wet electrodes and numerous electrodes to enhance signal quality, making them impractical for everyday use. Portable and wearable devices offer a promising solution, but the limited number of electrodes in specific regions can lead to missing channels and reduced BCI performance. To overcome these challenges and enable better integration of BCI systems with external devices, this study developed an EEG signal acquisition platform (Gaitech BCI) based on the Robot Operating System (ROS) using a 10-channel dry electrode EEG device. Additionally, a multi-scale channel attention selection network based on the Squeeze-and-Excitation (SE) module (SEMSCS) is proposed to improve the classification performance of portable BCI devices with limited channels. Steady-state visual evoked potential (SSVEP) data were collected using the developed BCI system to evaluate both the system and network performance. Offline data from ten subjects were analyzed using within-subject and cross-subject experiments, along with ablation studies. The results demonstrated that the SEMSCS model achieved better classification performance than the comparative reference model, even with a limited number of channels. Additionally, the implementation of online experiments offers a rational solution for controlling external devices via BCI.

Real-time EEG-based detection of driving fatigue using a novel semi-dry electrode with self-replenishment of conductive fluid

A novel semi-dry electrode that can realize self-replenishment of conductive liquid is proposed in this study. Driving fatigue is detected by extracting the refined composite multiscale fluctuation dispersion entropy (RCMFDE) features in electroencephalogram (EEG) signals collected by this electrode. The results show that the new semi-dry electrode can automatically complete the conductive fluid supplement according to its own humidity conditions, which not only notably improves the effective working time, but also significantly reduces the skin impedance. By comparing with the classical entropy algorithms, the computational speed and the stability of the RCMFDE method are Substantially enhanced.

Objective assessment of cognitive fatigue: a bibliometric analysis

Aim

The objective of this study was to gain insight into the nature of cognitive fatigue and to identify future trends of objective assessment techniques in this field.

Methods

One thousand and eighty-five articles were retrieved from the Web of Science Core Collection database. R version 4.3.1, VOSviewer 1.6.20, CiteSpace 6.2.R4, and Microsoft Excel 2019 were used to perform the analysis.

Results

A total of 704 institutes from 56 countries participated in the relevant research, while the People's Republic of China contributed 126 articles and was the leading country. The most productive institute was the University of Gothenburg. Johansson Birgitta from the University of Gothenburg has posted the most articles (n = 13). The PLOS ONE published most papers (n = 38). The Neurosciences covered the most citations (n = 1,094). A total of 3,116 keywords were extracted and those with high frequency were mental fatigue, performance, quality-of-life, etc. Keywords mapping analysis indicated that cognitive fatigue caused by continuous work and traumatic brain injury, as well as its rehabilitation, have become the current research trend. The most co-cited literature was published in Sports Medicine. The strongest citation burst was related to electroencephalogram (EEG) event-related potential and spectral power analysis.

Conclusion

Publication information of related literature on the objective assessment of cognitive fatigue from 2007 to 2024 was summarized, including country and institute of origin, authors, and published journal, offering the current hotspots and novel directions in this field.

Driving Fatigue Detection Based on Hybrid Electroencephalography and Eye Tracking

EEG-based unimodal method has demonstrated significant success in the detection of driving fatigue. Nonetheless, data from a single modality might be not sufficient to optimize fatigue detection due to incomplete information. To address this limitation and enhance the performance of driving fatigue detection, a novel multimodal architecture combining hybrid electroencephalograph (EEG) and eye tracking data was proposed in this work. Specifically, the EEG and eye tracking data were separately input into encoders, generating two one-dimensional (1D) features. Subsequently, these 1D features were fed into a cross-modal predictive alignment module to improve fusion efficiency and two 1D attention modules to enhance feature representation. Furthermore, the fused features were recognized by a linear classifier. To evaluate the effectiveness of the proposed multimodal method, comprehensive validation tasks were conducted, including intra-session, cross-session, and cross-subject evaluations. In the intra-session task, the proposed architecture achieves an exceptional average accuracy of 99.93%. Moreover, in the cross-session task, our method demonstrates an average accuracy of 88.67%, surpassing the performance of EEG-only approach by 8.52%, eye tracking-only method by 5.92%, multimodal deep canonical correlation analysis (DCCA) technique by 0.42%, and multimodal deep generalized canonical correlation analysis (DGCCA) approach by 0.84%. Similarly, in the cross-subject task, the proposed approach achieves an average accuracy of 78.19%, outperforming EEG-only method by 5.87%, eye tracking-only approach by 4.21%, DCCA method by 0.55%, and DGCCA approach by 0.44%. The experimental results conclusively illustrate the superior effectiveness of the proposed method compared to both single modality approaches and canonical correlation analysis-based multimodal methods.

Research on low-power driving fatigue monitoring method based on spiking neural network

Fatigue driving is one of the leading causes of traffic accidents, and the rapid and accurate detection of driver fatigue is of paramount importance for enhancing road safety. However, the application of deep learning models in fatigue driving detection has long been constrained by high computational costs and power consumption. To address this issue, this study proposes an approach that combines Self-Organizing Map (SOM) and Spiking Neural Networks (SNN) to develop a low-power model capable of accurately recognizing the driver's mental state. Initially, spatial features are extracted from electroencephalogram (EEG) signals using the SOM network. Subsequently, the extracted weight vectors are encoded and fed into the SNN for fatigue driving classification. The research results demonstrate that the proposed method effectively considers the spatiotemporal characteristics of EEG signals, achieving efficient fatigue detection. Simultaneously, this approach successfully reduces the model's power consumption. When compared to traditional artificial neural networks, our method reduces energy consumption by approximately 12.21-42.59%.

The architecture of functional brain network modulated by driving under train running noise exposure

A noisy environment can considerably impact drivers' attention and fatigue, endangering driving safety. Consequently, this study designed a simulated driving experimental scenario to analyse the effects of noise generated during urban rail transit train operation on drivers' functional brain networks. The experiment recruited 16 participants, and the simulated driving scenario was conducted at noise levels of 50, 60, 70, and 80 dB. Functional connectivity between all electrode pairs across various frequency bands was evaluated using the weighted phase lag index (WPLI), and a brain network based on this was constructed. Graph theoretic analysis employed network global efficiency, degree, and clustering coefficient as metrics. Significant increases in the WPLI values of theta and alpha frequency bands were observed in high noise environments (70 dB, 80 dB), as well as enhanced brain synchronisation. Furthermore, concerning the topological metrics of brain networks, it was observed that the global efficiency of brain networks in theta and alpha frequency ranges, as well as the node degree and clustering coefficients, experienced substantial growth in high noise environments (70 dB, 80 dB) as opposed to 50 dB and 60 dB. This finding indicates that high-noise environments impact the reorganisation of functional brain networks, leading to a preference for network structures with improved global efficiency. Such findings may improve our understanding of the neural mechanisms of driving under noise exposure, and thus potentially reduce road accidents to some extent.

Intelligent Cockpits for Connected Vehicles: Taxonomy, Architecture, Interaction Technologies, and Future Directions

Highly integrated information sharing among people, vehicles, roads, and cloud systems, along with the rapid development of autonomous driving technologies, has spurred the evolution of automobiles from simple "transportation tools" to interconnected "intelligent systems". The intelligent cockpit is a comprehensive application space for various new technologies in intelligent vehicles, encompassing the domains of driving control, riding comfort, and infotainment. It provides drivers and passengers with safety, comfort, and pleasant driving experiences, serving as the gateway for traditional automobile manufacturing to upgrade towards an intelligent automotive industry ecosystem. This is the optimal convergence point for the intelligence, connectivity, electrification, and sharing of automobiles. Currently, the form, functions, and interaction methods of the intelligent cockpit are gradually changing, transitioning from the traditional "human adapts to the vehicle" viewpoint to the "vehicle adapts to human", and evolving towards a future of natural interactive services where "humans and vehicles mutually adapt". This article reviews the definitions, intelligence levels, functional domains, and technical frameworks of intelligent automotive cockpits. Additionally, combining the core mechanisms of human-machine interactions in intelligent cockpits, this article proposes an intelligent-cockpit human-machine interaction process and summarizes the current state of key technologies in intelligent-cockpit human-machine interactions. Lastly, this article analyzes the current challenges faced in the field of intelligent cockpits and forecasts future trends in intelligent cockpit technologies.

A novel methodology for emotion recognition through 62-lead EEG signals: multilevel heterogeneous recurrence analysis

Objective

Recognizing emotions from electroencephalography (EEG) signals is a challenging task due to the complex, nonlinear, and nonstationary characteristics of brain activity. Traditional methods often fail to capture these subtle dynamics, while deep learning approaches lack explainability. In this research, we introduce a novel three-phase methodology integrating manifold embedding, multilevel heterogeneous recurrence analysis (MHRA), and ensemble learning to address these limitations in EEG-based emotion recognition.

Approach

The proposed methodology was evaluated using the SJTU-SEED IV database. We first applied uniform manifold approximation and projection (UMAP) for manifold embedding of the 62-lead EEG signals into a lower-dimensional space. We then developed MHRA to characterize the complex recurrence dynamics of brain activity across multiple transition levels. Finally, we employed tree-based ensemble learning methods to classify four emotions (neutral, sad, fear, happy) based on the extracted MHRA features.

Main results

Our approach achieved high performance, with an accuracy of 0.7885 and an AUC of 0.7552, outperforming existing methods on the same dataset. Additionally, our methodology provided the most consistent recognition performance across different emotions. Sensitivity analysis revealed specific MHRA metrics that were strongly associated with each emotion, offering valuable insights into the underlying neural dynamics.

Significance

This study presents a novel framework for EEG-based emotion recognition that effectively captures the complex nonlinear and nonstationary dynamics of brain activity while maintaining explainability. The proposed methodology offers significant potential for advancing our understanding of emotional processing and developing more reliable emotion recognition systems with broad applications in healthcare and beyond.

Analysis of modulations of mental fatigue on intra-individual variability from single-trial event related potentials

BACKGROUND

Intra-individual variability (IIV), a measure of variance within an individual's performance, has been demonstrated as metrics of brain responses for neural functionality. However, how mental fatigue modulates IIV remains unclear. Consequently, the development of robust mental fatigue detection methods at the single-trial level is challenging.

NEW METHODS

Based on a long-duration flanker task EEG dataset, the modulations of mental fatigue on IIV were explored in terms of response time (RT) and trial-to-trial latency variations of event-related potentials (ERPs). Specifically, latency variations were quantified using residue iteration decomposition (RIDE) to reconstruct latency-corrected ERPs. We compared reconstructed ERPs with raw ERPs by means of temporal principal component analysis (PCA). Furthermore, a single-trial classification pipeline was developed to detect the changes of mental fatigue levels.

RESULTS

We found an increased IIV in the RT metric in the fatigue state compared to the alert state. The same sequence of ERPs (N1, P2, N2, P3a, P3b, and slow wave, or SW) was separated from both raw and reconstructed ERPs using PCA, whereas differences between raw and reconstructed ERPs in explained variances for separated ERPs were found owing to IIV. Particularly, a stronger N2 was detected in the fatigue than alert state after RIDE. The single-trial fatigue detection pipeline yielded an acceptable accuracy of 73.3%.

COMPARISON WITH EXISTING METHODS

The IIV has been linked to aging and brain disorders, and as an extension, our finding demonstrates IIV as an efficient indicator of mental fatigue.

CONCLUSIONS

This study reveals significant modulations of mental fatigue on IIV at the behavioral and neural levels and establishes a robust mental fatigue detection pipeline.

EEG Information Transfer Changes in Different Daily Fatigue Levels During Drowsy Driving

A significant issue for traffic safety has been drowsy driving for decades. A number of studies have investigated the effects of acute fatigue on spectral power; and recent research has revealed that drowsy driving is associated with a variety of brain connections in a specific cortico-cortical pathway. In spite of this, it is still unclear how different brain regions are connected in drowsy driving at different levels of daily fatigue. This study identified the brain connectivity-behavior relationship among three different daily fatigue levels (low-, median- and high-fatigue) with the EEG data transfer entropy. According to the results, only low- and medium-fatigue groups demonstrated an inverted U-shaped change in connectivity from high performance to poor behavioral performance. In addition, from low- to high-fatigue groups, connectivity magnitude decreased in the frontal region and increased in the occipital region. These study results suggest that brain connectivity and driving behavior would be affected by different levels of daily fatigue.

Resting-state EEG recorded with gel-based vs. consumer dry electrodes: spectral characteristics and across-device correlations

Introduction

Recordings of electroencephalographic (EEG) rhythms and their analyses have been instrumental in basic neuroscience, clinical diagnostics, and the field of brain-computer interfaces (BCIs). While in the past such measurements have been conducted mostly in laboratory settings, recent advancements in dry electrode technology pave way to a broader range of consumer and medical application because of their greater convenience compared to gel-based electrodes.

Methods

Here we conducted resting-state EEG recordings in two groups of healthy participants using three dry-electrode devices, the PSBD Headband, the PSBD Headphones and the Muse Headband, and one standard gel electrode-based system, the NVX. We examined signal quality for various spatial and spectral ranges which are essential for cognitive monitoring and consumer applications.

Results

Distinctive characteristics of signal quality were found, with the PSBD Headband showing sensitivity in low-frequency ranges and replicating the modulations of delta, theta and alpha power corresponding to the eyes-open and eyes-closed conditions, and the NVX system performing well in capturing high-frequency oscillations. The PSBD Headphones were more prone to low-frequency artifacts compared to the PSBD Headband, yet recorded modulations in the alpha power and had a strong alignment with the NVX at the higher EEG frequencies. The Muse Headband had several limitations in signal quality.

Discussion

We suggest that while dry-electrode technology appears to be appropriate for the EEG rhythm-based applications, the potential benefits of these technologies in terms of ease of use and accessibility should be carefully weighed against the capacity of each given system.

A Novel Approach for Automatic Detection of Driver Fatigue Using EEG Signals Based on Graph Convolutional Networks

Nowadays, the automatic detection of driver fatigue has become one of the important measures to prevent traffic accidents. For this purpose, a lot of research has been conducted in this field in recent years. However, the diagnosis of fatigue in recent research is binary and has no operational capability. This research presents a multi-class driver fatigue detection system based on electroencephalography (EEG) signals using deep learning networks. In the proposed system, a standard driving simulator has been designed, and a database has been collected based on the recording of EEG signals from 20 participants in five different classes of fatigue. In addition to self-report questionnaires, changes in physiological patterns are used to confirm the various stages of weariness in the suggested model. To pre-process and process the signal, a combination of generative adversarial networks (GAN) and graph convolutional networks (GCN) has been used. The proposed deep model includes five convolutional graph layers, one dense layer, and one fully connected layer. The accuracy obtained for the proposed model is 99%, 97%, 96%, and 91%, respectively, for the four different considered practical cases. The proposed model is compared to one developed through recent methods and research and has a promising performance.