A Review of EEG Signal Features and their Application in Driver Drowsiness Detection Systems

A multidimensional adaptive transformer network for fatigue detection

Variations in information processing patterns induced by operational directives under varying fatigue conditions within the cerebral cortex can be identified and analyzed through electroencephalogram (EEG) signals. The inherent complexity of EEG signals poses significant challenges in the effective detection of driver fatigue across diverse task scenarios. Recent advancements in deep learning, particularly the Transformer architecture, have shown substantial benefits in the retrieval and integration of multi-dimensional information. Nevertheless, the majority of current research primarily focuses on the application of Transformers for temporal information extraction, often overlooking other dimensions of EEG data. In response to this gap, the present study introduces a Multidimensional Adaptive Transformer Recognition Network specifically tailored for the identification of driving fatigue states. This network features a multidimensional Transformer architecture for feature extraction that adaptively assigns weights to various information dimensions, thereby facilitating feature compression and the effective extraction of structural information. This methodology ultimately enhances the model's accuracy and generalization capabilities. The experimental results indicate that the proposed methodology outperforms existing research methods when utilized with the SEED-VIG and SFDE datasets. Additionally, the analysis of multidimensional and frequency band features highlights the ability of the proposed network framework to elucidate differences in various multidimensional features during the identification of fatigue states.

DP-MP: a novel cross-subject fatigue detection framework with DANN-based prototypical representation and mix-up pairwise learning

Objective. Electroencephalography (EEG) is widely recognized as an effective method for detecting fatigue. However, practical applications of EEG for fatigue detection in real-world scenarios are often challenging, particularly in cases involving subjects not included in the training datasets, owing to bio-individual differences and noisy labels. This study aims to develop an effective framework for cross-subject fatigue detection by addressing these challenges.Approach. In this study, we propose a novel framework, termed DP-MP, for cross-subject fatigue detection, which utilizes a domain-adversarial neural network-based prototypical representation in conjunction with Mix-up pairwise learning. Our proposed DP-MP framework aims to mitigate the impact of bio-individual differences by encoding fatigue-related semantic structures within EEG signals and exploring shared fatigue prototype features across individuals. Notably, to the best of our knowledge, this work is the first to conceptualize fatigue detection as a pairwise learning task, thereby effectively reducing the interference from noisy labels. Furthermore, we propose the Mix-up pairwise learning (MixPa) approach in the field of fatigue detection, which broadens the advantages of pairwise learning by introducing more diverse and informative relationships among samples.Main results. Cross-subject experiments were conducted on two benchmark databases, SEED-VIG and FTEF, achieving state-of-the-art performance with average accuracies of 88.14%and 97.41%, respectively. These promising results demonstrate our model's effectiveness and excellent generalization capability.Significance. This is the first time EEG-based fatigue detection has been conceptualized as a pairwise learning task, offering a novel perspective to this field. Moreover, our proposed DP-MP framework effectively tackles the challenges of bio-individual differences and noisy labels in the fatigue detection field and demonstrates superior performance. Our work provides valuable insights for future research, promoting the practical application of brain-computer interfaces for fatigue detection.

On-Demand Seizures Facilitate Rapid Screening of Therapeutics for Epilepsy

Animal models of epilepsy are critical in drug development and therapeutic testing, but dominant methods for pharmaceutical evaluation face a tradeoff between higher throughput and etiological relevance. For example, in temporal lobe epilepsy, a type of epilepsy where seizures originate from limbic structures like the hippocampus, the main screening models are either based on acutely induced seizures in wild type, naïve animals or spontaneous seizures in chronically epileptic animals. Both types have their disadvantages - the acute convulsant or kindling induced seizures do not account for the myriad neuropathological changes in the diseased, epileptic brains, and spontaneous behavioral seizures are sparse in the chronically epileptic models, making it time-intensive to sufficiently power experiments. In this study, we took a mechanistic approach to precipitate seizures "on demand" in chronically epileptic mice. We briefly synchronized principal cells in the CA1 region of the diseased hippocampus to reliably induce stereotyped on-demand behavioral seizures. These induced seizures resembled naturally occurring spontaneous seizures in the epileptic animals and could be stopped by commonly prescribed anti-seizure medications such as levetiracetam and diazepam. Furthermore, we showed that seizures induced in chronically epileptic animals differed from those in naïve animals, highlighting the importance of evaluating therapeutics in the diseased circuit. Taken together, we envision our model to advance the speed at which both pharmacological and closed loop interventions for temporal lobe epilepsy are evaluated.

Mind the road: attention related neuromarkers during automated and manual simulated driving captured with a new mobile EEG sensor system

Background

Decline in vigilance due to fatigue is a common concern in traffic safety. Partially automated driving (PAD) systems can aid driving but decrease the driver's vigilance over time, due to reduced task engagement. Mobile EEG solutions can obtain neural information while operating a vehicle. The purpose of this study was to investigate how the behavior and brain activity associated with vigilance (i.e., alpha, beta and theta power) differs between PAD and manual driving, as well as changes over time, and how these effects can be detected using two different EEG systems.

Methods

Twenty-eight participants performed two 1-h simulated driving tasks, while wearing both a standard 24 channel EEG cap and a newly developed, unobtrusive and easy to apply 10 channel mobile EEG sensor-grid system. One scenario required manual control of the vehicle (manual) while the other required only monitoring the vehicle (PAD). Additionally, lane deviation, percentage eye-closure (PERCLOS) and subjective ratings of workload, fatigue and stress were obtained.

Results

Alpha, beta and theta power of the EEG as well as PERCLOS were higher in the PAD condition and increased over time in both conditions. The same spectral EEG effects were evident in both EEG systems. Lane deviation as an index of driving performance in the manual driving condition increased over time.

Conclusion

These effects indicate significant increases in fatigue and vigilance decrement over time while driving, and overall higher levels of fatigue and vigilance decrement associated with PAD. The EEG measures revealed significant effects earlier than the behavioral measures, demonstrating that EEG might allow faster detection of decreased vigilance than behavioral driving measures. This new, mobile EEG-grid system could be used to evaluate and improve driver monitoring systems in the field or even be used in the future as additional sensor to inform drivers of critical changes in their level of vigilance. In addition to driving, further areas of application for this EEG-sensor grid are safety critical work environments where vigilance monitoring is pivotal.

How predictive-forward-collision-warning reduces the collision risk of leading vehicle driver

Mixed platoon with a human-driven leading vehicle may be a transition mode prior to the widespread adoption of fully autonomous platoon. Enhancing the driving safety of the leading vehicle driver is crucial for improving the overall operational safety of the mixed platoon. Predictive-Forward-Collision-Warning (PFCW), an emerging technology in transportation, holds promise in mitigating collision risks for drivers by presenting traffic information beyond their immediate visual range. However, the influence characteristics of this function and how it influences the evolution of collision risk in leading vehicle driver remain unclear. Therefore, this paper attempts to analyse the quantitative impact of PFCW on the collision risk of leading vehicle driver. A test platform for connected mixed platoon was built utilizing driving simulation technology, alongside the development of a connected Human-Machine Interface (HMI) incorporating PFCW functionality. To evaluate the longitudinal collision risk of leading vehicle driver, a time-frequency analysis method was employed, focusing on key indicators: deceleration rate to avoid collision (DRAC), time to collision (TTC), and proportion of stopping distance (PSD). The time-domain analysis results indicated that PFCW can significantly mitigate the collision risk of leading vehicle. Wavelet transform results demonstrated that PFCW can ameliorate drivers' abnormal driving behavior and mitigate the collision risk in emergency situation of impending collision moment. Meanwhile, PFCW can enhance the overall operation safety of the mixed platoon. This paper leverages driving simulation technology and multidimensional indicators to analyze the quantitative impact of PFCW on the collision risk of leading vehicle driver during rapid deceleration of preceding vehicles. The findings can guide the development of test standards for connected mixed platoon, the promotion and application of PFCW, and the advancement of Navigate on Autopilot (NOA). Additionally, the test platform and framework developed in this study can accommodate various experimental needs for connected mixed platoon testing.

Depression diagnosis: EEG-based cognitive biomarkers and machine learning

Depression is a complex mental illness that has significant effects on people as well as society. The traditional techniques for the diagnosis of depression, along with the potential of nascent biomarkers especially EEG-based biomarkers, are studied. This review explores the significance of cognitive biomarkers, offering a comprehensive understanding of their role in the overall assessment of depression. It also investigates the effects of depression on various brain regions, outlines promising areas for future research, and emphasizes the importance of understanding the neurophysiological roots of depression. Furthermore, it elucidates how machine learning and deep learning models are integrated into EEG-based depression diagnosis, emphasizing their importance in optimizing personalized therapeutic protocols and improving diagnostic accuracy with EEG data analysis.

Towards predicting posttraumatic stress symptom severity using portable EEG-derived biomarkers

Posttraumatic Stress Disorder (PTSD) is a heterogeneous mental health disorder that can develop following a traumatic experience. Understanding its neurobiological basis is crucial to advance early diagnosis and treatment. Electroencephalography (EEG) can be used to explore the neurobiological basis of PTSD. However, only limited research has explored mobile EEG, which is important for scalability. This proof-of-concept study delves into mobile EEG-derived biomarkers for posttraumatic stress (PTS) symptom severity and their potential implications. Participants with partial PTSD, defined as meeting for at least three out of four symptom clusters, including hyperarousal symptoms, were enrolled in the study. Over four weeks, we measured PTS symptom severity using the PTSD checklist for DSM-5 (PCL-5) at multiple timepoints, and we recorded multiple EEG sessions from 21 individuals using a mobile EEG device. In total, we captured 38 EEG sessions, each comprising two recordings ("Recording A" and "Recording B") that lasted approximately 180 s, to evaluate reproducibility. Next, we extracted Shannon entropy, as a measure of the brain flexibility and complexity of the signal and spectral power for the fronto-temporal regions of interest, including electrodes at AF3, AF4, T7, and T8 for each EEG recording session. We calculated the partial correlation between the EEG variables and PCL- 5 measured closest to the EEG session, using age, sex, and the grouping variable 'batch' as covariates. We observed a significant negative correlation between Shannon entropy in fronto- temporal regions and PCL-5 scores. Specifically, this association was evident in the AF3 (r = -0.456, FDR-corrected p = 0.01), AF4 (r = -0.362, FDR-corrected p = 0.04), and T7 (r = -0.472, FDR-corrected p = 0.01) regions. Additionally, we found a significant negative association between the alpha power estimated from AF4 and PCL-5 (r = -0.429, FDR-corrected p = 0.04). Our findings suggest that EEG markers acquired using a mobile EEG device are associated with PTS symptom severity, offering valuable insights into the neurobiological mechanisms underlying PTSD and highlighting the potential benefits of this innovative technology in assessing and monitoring PTSD.

Comparing Manual and Automatic Artifact Detection in Sleep EEG Recordings

Sleep electroencephalogram (EEG) recordings can be contaminated by artifacts. Visual and automatic methods have been developed to mark such erroneous segments of EEG data. Here, we systematically explored the effect of artifacts on the sleep EEG power spectrum density (PSD), and we compared gold-standard visual detections to a simple automatic detector using Hjorth parameters to identify artifacts. We found that most distortions in the all-night average PSD occur because of a small minority of highly anomalous artifacts, which mainly affect the beta and gamma frequency ranges and NREM delta. Visual and automatic detections only showed moderate agreement in which data segments are artifactual. However, the resulting all-night average PSD is highly similar across all methods, and PSDs calculated with all methods successfully recover the known correlations of PSD with age and sex. No parameter settings of the automatic detector clearly outperformed others. Additionally, we showed that accurate average PSD estimates can be recovered from just a fraction of available data epochs. Our results suggest that artifacts represent a minor and easily solvable problem in sleep EEG recordings. Most visually identified artifacts do not seriously distort estimates of mid-frequency activity in the sleep EEG spectrum, and distortions to low and high frequencies can be eliminated using a simple automatic detection method nearly as well as with visual detections. These findings show that the visual inspection of EEG data is not necessary to eliminate the effects of artifacts, which is encouraging for the expected performance of automatic preprocessing in large sleep EEG databases.

Neuroimaging features for cognitive fatigue and its recovery with VR intervention: An EEG microstates analysis

INTRODUCTION

Cognitive fatigue is mainly caused by enduring mental stress or monotonous work, impairing cognitive and physical performance. Natural scene exposure is a promising intervention for relieving cognitive fatigue, but the efficacy of virtual reality (VR) simulated natural scene exposure is unclear. We aimed to investigate the effect of VR natural scene on cognitive fatigue and further explored its underlying neurophysiological alterations with electroencephalogram (EEG) microstates analysis.

METHODS

Ten participants performed a 20-minute 1-back task before and after VR intervention while EEG was recorded (pre-task, post-task). Performance was measured with mean accuracy rate (MAR) and mean reaction time (MRT) of the continuous 1-back task. VR simulation of the Canal Town scene was utilized to alleviate cognitive fatigue caused by 1-back tasks. Four resting-state phases were identified: beginning, pre, post, and end phases.

RESULTS

Post-task had a higher MAR and a lower MRT than pre-task. For pre-task, MAR was negatively correlated with trials, while MRT was positively correlated with trials. Four EEG microstates classes (A-D) were identified, and their temporal parameters (mean duration, time coverage and occurrence) and transition probabilities were calculated. After intervention, mean duration and time coverage of class B decreased, all parameters of class C increased, while all parameters of class D decreased. Transition probabilities between classes B and D decreased but increased between classes A and C.

CONCLUSION

VR simulation of Canal Town scene is a potentially effective method to alleviate cognitive fatigue. Microstate is an electrophysiological trait characteristic of cognitive fatigue and might be used to indicate the effect of VR intervention.

Fast processing and classification of epileptic seizures based on compressed EEG signals

The diagnosis of epilepsy based on visual inspection of electroencephalogram (EEG) signals is inherently complex and prone to error, even for physicians, mainly due to the large number of signals involved and the variability between individuals. These same challenges make the development of portable epilepsy diagnostic systems for everyday use difficult. Key obstacles include the immense complexity of signal processing and the inherent ambiguity in accurately classifying disease. For these reasons, we propose in this paper the deployment of compressive sensing to condense EEG signals while preserving relevant information, allowing seizure classification based on systematically selected features of the reconstructed signals. Based on a dataset comprising EEG recordings from 13 epileptic patients with various seizure types, we explore the deployment of the discrete cosine transform (DCT) and random matrix multiplication for compression ratios ranging from 5% to 70%, balancing data reduction with signal fidelity. Following the extraction of relevant features, selection was performed based on mutual information and a correlation matrix to preserve only the most relevant features for analysis. For classification, following a comparison of adequate machine learning models, XGBoost is chosen as it realizes a classification accuracy of 98.78%. The CS method was implemented on an STM32 microcontroller and a Raspberry Pi for reconstruction and classification, to demonstrate feasibility as an embedded system. At 70% compression, significant improvements have been observed: 70% file size reduction, 84% decrease in transmission time (from 2518.532s to 400.392s), and substantial energy savings (e.g., from 11.5±0.707 mWh to 4.5±0.707 mWh for Patient 12). Thereby, the signal quality was maintained with PSNR of 16.15±3.98 and Pearson correlation coefficient of 0.68±0.15. The proposed system highlights the potential for efficient, portable, real-time epilepsy diagnosis systems that achieve precise and fully automated seizure classification.

Driving Fatigue Onset and Visual Attention: An Electroencephalography-Driven Analysis of Ocular Behavior in a Driving Simulation Task

Attentional deficits have tragic consequences on road safety. These deficits are not solely caused by distraction, since they can also arise from other mental impairments such as, most frequently, mental fatigue. Fatigue is among the most prevalent impairing conditions while driving, degrading drivers' cognitive and physical abilities. This issue is particularly relevant for professional drivers, who spend most of their time behind the wheel. While scientific literature already documented the behavioral effects of driving fatigue, most studies have focused on drivers under sleep deprivation or anyhow at severe fatigue degrees, since it is difficult to recognize the onset of fatigue. The present study employed an EEG-driven approach to detect early signs of fatigue in professional drivers during a simulated task, with the aim of studying visual attention as fatigue begins to set in. Short-range and long-range professional drivers were recruited to take part in a 45-min-long simulated driving experiment. Questionnaires were used to validate the experimental protocol. A previously validated EEG index, the MDrow, was adopted as the benchmark measure for identifying the "fatigued" spans. Results of the eye-tracking analysis showed that, when fatigued, professional drivers tended to focus on non-informative portions of the driving environment. This paper presents evidence that an EEG-driven approach can be used to detect the onset of fatigue while driving and to study the related visual attention patterns. It was found that the onset of fatigue did not differentially impact drivers depending on their professional activity (short- vs. long-range delivery).

Objective assessment of cognitive fatigue: a bibliometric analysis

Aim

The objective of this study was to gain insight into the nature of cognitive fatigue and to identify future trends of objective assessment techniques in this field.

Methods

One thousand and eighty-five articles were retrieved from the Web of Science Core Collection database. R version 4.3.1, VOSviewer 1.6.20, CiteSpace 6.2.R4, and Microsoft Excel 2019 were used to perform the analysis.

Results

A total of 704 institutes from 56 countries participated in the relevant research, while the People's Republic of China contributed 126 articles and was the leading country. The most productive institute was the University of Gothenburg. Johansson Birgitta from the University of Gothenburg has posted the most articles (n = 13). The PLOS ONE published most papers (n = 38). The Neurosciences covered the most citations (n = 1,094). A total of 3,116 keywords were extracted and those with high frequency were mental fatigue, performance, quality-of-life, etc. Keywords mapping analysis indicated that cognitive fatigue caused by continuous work and traumatic brain injury, as well as its rehabilitation, have become the current research trend. The most co-cited literature was published in Sports Medicine. The strongest citation burst was related to electroencephalogram (EEG) event-related potential and spectral power analysis.

Conclusion

Publication information of related literature on the objective assessment of cognitive fatigue from 2007 to 2024 was summarized, including country and institute of origin, authors, and published journal, offering the current hotspots and novel directions in this field.

A review of artificial intelligence methods enabled music-evoked EEG emotion recognition and their applications

Music is an archaic form of emotional expression and arousal that can induce strong emotional experiences in listeners, which has important research and practical value in related fields such as emotion regulation. Among the various emotion recognition methods, the music-evoked emotion recognition method utilizing EEG signals provides real-time and direct brain response data, playing a crucial role in elucidating the neural mechanisms underlying music-induced emotions. Artificial intelligence technology has greatly facilitated the research on the recognition of music-evoked EEG emotions. AI algorithms have ushered in a new era for the extraction of characteristic frequency signals and the identification of novel feature signals. The robust computational capabilities of AI have provided fresh perspectives for the development of innovative quantitative models of emotions, tailored to various emotion recognition paradigms. The discourse surrounding AI algorithms in the context of emotional classification models is gaining momentum, with their applications in music therapy, neuroscience, and social activities increasingly coming under the spotlight. Through an in-depth analysis of the complete process of emotion recognition induced by music through electroencephalography (EEG) signals, we have systematically elucidated the influence of AI on pertinent research issues. This analysis offers a trove of innovative approaches that could pave the way for future research endeavors.

A novel deep learning model based on transformer and cross modality attention for classification of sleep stages

The classification of sleep stages is crucial for gaining insights into an individual's sleep patterns and identifying potential health issues. Employing several important physiological channels in different views, each providing a distinct perspective on sleep patterns, can have a great impact on the efficiency of the classification models. In the context of neural networks and deep learning models, transformers are very effective, especially when dealing with time series data, and have shown remarkable compatibility with sequential data analysis as physiological channels. On the other hand, cross-modality attention by integrating information from multiple views of the data enables to capture relationships among different modalities, allowing models to selectively focus on relevant information from each modality. In this paper, we introduce a novel deep-learning model based on transformer encoder-decoder and cross-modal attention for sleep stage classification. The proposed model processes information from various physiological channels with different modalities using the Sleep Heart Health Study Dataset (SHHS) data and leverages transformer encoders for feature extraction and cross-modal attention for effective integration to feed into the transformer decoder. The combination of these elements increased the accuracy of the model up to 91.33% in classifying five classes of sleep stages. Empirical evaluations demonstrated the model's superior performance compared to standalone approaches and other state-of-the-art techniques, showcasing the potential of combining transformer and cross-modal attention for improved sleep stage classification.

A Machine Learning Approach to Classifying EEG Data Collected with or without Haptic Feedback during a Simulated Drilling Task

Artificial Intelligence (AI), computer simulations, and virtual reality (VR) are increasingly becoming accessible tools that can be leveraged to implement training protocols and educational resources. Typical assessment tools related to sensory and neural processing associated with task performance in virtual environments often rely on self-reported surveys, unlike electroencephalography (EEG), which is often used to compare the effects of different types of sensory feedback (e.g., auditory, visual, and haptic) in simulation environments in an objective manner. However, it can be challenging to know which aspects of the EEG signal represent the impact of different types of sensory feedback on neural processing. Machine learning approaches offer a promising direction for identifying EEG signal features that differentiate the impact of different types of sensory feedback during simulation training. For the current study, machine learning techniques were applied to differentiate neural circuitry associated with haptic and non-haptic feedback in a simulated drilling task. Nine EEG channels were selected and analyzed, extracting different time-domain, frequency-domain, and nonlinear features, where 360 features were tested (40 features per channel). A feature selection stage identified the most relevant features, including the Hurst exponent of 13-21 Hz, kurtosis of 21-30 Hz, power spectral density of 21-30 Hz, variance of 21-30 Hz, and spectral entropy of 13-21 Hz. Using those five features, trials with haptic feedback were correctly identified from those without haptic feedback with an accuracy exceeding 90%, increasing to 99% when using 10 features. These results show promise for the future application of machine learning approaches to predict the impact of haptic feedback on neural processing during VR protocols involving drilling tasks, which can inform future applications of VR and simulation for occupational skill acquisition.

Multi-Index Driver Drowsiness Detection Method Based on Driver's Facial Recognition Using Haar Features and Histograms of Oriented Gradients

It is estimated that 10% to 20% of road accidents are related to fatigue, with accidents caused by drowsiness up to twice as deadly as those caused by other factors. In order to reduce these numbers, strategies such as advertising campaigns, the implementation of driving recorders in vehicles used for road transport of goods and passengers, or the use of drowsiness detection systems in cars have been implemented. Within the scope of the latter area, the technologies used are diverse. They can be based on the measurement of signals such as steering wheel movement, vehicle position on the road, or driver monitoring. Driver monitoring is a technology that has been exploited little so far and can be implemented in many different approaches. This work addresses the evaluation of a multidimensional drowsiness index based on the recording of facial expressions, gaze direction, and head position and studies the feasibility of its implementation in a low-cost electronic package. Specifically, the aim is to determine the driver's state by monitoring their facial expressions, such as the frequency of blinking, yawning, eye-opening, gaze direction, and head position. For this purpose, an algorithm capable of detecting drowsiness has been developed. Two approaches are compared: Facial recognition based on Haar features and facial recognition based on Histograms of Oriented Gradients (HOG). The implementation has been carried out on a Raspberry Pi, a low-cost device that allows the creation of a prototype that can detect drowsiness and interact with peripherals such as cameras or speakers. The results show that the proposed multi-index methodology performs better in detecting drowsiness than algorithms based on one-index detection.

Wireless ear EEG to monitor drowsiness

Neural wearables can enable life-saving drowsiness and health monitoring for pilots and drivers. While existing in-cabin sensors may provide alerts, wearables can enable monitoring across more environments. Current neural wearables are promising but most require wet-electrodes and bulky electronics. This work showcases in-ear, dry-electrode earpieces used to monitor drowsiness with compact hardware. The employed system integrates additive-manufacturing for dry, user-generic earpieces, existing wireless electronics, and offline classification algorithms. Thirty-five hours of electrophysiological data were recorded across nine subjects performing drowsiness-inducing tasks. Three classifier models were trained with user-specific, leave-one-trial-out, and leave-one-user-out splits. The support-vector-machine classifier achieved an accuracy of 93.2% while evaluating users it has seen before and 93.3% when evaluating a never-before-seen user. These results demonstrate wireless, dry, user-generic earpieces used to classify drowsiness with comparable accuracies to existing state-of-the-art, wet electrode in-ear and scalp systems. Further, this work illustrates the feasibility of population-trained classification in future electrophysiological applications.

A Review of EEG-based Localization of Epileptic Seizure Foci: Common Points with Multimodal Fusion of Brain Data

Unexpected seizures significantly decrease the quality of life in epileptic patients. Seizure attacks are caused by hyperexcitability and anatomical lesions of special regions of the brain, and cognitive impairments and memory deficits are their most common concomitant effects. In addition to seizure reduction treatments, medical rehabilitation involving brain-computer interfaces and neurofeedback can improve cognition and quality of life in patients with focal epilepsy in most cases, in particular when resective epilepsy surgery has been considered treatment in drug-resistant epilepsy. Source estimation and precise localization of epileptic foci can improve such rehabilitation and treatment. Electroencephalography (EEG) monitoring and multimodal noninvasive neuroimaging techniques such as ictal/interictal single-photon emission computerized tomography (SPECT) imaging and structural magnetic resonance imaging are common practices for the localization of epileptic foci and have been studied in several kinds of researches. In this article, we review the most recent research on EEG-based localization of seizure foci and discuss various methods, their advantages, limitations, and challenges with a focus on model-based data processing and machine learning algorithms. In addition, we survey whether combined analysis of EEG monitoring and neuroimaging techniques, which is known as multimodal brain data fusion, can potentially increase the precision of the seizure foci localization. To this end, we further review and summarize the key parameters and challenges of processing, fusion, and analysis of multiple source data, in the framework of model-based signal processing, for the development of a multimodal brain data analyzing system. This article has the potential to be used as a valuable resource for neuroscience researchers for the development of EEG-based rehabilitation systems based on multimodal data analysis related to focal epilepsy.

Towards predicting PTSD symptom severity using portable EEG-derived biomarkers

Posttraumatic Stress Disorder (PTSD) is a heterogeneous mental health disorder that occurs following traumatic experience. Understanding its neurobiological basis is crucial to advance early diagnosis and treatment. Electroencephalography (EEG) can be used to explore the neurobiological basis of PTSD. However, only limited research has explored mobile EEG, which is important for scalability. This proof-of-concept study delves into mobile EEG-derived biomarkers for PTSD and their potential implications. Over four weeks, we measured PTSD symptoms using the PTSD checklist for DSM-5 (PCL-5) at multiple timepoints, and we recorded multiple EEG sessions from 21 individuals using a mobile EEG device. In total, we captured 38 EEG sessions, each comprising two recordings that lasted approximately 180 seconds, to evaluate reproducibility. Next, we extracted Shannon entropy, as a measure of the randomness or unpredictability of the signal and spectral power for the fronto-temporal regions of interest, including electrodes at AF3, AF4, T7, and T8 for each EEG recording session. We calculated the partial correlation between the EEG variables and PCL-5 measured closest to the EEG session, using age, sex, and the grouping variable 'batch' as covariates. We observed a significant negative correlation between Shannon entropy in fronto-temporal regions and PCL-5 scores. Specifically, this association was evident in the AF3 (r = -0.456, FDR-corrected p = 0.01), AF4 (r = -0.362, FDR-corrected p = 0.04), and T7 (r = -0.472, FDR-corrected p = 0.01) regions. Additionally, we found a significant negative association between the alpha power estimated from AF4 and PCL-5 (r=-0.429, FDR-corrected p=0.04). Our findings suggest that EEG data acquired using a mobile EEG device is associated with PTSD symptom severity, offering valuable insights into the neurobiological mechanisms underlying PTSD.

Efficient Generalized Electroencephalography-Based Drowsiness Detection Approach with Minimal Electrodes

Drowsiness is a main factor for various costly defects, even fatal accidents in areas such as construction, transportation, industry and medicine, due to the lack of monitoring vigilance in the mentioned areas. The implementation of a drowsiness detection system can greatly help to reduce the defects and accident rates by alerting individuals when they enter a drowsy state. This research proposes an electroencephalography (EEG)-based approach for detecting drowsiness. EEG signals are passed through a preprocessing chain composed of artifact removal and segmentation to ensure accurate detection followed by different feature extraction methods to extract the different features related to drowsiness. This work explores the use of various machine learning algorithms such as Support Vector Machine (SVM), the K nearest neighbor (KNN), the Naive Bayes (NB), the Decision Tree (DT), and the Multilayer Perceptron (MLP) to analyze EEG signals sourced from the DROZY database, carefully labeled into two distinct states of alertness (awake and drowsy). Segmentation into 10 s intervals ensures precise detection, while a relevant feature selection layer enhances accuracy and generalizability. The proposed approach achieves high accuracy rates of 99.84% and 96.4% for intra (subject by subject) and inter (cross-subject) modes, respectively. SVM emerges as the most effective model for drowsiness detection in the intra mode, while MLP demonstrates superior accuracy in the inter mode. This research offers a promising avenue for implementing proactive drowsiness detection systems to enhance occupational safety across various industries.

Individual Variability in Brain Connectivity Patterns and Driving-Fatigue Dynamics

Mental fatigue during driving poses significant risks to road safety, necessitating accurate assessment methods to mitigate potential hazards. This study explores the impact of individual variability in brain networks on driving fatigue assessment, hypothesizing that subject-specific connectivity patterns play a pivotal role in understanding fatigue dynamics. By conducting a linear regression analysis of subject-specific brain networks in different frequency bands, this research aims to elucidate the relationships between frequency-specific connectivity patterns and driving fatigue. As such, an EEG sustained driving simulation experiment was carried out, estimating individuals' brain networks using the Phase Lag Index (PLI) to capture shared connectivity patterns. The results unveiled notable variability in connectivity patterns across frequency bands, with the alpha band exhibiting heightened sensitivity to driving fatigue. Individualized connectivity analysis underscored the complexity of fatigue assessment and the potential for personalized approaches. These findings emphasize the importance of subject-specific brain networks in comprehending fatigue dynamics, while providing sensor space minimization, advocating for the development of efficient mobile sensor applications for real-time fatigue detection in driving scenarios.

Biomarkers of Immersion in Virtual Reality Based on Features Extracted from the EEG Signals: A Machine Learning Approach

Virtual reality (VR) enables the development of virtual training frameworks suitable for various domains, especially when real-world conditions may be hazardous or impossible to replicate because of unique additional resources (e.g., equipment, infrastructure, people, locations). Although VR technology has significantly advanced in recent years, methods for evaluating immersion (i.e., the extent to which the user is engaged with the sensory information from the virtual environment or is invested in the intended task) continue to rely on self-reported questionnaires, which are often administered after using the virtual scenario. Having an objective method to measure immersion is particularly important when using VR for training, education, and applications that promote the development, fine-tuning, or maintenance of skills. The level of immersion may impact performance and the translation of knowledge and skills to the real-world. This is particularly important in tasks where motor skills are combined with complex decision making, such as surgical procedures. Efforts to better measure immersion have included the use of physiological measurements including heart rate and skin response, but so far they do not offer robust metrics that provide the sensitivity to discriminate different states (idle, easy, and hard), which is critical when using VR for training to determine how successful the training is in engaging the user's senses and challenging their cognitive capabilities. In this study, electroencephalography (EEG) data were collected from 14 participants who completed VR jigsaw puzzles with two different levels of task difficulty. Machine learning was able to accurately classify the EEG data collected during three different states, obtaining accuracy rates of 86% and 97% for differentiating easy versus hard difficulty states and baseline vs. VR states. Building on these results may enable the identification of robust biomarkers of immersion in VR, enabling real-time recognition of the level of immersion that can be used to design more effective and translative VR-based training. This method has the potential to adjust aspects of VR related to task difficulty to ensure that participants are immersed in VR.

Driver Drowsiness Multi-Method Detection for Vehicles with Autonomous Driving Functions

The article outlines various approaches to developing a fuzzy decision algorithm designed for monitoring and issuing warnings about driver drowsiness. This algorithm is based on analyzing EOG (electrooculography) signals and eye state images with the aim of preventing accidents. The drowsiness warning system comprises key components that learn about, analyze and make decisions regarding the driver's alertness status. The outcomes of this analysis can then trigger warnings if the driver is identified as being in a drowsy state. Driver drowsiness is characterized by a gradual decline in attention to the road and traffic, diminishing driving skills and an increase in reaction time, all contributing to a higher risk of accidents. In cases where the driver does not respond to the warnings, the ADAS (advanced driver assistance systems) system should intervene, assuming control of the vehicle's commands.

AS, Hatlestad-Hall C, Yazidi A, Hammer HL, Hebold Haraldsen IRJ. Introducing Region Based Pooling for handling a varied number of EEG channels for deep learning models

Introduction

A challenge when applying an artificial intelligence (AI) deep learning (DL) approach to novel electroencephalography (EEG) data, is the DL architecture's lack of adaptability to changing numbers of EEG channels. That is, the number of channels cannot vary neither in the training data, nor upon deployment. Such highly specific hardware constraints put major limitations on the clinical usability and scalability of the DL models.

Methods

In this work, we propose a technique for handling such varied numbers of EEG channels by splitting the EEG montages into distinct regions and merge the channels within the same region to a region representation. The solution is termed Region Based Pooling (RBP). The procedure of splitting the montage into regions is performed repeatedly with different region configurations, to minimize potential loss of information. As RBP maps a varied number of EEG channels to a fixed number of region representations, both current and future DL architectures may apply RBP with ease. To demonstrate and evaluate the adequacy of RBP to handle a varied number of EEG channels, sex classification based solely on EEG was used as a test example. The DL models were trained on 129 channels, and tested on 32, 65, and 129-channels versions of the data using the same channel positions scheme. The baselines for comparison were zero-filling the missing channels and applying spherical spline interpolation. The performances were estimated using 5-fold cross validation.

Results

For the 32-channel system version, the mean AUC values across the folds were: RBP (93.34%), spherical spline interpolation (93.36%), and zero-filling (76.82%). Similarly, on the 65-channel system version, the performances were: RBP (93.66%), spherical spline interpolation (93.50%), and zero-filling (85.58%). Finally, the 129-channel system version produced the following results: RBP (94.68%), spherical spline interpolation (93.86%), and zero-filling (91.92%).

Conclusion

In conclusion, RBP obtained similar results to spherical spline interpolation, and superior results to zero-filling. We encourage further research and development of DL models in the cross-dataset setting, including the use of methods such as RBP and spherical spline interpolation to handle a varied number of EEG channels.

Point of Care Testing (POCT) in Psychopathology Using Fractal Analysis and Hilbert Huang Transform of Electroencephalogram (EEG)

Research has shown that relying only on self-reports for diagnosing psychiatric disorders does not yield accurate results at all times. The advances of technology as well as artificial intelligence and other machine learning algorithms have allowed the introduction of point of care testing (POCT) including EEG characterization and correlations with possible psychopathology. Nonlinear methods of EEG analysis have significant advantages over linear methods. Empirical mode decomposition (EMD) is a reliable nonlinear method of EEG pre-processing. In this chapter, we compare two existing EEG complexity measures - Higuchi fractal dimension (HFD) and sample entropy (SE), with our newly proposed method using Higuchi fractal dimension from the Hilbert Huang transform (HFD-HHT). We present an example using the three complexity measures on a 2-minute EEG recorded from a healthy 20-year-old male after signal pre-processing. Furthermore, we showed the usefulness of these complexity measures in the classification of major depressive disorder (MDD) with healthy controls. Our study is in line with previous research and has shown an increase in HFD and SE values in the full, alpha and beta frequency bands suggestive of an increase in EEG irregularity. Moreover, the HFD-HHT values decreased in those three bands for majority of electrodes which is suggestive of a decrease in irregularity in the frequency-time domain. We conclude that all three complexity measures can be vital features useful for EEG analysis which could be incorporated in POCT systems.

Physiological measurements for driving drowsiness: A comparative study of multi-modality feature fusion and selection

A large number of traffic accidents were caused by drowsiness while driving. In-vehicle alert system based on physiological signals was one of the most promising solutions to monitor driving fatigue. However, different physiological modalities can be used, and many relative studies compared different modalities without considering the implementation feasibility of portable or wearable devices. Moreover, evaluations of each modality in previous studies were based on inconsistent choices of fatigue label and signal features, making it hard to compare the results of different studies. Therefore, the modality comparison and fusion for continuous drowsiness estimation while driving was still unclear. This work sought to comprehensively compare widely-used physiological modalities, including forehead electroencephalogram (EEG), electrooculogram (EOG), R-R intervals (RRI) and breath, in a hardware setting feasible for portable or wearable devices to monitor driving fatigue. Moreover, a more general conclusion on modality comparison and fusion was reached based on the regression of features or their combinations and the awake-to-drowsy transition. Finally, the feature subset of fused modalities was produced by feature selection method, to select the optimal feature combination and reduce computation consumption. Considering practical feasibility, the most effective combination with the highest correlation coefficient was using forehead EEG or EOG, along with RRI and RRI-derived breath. If more comfort and convenience was required, the combination of RRI and RRI-derived breath was also promising.

Neurophysiological mental fatigue assessment for developing user-centered Artificial Intelligence as a solution for autonomous driving

The human factor plays a key role in the automotive field since most accidents are due to drivers' unsafe and risky behaviors. The industry is now pursuing two main solutions to deal with this concern: in the short term, there is the development of systems monitoring drivers' psychophysical states, such as inattention and fatigue, and in the medium-long term, there is the development of fully autonomous driving. This second solution is promoted by recent technological progress in terms of Artificial Intelligence and sensing systems aimed at making vehicles more and more accurately aware of their "surroundings." However, even with an autonomous vehicle, the driver should be able to take control of the vehicle when needed, especially during the current transition from the lower (SAE < 3) to the highest level (SAE = 5) of autonomous driving. In this scenario, the vehicle has to be aware not only of its "surroundings" but also of the driver's psychophysical state, i.e., a user-centered Artificial Intelligence. The neurophysiological approach is one the most effective in detecting improper mental states. This is particularly true if considering that the more automatic the driving will be, the less available the vehicular data related to the driver's driving style. The present study aimed at employing a holistic approach, considering simultaneously several neurophysiological parameters, in particular, electroencephalographic, electrooculographic, photopletismographic, and electrodermal activity data to assess the driver's mental fatigue in real time and to detect the onset of fatigue increasing. This would ideally work as an information/trigger channel for the vehicle AI. In all, 26 professional drivers were engaged in a 45-min-lasting realistic driving task in simulated conditions, during which the previously listed biosignals were recorded. Behavioral (reaction times) and subjective measures were also collected to validate the experimental design and to support the neurophysiological results discussion. Results showed that the most sensitive and timely parameters were those related to brain activity. To a lesser extent, those related to ocular parameters were also sensitive to the onset of mental fatigue, but with a delayed effect. The other investigated parameters did not significantly change during the experimental session.

Entropy-Based Machine Learning Model for Fast Diagnosis and Monitoring of Parkinson's Disease

This study presents the concept of a computationally efficient machine learning (ML) model for diagnosing and monitoring Parkinson's disease (PD) using rest-state EEG signals (rs-EEG) from 20 PD subjects and 20 normal control (NC) subjects at a sampling rate of 128 Hz. Based on the comparative analysis of the effectiveness of entropy calculation methods, fuzzy entropy showed the best results in diagnosing and monitoring PD using rs-EEG, with classification accuracy (ARKF) of ~99.9%. The most important frequency range of rs-EEG for PD-based diagnostics lies in the range of 0-4 Hz, and the most informative signals were mainly received from the right hemisphere of the head. It was also found that ARKF significantly decreased as the length of rs-EEG segments decreased from 1000 to 150 samples. Using a procedure for selecting the most informative features, it was possible to reduce the computational costs of classification by 11 times, while maintaining an ARKF ~99.9%. The proposed method can be used in the healthcare internet of things (H-IoT), where low-performance edge devices can implement ML sensors to enhance human resilience to PD.

Multimodal Approach for Pilot Mental State Detection Based on EEG

The safety of flight operations depends on the cognitive abilities of pilots. In recent years, there has been growing concern about potential accidents caused by the declining mental states of pilots. We have developed a novel multimodal approach for mental state detection in pilots using electroencephalography (EEG) signals. Our approach includes an advanced automated preprocessing pipeline to remove artefacts from the EEG data, a feature extraction method based on Riemannian geometry analysis of the cleaned EEG data, and a hybrid ensemble learning technique that combines the results of several machine learning classifiers. The proposed approach provides improved accuracy compared to existing methods, achieving an accuracy of 86% when tested on cleaned EEG data. The EEG dataset was collected from 18 pilots who participated in flight experiments and publicly released at NASA's open portal. This study presents a reliable and efficient solution for detecting mental states in pilots and highlights the potential of EEG signals and ensemble learning algorithms in developing cognitive cockpit systems. The use of an automated preprocessing pipeline, feature extraction method based on Riemannian geometry analysis, and hybrid ensemble learning technique set this work apart from previous efforts in the field and demonstrates the innovative nature of the proposed approach.

A Bio-Inspired Chaos Sensor Model Based on the Perceptron Neural Network: Machine Learning Concept and Application for Computational Neuro-Science

The study presents a bio-inspired chaos sensor model based on the perceptron neural network for the estimation of entropy of spike train in neurodynamic systems. After training, the sensor on perceptron, having 50 neurons in the hidden layer and 1 neuron at the output, approximates the fuzzy entropy of a short time series with high accuracy, with a determination coefficient of R2~0.9. The Hindmarsh-Rose spike model was used to generate time series of spike intervals, and datasets for training and testing the perceptron. The selection of the hyperparameters of the perceptron model and the estimation of the sensor accuracy were performed using the K-block cross-validation method. Even for a hidden layer with one neuron, the model approximates the fuzzy entropy with good results and the metric R2~0.5 ÷ 0.8. In a simplified model with one neuron and equal weights in the first layer, the principle of approximation is based on the linear transformation of the average value of the time series into the entropy value. An example of using the chaos sensor on spike train of action potential recordings from the L5 dorsal rootlet of rat is provided. The bio-inspired chaos sensor model based on an ensemble of neurons is able to dynamically track the chaotic behavior of a spike signal and transmit this information to other parts of the neurodynamic model for further processing. The study will be useful for specialists in the field of computational neuroscience, and also to create humanoid and animal robots, and bio-robots with limited resources.

Influence of Multimodal Emotional Stimulations on Brain Activity: An Electroencephalographic Study

This study aimed to reveal the influence of emotional valence and sensory modality on neural activity in response to multimodal emotional stimuli using scalp EEG. In this study, 20 healthy participants completed the emotional multimodal stimulation experiment for three stimulus modalities (audio, visual, and audio-visual), all of which are from the same video source with two emotional components (pleasure or unpleasure), and EEG data were collected using six experimental conditions and one resting state. We analyzed power spectral density (PSD) and event-related potential (ERP) components in response to multimodal emotional stimuli, for spectral and temporal analysis. PSD results showed that the single modality (audio only/visual only) emotional stimulation PSD differed from multi-modality (audio-visual) in a wide brain and band range due to the changes in modality and not from the changes in emotional degree. The most pronounced N200-to-P300 potential shifts occurred in monomodal rather than multimodal emotional stimulations. This study suggests that emotional saliency and sensory processing efficiency perform a significant role in shaping neural activity during multimodal emotional stimulation, with the sensory modality being more influential in PSD. These findings contribute to our understanding of the neural mechanisms involved in multimodal emotional stimulation.

STGATE: Spatial-temporal graph attention network with a transformer encoder for EEG-based emotion recognition

Electroencephalogram (EEG) is a crucial and widely utilized technique in neuroscience research. In this paper, we introduce a novel graph neural network called the spatial-temporal graph attention network with a transformer encoder (STGATE) to learn graph representations of emotion EEG signals and improve emotion recognition performance. In STGATE, a transformer-encoder is applied for capturing time-frequency features which are fed into a spatial-temporal graph attention for emotion classification. Using a dynamic adjacency matrix, the proposed STGATE adaptively learns intrinsic connections between different EEG channels. To evaluate the cross-subject emotion recognition performance, leave-one-subject-out experiments are carried out on three public emotion recognition datasets, i.e., SEED, SEED-IV, and DREAMER. The proposed STGATE model achieved a state-of-the-art EEG-based emotion recognition performance accuracy of 90.37% in SEED, 76.43% in SEED-IV, and 76.35% in DREAMER dataset, respectively. The experiments demonstrated the effectiveness of the proposed STGATE model for cross-subject EEG emotion recognition and its potential for graph-based neuroscience research.

Byte-Pair Encoding for Classifying Routine Clinical Electroencephalograms in Adults Over the Lifespan

Routine clinical EEG is a standard test used for the neurological evaluation of patients. A trained specialist interprets EEG recordings and classifies them into clinical categories. Given time demands and high inter-reader variability, there is an opportunity to facilitate the evaluation process by providing decision support tools that can classify EEG recordings automatically. Classifying clinical EEG is associated with several challenges: classification models are expected to be interpretable; EEGs vary in duration and EEGs are recorded by multiple technicians operating various devices. Our study aimed to test and validate a framework for EEG classification which satisfies these requirements by transforming EEG into unstructured text. We considered a highly heterogeneous and extensive sample of routine clinical EEGs (n = 5785), with a wide range of participants aged between 15 and 99 years. EEG scans were recorded at a public hospital, according to 10/20 electrode positioning with 20 electrodes. The proposed framework was based on symbolizing EEG signals and adapting a previously proposed method from natural language processing (NLP) to break symbols into words. Specifically, we symbolized the multichannel EEG time series and applied a byte-pair encoding (BPE) algorithm to extract a dictionary of the most frequent patterns (tokens) reflecting the variability of EEG waveforms. To demonstrate the performance of our framework, we used newly-reconstructed EEG features to predict patients' biological age with a Random Forest regression model. This age prediction model achieved a mean absolute error of 15.7 years. We also correlated tokens' occurrence frequencies with age. The highest correlations between the frequencies of tokens and age were observed at frontal and occipital EEG channels. Our findings demonstrated the feasibility of applying an NLP-based approach to classifying routine clinical EEG. Notably, the proposed algorithm could be instrumental in classifying clinical EEG with minimal preprocessing and identifying clinically-relevant short events, such as epileptic spikes.

Assessing the development of mental fatigue during simulated flights with concurrent EEG-fNIRS measurement

Mental fatigue (MF) can impair pilots' performance and reactions to unforeseen events and is therefore an important concept within aviation. The physiological measurement of MF, especially with EEG and, in recent years, fNIRS, has gained much attention. However, a systematic investigation and comparison of the measurements is seldomly done. We induced MF via time on task during a 90-min simulated flight task and collected concurrent EEG-fNIRS, performance and self-report data from 31 participants. While their subjective MF increased linearly, the participants were able to keep their performance stable over the course of the experiment. EEG data showed an early increase and levelling in parietal alpha power and a slower, but steady increase in frontal theta power. No consistent trend could be observed in the fNIRS data. Thus, more research on fNIRS is needed to understand its possibilities and limits for MF assessment, and a combination with EEG is advisable to compare and validate results. Until then, EEG remains the better choice for continuous MF assessment in cockpit applications because of its high sensitivity to a transition from alert to fatigued, even before performance is impaired.

Automatic Identification of Children with ADHD from EEG Brain Waves

EEG (electroencephalogram) signals could be used reliably to extract critical information regarding ADHD (attention deficit hyperactivity disorder), a childhood neurodevelopmental disorder. The early detection of ADHD is important to lessen the development of this disorder and reduce its long-term impact. This study aimed to develop a computer algorithm to identify children with ADHD automatically from the characteristic brain waves. An EEG machine learning pipeline is presented here, including signal preprocessing and data preparation steps, with thorough explanations and rationale. A large public dataset of 120 children was selected, containing large variability and minimal measurement bias in data collection and reproducible child-friendly visual attentional tasks. Unlike other studies, EEG linear features were extracted to train a Gaussian SVM-based model from only the first four sub-bands of EEG. This eliminates signals more than 30 Hz, thus reducing the computational load for model training while keeping mean accuracy of ~94%. We also performed rigorous validation (obtained 93.2% and 94.2% accuracy, respectively, for holdout and 10-fold cross-validation) to ensure that the developed model is minimally impacted by bias and overfitting that commonly appear in the ML pipeline. These performance metrics indicate the ability to automatically identify children with ADHD from a local clinical setting and provide a baseline for further clinical evaluation and timely therapeutic attempts.

EEG microstate changes according to mental fatigue induced by aircraft piloting simulation: An exploratory study

BACKGROUND

A continuous flight task load can induce fatigue and lead to changes in electroencephalography (EEG). EEG microstates can reflect the activities of large-scale neural networks during mental fatigue. This exploratory experiment explored the effects of mental fatigue induced by continuous simulated flight multitasking on EEG microstate indices.

METHODS

Twenty-four participants performed continuous 2-hour aircraft piloting simulation while EEG were recorded. The Stanford sleepiness scale (SSS) and critical flicker fusion frequency (CFF) were measured before and after the task. Microstate analysis was applied to EEG. Four microstate classes (A-D) were identified during the pre-task, post-task, beginning, and end phases. The effects of mental fatigue were analyzed.

RESULTS

Compared with the pre-task, the post-task had a higher global explained variance (GEV) and time parameters of class C but lower occurrence and coverage of class D. The end had a higher GEV but lower duration and coverage of class D than at the beginning. After 2 h of multitasking, the transition probability between A and D, and between B and D decreased but between A and C increased. Subjective fatigue scores were negatively correlated with occurrence and coverage of class D. Task performance was negatively correlated with duration and coverage of class C but positively correlated with duration and occurrence of class B.

CONCLUSION

Time parameters and transition probability of EEG microstates can detect mental fatigue induced by continuous aircraft piloting simulation. The global brain network activation of mental fatigue can be detected by EEG microstates that can evaluate flight fatigue.

Designing an Embedded Feature Selection Algorithm for a Drowsiness Detector Model Based on Electroencephalogram Data

Driver fatigue reduces the safety of traditional driving and limits the widespread adoption of self-driving cars; hence, the monitoring and early detection of drivers' drowsiness plays a key role in driving automation. When representing the drowsiness indicators as large feature vectors, fitting a machine learning model to the problem becomes challenging, and the problem's perspicuity decreases, making dimensionality reduction crucial in practice. For this reason, we propose an embedded feature selection algorithm that can be later utilized as a building block in the system development of a neural network-based drowsiness detector. We have adopted a technique: a so-called Feature Prune Layer is placed in front of the first layer in the architecture; as a result, its weights change regarding the importance of the corresponding input features and are deleted iteratively until the desired number is reached. We test the algorithm on EEG data, as it is one of the best indicators of drowsiness based on the literature. The proposed FS algorithm is able to reduce the original feature set by 95% with only 1% degradation in precision, while the precision increases by 1.5% and 2.7% respectively when selecting the top 10% and top 20% of the initial features. Moreover, the proposed method outperforms the widely popular Principal Component Analysis and the Chi-squared test when reducing the original feature set by 95%: it achieves 24.3% and 3.2% higher precision respectively.

On the Influence of Aging on Classification Performance in the Visual EEG Oddball Paradigm Using Statistical and Temporal Features

The utilization of a non-invasive electroencephalogram (EEG) as an input sensor is a common approach in the field of the brain-computer interfaces (BCI). However, the collected EEG data pose many challenges, one of which may be the age-related variability of event-related potentials (ERPs), which are often used as primary EEG BCI signal features. To assess the potential effects of aging, a sample of 27 young and 43 older healthy individuals participated in a visual oddball study, in which they passively viewed frequent stimuli among randomly occurring rare stimuli while being recorded with a 32-channel EEG set. Two types of EEG datasets were created to train the classifiers, one consisting of amplitude and spectral features in time and another with extracted time-independent statistical ERP features. Among the nine classifiers tested, linear classifiers performed best. Furthermore, we show that classification performance differs between dataset types. When temporal features were used, maximum individuals' performance scores were higher, had lower variance, and were less affected overall by within-class differences such as age. Finally, we found that the effect of aging on classification performance depends on the classifier and its internal feature ranking. Accordingly, performance will differ if the model favors features with large within-class differences. With this in mind, care must be taken in feature extraction and selection to find the correct features and consequently avoid potential age-related performance degradation in practice.

Making a case for endovascular approaches for neural recording and stimulation

There are many electrode types for recording and stimulating neural tissue, most of which necessitate direct contact with the target tissue. These electrodes range from large, scalp electrodes which are used to non-invasively record averaged, low frequency electrical signals from large areas/volumes of the brain, to penetrating microelectrodes which are implanted directly into neural tissue and interface with one or a few neurons. With the exception of scalp electrodes (which provide very low-resolution recordings), each of these electrodes requires a highly invasive, open brain surgical procedure for implantation, which is accompanied by significant risk to the patient. To mitigate this risk, a minimally invasive endovascular approach can be used. Several types of endovascular electrodes have been developed to be delivered into the blood vessels in the brain via a standard catheterization procedure. In this review, the existing body of research on the development and application of endovascular electrodes is presented. The capabilities of each of these endovascular electrodes is compared to commonly used direct-contact electrodes to demonstrate the relative efficacy of the devices. Potential clinical applications of endovascular recording and stimulation and the advantages of endovascular versus direct-contact approaches are presented.

An Efficient Machine Learning-Based Emotional Valence Recognition Approach Towards Wearable EEG

Emotion artificial intelligence (AI) is being increasingly adopted in several industries such as healthcare and education. Facial expressions and tone of speech have been previously considered for emotion recognition, yet they have the drawback of being easily manipulated by subjects to mask their true emotions. Electroencephalography (EEG) has emerged as a reliable and cost-effective method to detect true human emotions. Recently, huge research effort has been put to develop efficient wearable EEG devices to be used by consumers in out of the lab scenarios. In this work, a subject-dependent emotional valence recognition method is implemented that is intended for utilization in emotion AI applications. Time and frequency features were computed from a single time series derived from the Fp1 and Fp2 channels. Several analyses were performed on the strongest valence emotions to determine the most relevant features, frequency bands, and EEG timeslots using the benchmark DEAP dataset. Binary classification experiments resulted in an accuracy of 97.42% using the alpha band, by that outperforming several approaches from literature by ~3-22%. Multiclass classification gave an accuracy of 95.0%. Feature computation and classification required less than 0.1 s. The proposed method thus has the advantage of reduced computational complexity as, unlike most methods in the literature, only two EEG channels were considered. In addition, minimal features concluded from the thorough analyses conducted in this study were used to achieve state-of-the-art performance. The implemented EEG emotion recognition method thus has the merits of being reliable and easily reproducible, making it well-suited for wearable EEG devices.

Supervised Machine Learning and Deep Learning Techniques for Epileptic Seizure Recognition Using EEG Signals-A Systematic Literature Review

Electroencephalography (EEG) is a complicated, non-stationary signal that requires extensive preprocessing and feature extraction approaches to be accurately analyzed. In recent times, Deep learning (DL) has shown great promise in exploiting the characteristics of EEG signals as it can learn relevant features from raw data autonomously. Although studies involving DL have become more common in the last two years, the topic of whether DL truly delivers advantages over conventional Machine learning (ML) methodologies remains unsettled. This study aims to present a detailed overview of the main challenges in the field of seizure detection, prediction, and classification utilizing EEG data, and the approaches taken to solve them using ML and DL methods. A systematic review was conducted surveying peer-reviewed publications published between 2017 and 16 July 2022 using two scientific databases (Web of Science and Scopus) totaling 6822 references after discarding duplicate publications. Whereas 2262 articles were screened based on the title, abstract, and keywords, only 214 were eligible for full-text assessment. A total of 91 papers have been included in this survey after meeting the eligible inclusion and exclusion criteria. The most significant findings from the review are summarized, and several important concepts involving ML and DL for seizure detection, prediction, and classification are discussed in further depth. This review aims to learn more about the different approaches for identifying different types and stages of epileptic seizures, which may then be employed to enhance the lives of epileptic patients in the future, as well as aid experts in the field.

Applications of artificial intelligence in dementia research

More than 50 million older people worldwide are suffering from dementia, and this number is estimated to increase to 150 million by 2050. Greater caregiver burdens and financial impacts on the healthcare system are expected as we wait for an effective treatment for dementia. Researchers are constantly exploring new therapies and screening approaches for the early detection of dementia. Artificial intelligence (AI) is widely applied in dementia research, including machine learning and deep learning methods for dementia diagnosis and progression detection. Computerized apps are also convenient tools for patients and caregivers to monitor cognitive function changes. Furthermore, social robots can potentially provide daily life support or guidance for the elderly who live alone. This review aims to provide an overview of AI applications in dementia research. We divided the applications into three categories according to different stages of cognitive impairment: (1) cognitive screening and training, (2) diagnosis and prognosis for dementia, and (3) dementia care and interventions. There are numerous studies on AI applications for dementia research. However, one challenge that remains is comparing the effectiveness of different AI methods in real clinical settings.

Coordinating human-robot collaboration by EEG-based human intention prediction and vigilance control

In human-robot collaboration scenarios with shared workspaces, a highly desired performance boost is offset by high requirements for human safety, limiting speed and torque of the robot drives to levels which cannot harm the human body. Especially for complex tasks with flexible human behavior, it becomes vital to maintain safe working distances and coordinate tasks efficiently. An established approach in this regard is reactive servo in response to the current human pose. However, such an approach does not exploit expectations of the human's behavior and can therefore fail to react to fast human motions in time. To adapt the robot's behavior as soon as possible, predicting human intention early becomes a factor which is vital but hard to achieve. Here, we employ a recently developed type of brain-computer interface (BCI) which can detect the focus of the human's overt attention as a predictor for impending action. In contrast to other types of BCI, direct projection of stimuli onto the workspace facilitates a seamless integration in workflows. Moreover, we demonstrate how the signal-to-noise ratio of the brain response can be used to adjust the velocity of the robot movements to the vigilance or alertness level of the human. Analyzing this adaptive system with respect to performance and safety margins in a physical robot experiment, we found the proposed method could improve both collaboration efficiency and safety distance.

Comparison of Eye and Face Features on Drowsiness Analysis

Drowsiness is one of the leading causes of traffic accidents. For those who operate large machinery or motor vehicles, incidents due to lack of sleep can cause property damage and sometimes lead to grave consequences of injuries and fatality. This study aims to design learning models to recognize drowsiness through human facial features. In addition, this work analyzes the attentions of individual neurons in the learning model to understand how neural networks interpret drowsiness. For this analysis, gradient-weighted class activation mapping (Grad-CAM) is implemented in the neural networks to display the attention of neurons. The eye and face images are processed separately to the model for the training process. The results initially show that better results can be obtained by delivering eye images alone. The effect of Grad-CAM is also more reasonable using eye images alone. Furthermore, this work proposed a feature analysis method, K-nearest neighbors Sigma (KNN-Sigma), to estimate the homogeneous concentration and heterogeneous separation of the extracted features. In the end, we found that the fusion of face and eye signals gave the best results for recognition accuracy and KNN-sigma. The area under the curve (AUC) of using face, eye, and fusion images are 0.814, 0.897, and 0.935, respectively.

Evaluation of Machine Learning Algorithms for Classification of EEG Signals

In brain–computer interfaces (BCIs), it is crucial to process brain signals to improve the accuracy of the classification of motor movements. Machine learning (ML) algorithms such as artificial neural networks (ANNs), linear discriminant analysis (LDA), decision tree (D.T.), K-nearest neighbor (KNN), naive Bayes (N.B.), and support vector machine (SVM) have made significant progress in classification issues. This paper aims to present a signal processing analysis of electroencephalographic (EEG) signals among different feature extraction techniques to train selected classification algorithms to classify signals related to motor movements. The motor movements considered are related to the left hand, right hand, both fists, feet, and relaxation, making this a multiclass problem. In this study, nine ML algorithms were trained with a dataset created by the feature extraction of EEG signals.The EEG signals of 30 Physionet subjects were used to create a dataset related to movement. We used electrodes C3, C1, CZ, C2, and C4 according to the standard 10-10 placement. Then, we extracted the epochs of the EEG signals and applied tone, amplitude levels, and statistical techniques to obtain the set of features. LabVIEW™2015 version custom applications were used for reading the EEG signals; for channel selection, noise filtering, band selection, and feature extraction operations; and for creating the dataset. MATLAB 2021a was used for training, testing, and evaluating the performance metrics of the ML algorithms. In this study, the model of Medium-ANN achieved the best performance, with an AUC average of 0.9998, Cohen’s Kappa coefficient of 0.9552, a Matthews correlation coefficient of 0.9819, and a loss of 0.0147. These findings suggest the applicability of our approach to different scenarios, such as implementing robotic prostheses, where the use of superficial features is an acceptable option when resources are limited, as in embedded systems or edge computing devices.

Drowsiness Detection Using Ocular Indices from EEG Signal

Drowsiness is one of the main causes of road accidents and endangers the lives of road users. Recently, there has been considerable interest in utilizing features extracted from electroencephalography (EEG) signals to detect driver drowsiness. However, in most of the work performed in this area, the eyeblink or ocular artifacts present in EEG signals are considered noise and are removed during the preprocessing stage. In this study, we examined the possibility of extracting features from the EEG ocular artifacts themselves to perform classification between alert and drowsy states. In this study, we used the BLINKER algorithm to extract 25 blink-related features from a public dataset comprising raw EEG signals collected from 12 participants. Different machine learning classification models, including the decision tree, the support vector machine (SVM), the K-nearest neighbor (KNN) method, and the bagged and boosted tree models, were trained based on the seven selected features. These models were further optimized to improve their performance. We were able to show that features from EEG ocular artifacts are able to classify drowsy and alert states, with the optimized ensemble-boosted trees yielding the highest accuracy of 91.10% among all classic machine learning models.

EEG-Based Index for Timely Detecting User's Drowsiness Occurrence in Automotive Applications

Human errors are widely considered among the major causes of road accidents. Furthermore, it is estimated that more than 90% of vehicle crashes causing fatal and permanent injuries are directly related to mental tiredness, fatigue, and drowsiness of the drivers. In particular, driving drowsiness is recognized as a crucial aspect in the context of road safety, since drowsy drivers can suddenly lose control of the car. Moreover, the driving drowsiness episodes mostly appear suddenly without any prior behavioral evidence. The present study aimed at characterizing the onset of drowsiness in car drivers by means of a multimodal neurophysiological approach to develop a synthetic electroencephalographic (EEG)-based index, able to detect drowsy events. The study involved 19 participants in a simulated scenario structured in a sequence of driving tasks under different situations and traffic conditions. The experimental conditions were designed to induce prominent mental drowsiness in the final part. The EEG-based index, so-called “MDrow index”, was developed and validated to detect the driving drowsiness of the participants. The MDrow index was derived from the Global Field Power calculated in the Alpha EEG frequency band over the parietal brain sites. The results demonstrated the reliability of the proposed MDrow index in detecting the driving drowsiness experienced by the participants, resulting also more sensitive and timely sensible with respect to more conventional autonomic parameters, such as the EyeBlinks Rate and the Heart Rate Variability, and to subjective measurements (self-reports).

Assessment of Combination of Automated Pupillometry and Heart Rate Variability to Detect Driving Fatigue

Objectives

Approximately 20~30% of all traffic accidents are caused by fatigue driving. However, limited practicability remains a barrier for the real application of available techniques to detect driving fatigue. Use of pupillary light reflex (PLR) may be potentially effective for driving fatigue detection.

Methods

A 90 min monotonous simulated driving task was utilized to induce driving fatigue. During the task, PLR measurements were performed at baseline and at an interval of 30 min. Subjective rating scales, heart rate variability (HRV) were monitored simultaneously.

Results

Thirty-two healthy volunteers in China participated in our study. Based on the results of subjective evaluation and behavioral performances, driving fatigue was verified to be successfully induced by a simulated driving task. Significant variations of PLR and HRV parameters were observed, which also showed significant relevance with the change in Karolinska Sleepiness Scale at several timepoints (|r| = 0.55 ~ 0.72, P < 0.001). Furthermore, PLR variations had excellent ability to detect driving fatigue with high sensitivity and specificity, of which maximum constriction velocity variations achieved a sensitivity of 85.00% and specificity of 72.34% for driving fatigue detection, vs. 82.50 and 78.72% with a combination of HRV variations, a nonsignificant difference (AUC = 0.835, 0.872, P > 0.05).

Conclusions

Pupillary light reflex variation may be a potential indicator in the detection of driving fatigue, achieving a comparative performance compared with the combination with heart rate variability. Further work may be involved in developing a commercialized driving fatigue detection system based on pupillary parameters.