1
|
Sun Y, Wang R, Zhang H, Ding N, Ferreira S, Shi X. Driving fingerprinting enhances drowsy driving detection: Tailoring to individual driver characteristics. ACCIDENT; ANALYSIS AND PREVENTION 2024; 208:107812. [PMID: 39423716 DOI: 10.1016/j.aap.2024.107812] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/27/2024] [Revised: 09/27/2024] [Accepted: 10/09/2024] [Indexed: 10/21/2024]
Abstract
BACKGROUND Drowsiness detection is a long-standing concern in preventing drowsiness-related accidents. Inter-individual differences seriously affect drowsiness detection accuracy. However, most existing studies neglected inter-individual differences in measurements' calculation parameters and drowsiness thresholds. Studies without considering inter-individual differences generally used selfsame measurements and drowsiness thresholds for each participant rather than individual optimal measurements and personalized thresholds, which reduces the contribution of measurements and drowsiness detection accuracy at the individual level. Additionally, Driving Fingerprinting (DF) that represents individual traits has not been well applied in drowsiness detection. METHODS We built the Individualized Drowsy driving Detection Model (IDDM) utilizing DF, extracting individual driver's optimal drowsiness characteristics to detect drowsiness. Firstly, we conducted simulated driving experiments with 24 participants (2:1 male-to-female ratio, diverse ages and occupations including professional taxi drivers and graduate students) and collected data on their driving behavior, facial expressions, and the Karolinska Sleepiness Scale (KSS). Secondly, we employed a Two-layer Sliding Time Window (TSTW) to calculate DF measurements. Thirdly, we utilized attribution directed graphs to visualize DF, understand changes in DF with drowsiness, and analyze accident risks. Finally, we used DF matrices to build the IDDM. The IDDM utilized an improved adaptive genetic algorithm to extract the optimal drowsiness characteristics of individual drivers. These DF matrices, constituted by the optimal drowsiness characteristics of individual drivers, were used to train the IDDM based on principal component analysis and radial basis function neural networks. The TSTW strengthened the variation of DF with drowsiness, and the trained IDDM excavated the relationships between DF characteristics and drowsiness, which improved the accuracy and end-to-end timeliness of practical applications. The DF visualization displayed DF variations with drowsiness, theoretically supporting the use of DF to enhance personalized drowsiness driving detection. RESULTS The DF visualization indicated drowsiness caused the distribution and transition probabilities of DF measurements to shift toward unsafe directions, thereby increasing the accident risk and demonstrating the rationality for utilizing DF to recognize drowsiness. The proposed IDDM achieved average accuracy, sensitivity, and specificity of 95.58 %, 96.50 %, and 94.70 %, respectively, outperforming most existing models. The trained IDDM demonstrated an average execution time of 0.0078 s and lower computational costs due to the reduction of PCA and simple RBFNN compared with models based on deep learning, and no requiring physiological data, which reduced invasiveness and enhanced feasibility for real-world implementation. The IDDM still committed challenges like integration with existing systems and concerns about privacy, which should be adjusted in implementations. This study supports the anti-drowsiness warning systems for preventing drowsiness-related accidents and promotes the integration of DF into dangerous driving research.
Collapse
Affiliation(s)
- Yifan Sun
- School of Transportation Engineering, East China Jiaotong University, Nanchang 330013, Jiangxi, China; Intelligent Transportation Systems Research Center, Wuhan University of Technology, Wuhan 430063, Hubei, China; Jiangxi Transportation Institute CO., LTD, Nanchang 330200, Jiangxi, China; Engineering Research Center of Transportation Information and Safety, Ministry of Education, Wuhan 430063, Hubei, China
| | - Rong Wang
- School of Transportation Engineering, East China Jiaotong University, Nanchang 330013, Jiangxi, China
| | - Hui Zhang
- Intelligent Transportation Systems Research Center, Wuhan University of Technology, Wuhan 430063, Hubei, China; Engineering Research Center of Transportation Information and Safety, Ministry of Education, Wuhan 430063, Hubei, China.
| | - Naikan Ding
- Intelligent Transportation Systems Research Center, Wuhan University of Technology, Wuhan 430063, Hubei, China; Engineering Research Center of Transportation Information and Safety, Ministry of Education, Wuhan 430063, Hubei, China
| | - Sara Ferreira
- Research Centre for Territory, Transports and Environment, University of Porto, Porto 4200-465, Portugal
| | - Xiang Shi
- School of Transportation Engineering, Chang'an University, Xi'an 710018, Shanxi, China
| |
Collapse
|
2
|
Mi P, Yan L, Cheng Y, Liu Y, Wang J, Shoukat MU, Yan F, Qin G, Han P, Zhai Y. Driver Cognitive Architecture Based on EEG Signals: A Review. IEEE SENSORS JOURNAL 2024; 24:36261-36286. [DOI: 10.1109/jsen.2024.3471699] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/12/2025]
Affiliation(s)
- Peiwen Mi
- School of Automotive Engineering, Wuhan University of Technology, Wuhan, China
| | - Lirong Yan
- School of Automotive Engineering, Wuhan University of Technology, Wuhan, China
| | - Yu Cheng
- School of Automotive Engineering, Wuhan University of Technology, Wuhan, China
| | - Yan Liu
- School of Automotive Engineering, Wuhan University of Technology, Wuhan, China
| | - Jun Wang
- School of Automotive Engineering, Wuhan University of Technology, Wuhan, China
| | | | - Fuwu Yan
- School of Automotive Engineering, Wuhan University of Technology, Wuhan, China
| | - Guofeng Qin
- Teachers College for Vocational and Education, Guangxi Normal University, Guilin, China
| | - Peng Han
- Xiangyang DAAN Automobile Test Center Corporation Ltd., Xiangyang, China
| | - Yikang Zhai
- Xiangyang DAAN Automobile Test Center Corporation Ltd., Xiangyang, China
| |
Collapse
|
3
|
Nguyen KH, Tran Y, Craig A, Nguyen H, Chai R. Clustering for mitigating subject variability in driving fatigue classification using electroencephalography source-space functional connectivity features. J Neural Eng 2024; 21:066002. [PMID: 39454613 DOI: 10.1088/1741-2552/ad8b6d] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/07/2024] [Accepted: 10/25/2024] [Indexed: 10/28/2024]
Abstract
Objective.While Electroencephalography (EEG)-based driver fatigue state classification models have demonstrated effectiveness, their real-world application remains uncertain. The substantial variability in EEG signals among individuals poses a challenge in developing a universal model, often necessitating retraining with the introduction of new subjects. However, obtaining sufficient data for retraining, especially fatigue data for new subjects, is impractical in real-world settings.Approach.In response to these challenges, this paper introduces a hybrid solution for fatigue detection that combines clustering with classification. Unsupervised clustering groups subjects based on their EEG functional connectivity (FC) in an alert state, and classification models are subsequently applied to each cluster for predicting alert and fatigue states.Main results. Results indicate that classification on clusters achieves higher accuracy than scenarios without clustering, suggesting successful grouping of subjects with similar FC characteristics through clustering, thereby enhancing the classification process.Significance.Furthermore, the proposed hybrid method ensures a practical and realistic retraining process, improving the adaptability and effectiveness of the fatigue detection system in real-world applications.
Collapse
Affiliation(s)
- Khanh Ha Nguyen
- School of Science, Computing and Engineering Technologies, Swinburne University of Technology, Hawthorn, Victoria 3122, Australia
| | - Yvonne Tran
- Macquarie University Hearing, Macquarie University, Sydney, Australia
| | - Ashley Craig
- John Walsh Centre for Rehabilitation Research, Faculty of Medicine and Health, Kolling Institute The University of Sydney, Sydney, Australia
| | - Hung Nguyen
- School of Science, Computing and Engineering Technologies, Swinburne University of Technology, Hawthorn, Victoria 3122, Australia
- Faculty of Engineering & Information Technology, University of Technology Sydney, Sydney, Australia
| | - Rifai Chai
- School of Science, Computing and Engineering Technologies, Swinburne University of Technology, Hawthorn, Victoria 3122, Australia
- Biomedical Engineering Study Program, Physics Department, Faculty of Science and Technology, Universitas Airlangga, Surabaya, Indonesia
- Center for Biomedical Research, Research Organization for Health, National Research and Innovation Agency (BRIN), Bogor, West Java, Indonesia
| |
Collapse
|
4
|
Rezaee K, Nazerian A, Ghayoumi Zadeh H, Attar H, Khosravi M, Kanan M. Smart IoT-driven biosensors for EEG-based driving fatigue detection: A CNN-XGBoost model enhancing healthcare quality. BIOIMPACTS : BI 2024; 15:30586. [PMID: 40256223 PMCID: PMC12008498 DOI: 10.34172/bi.30586] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 06/21/2024] [Revised: 08/27/2024] [Accepted: 09/17/2024] [Indexed: 04/22/2025]
Abstract
Introduction Drowsy driving is a significant contributor to accidents, accounting for 35 to 45% of all crashes. Implementation of an internet of things (IoT) system capable of alerting fatigued drivers has the potential to substantially reduce road fatalities and associated issues. Often referred to as the internet of medical things (IoMT), this system leverages a combination of biosensors, actuators, detectors, cloud-based and edge computing, machine intelligence, and communication networks to deliver reliable performance and enhance quality of life in smart societies. Methods Electroencephalogram (EEG) signals offer potential insights into fatigue detection. However, accurately identifying fatigue from brain signals is challenging due to inter-individual EEG variability and the difficulty of collecting sufficient data during periods of exhaustion. To address these challenges, a novel evolutionary optimization method combining convolutional neural networks (CNNs) and XGBoost, termed CNN-XGBoost Evolutionary Learning, was proposed to improve fatigue identification accuracy. The research explored various subbands of decomposed EEG data and introduced an innovative approach of transforming EEG recordings into RGB scalograms. These scalogram images were processed using a 2D Convolutional Neural Network (2DCNN) to extract essential features, which were subsequently fed into a dense layer for training. Results The resulting model achieved a noteworthy accuracy of 99.80% on a substantial driver fatigue dataset, surpassing existing methods. Conclusion By integrating this approach into an IoT framework, researchers effectively addressed previous challenges and established an artificial intelligence of things (AIoT) infrastructure for critical driving conditions. This IoT-based system optimizes data processing, reduces computational complexity, and enhances overall system performance, enabling accurate and timely detection of fatigue in extreme driving environments.
Collapse
Affiliation(s)
- Khosro Rezaee
- Department of Biomedical Engineering, Meybod University, Meybod, Iran
| | - Asmar Nazerian
- Department of Engineering, Islamic Azad University, Qods Branch, Tehran, Iran
| | - Hossein Ghayoumi Zadeh
- Department of Electrical Engineering, Faculty of Engineering, Vali-e-Asr University of Rafsanjan, Rafsanjan, Iran
| | - Hani Attar
- Faculty of Engineering, Zarqa University, Zarqa, Jordan
| | - Mohamadreza Khosravi
- Department of Medical Physics and Engineering, School of Medicine, Shiraz University of Medical Sciences, Shiraz, Iran
- IT Services, Lidoma Sanat Mehregan Part Ltd., Shiraz 71581, Fars, Iran
- SPUL4PH Lab, Weifang University of Science and Technology, Weifang 262700, China
| | - Mohammad Kanan
- College of Engineering, University of Business and Technology, Jeddah, Saudi Arabia
| |
Collapse
|
5
|
Siddiqui HUR, Akmal A, Iqbal M, Saleem AA, Raza MA, Zafar K, Zaib A, Dudley S, Arambarri J, Castilla ÁK, Rustam F. Ultra-Wide Band Radar Empowered Driver Drowsiness Detection with Convolutional Spatial Feature Engineering and Artificial Intelligence. SENSORS (BASEL, SWITZERLAND) 2024; 24:3754. [PMID: 38931541 PMCID: PMC11207316 DOI: 10.3390/s24123754] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/07/2024] [Revised: 06/01/2024] [Accepted: 06/05/2024] [Indexed: 06/28/2024]
Abstract
Driving while drowsy poses significant risks, including reduced cognitive function and the potential for accidents, which can lead to severe consequences such as trauma, economic losses, injuries, or death. The use of artificial intelligence can enable effective detection of driver drowsiness, helping to prevent accidents and enhance driver performance. This research aims to address the crucial need for real-time and accurate drowsiness detection to mitigate the impact of fatigue-related accidents. Leveraging ultra-wideband radar data collected over five minutes, the dataset was segmented into one-minute chunks and transformed into grayscale images. Spatial features are retrieved from the images using a two-dimensional Convolutional Neural Network. Following that, these features were used to train and test multiple machine learning classifiers. The ensemble classifier RF-XGB-SVM, which combines Random Forest, XGBoost, and Support Vector Machine using a hard voting criterion, performed admirably with an accuracy of 96.6%. Additionally, the proposed approach was validated with a robust k-fold score of 97% and a standard deviation of 0.018, demonstrating significant results. The dataset is augmented using Generative Adversarial Networks, resulting in improved accuracies for all models. Among them, the RF-XGB-SVM model outperformed the rest with an accuracy score of 99.58%.
Collapse
Affiliation(s)
- Hafeez Ur Rehman Siddiqui
- Institute of Computer Science, Khwaja Fareed University of Engineering and Information Technology, Abu Dhabi Road, Rahim Yar Khan 64200, Punjab, Pakistan; (A.A.); (A.A.S.); (M.A.R.); (K.Z.); (A.Z.)
| | - Ambreen Akmal
- Institute of Computer Science, Khwaja Fareed University of Engineering and Information Technology, Abu Dhabi Road, Rahim Yar Khan 64200, Punjab, Pakistan; (A.A.); (A.A.S.); (M.A.R.); (K.Z.); (A.Z.)
| | - Muhammad Iqbal
- Institute of Computer and Software Engineering, Khwaja Fareed University of Engineering and Information Technology, Rahim Yar Khan 64200, Punjab, Pakistan;
| | - Adil Ali Saleem
- Institute of Computer Science, Khwaja Fareed University of Engineering and Information Technology, Abu Dhabi Road, Rahim Yar Khan 64200, Punjab, Pakistan; (A.A.); (A.A.S.); (M.A.R.); (K.Z.); (A.Z.)
| | - Muhammad Amjad Raza
- Institute of Computer Science, Khwaja Fareed University of Engineering and Information Technology, Abu Dhabi Road, Rahim Yar Khan 64200, Punjab, Pakistan; (A.A.); (A.A.S.); (M.A.R.); (K.Z.); (A.Z.)
| | - Kainat Zafar
- Institute of Computer Science, Khwaja Fareed University of Engineering and Information Technology, Abu Dhabi Road, Rahim Yar Khan 64200, Punjab, Pakistan; (A.A.); (A.A.S.); (M.A.R.); (K.Z.); (A.Z.)
| | - Aqsa Zaib
- Institute of Computer Science, Khwaja Fareed University of Engineering and Information Technology, Abu Dhabi Road, Rahim Yar Khan 64200, Punjab, Pakistan; (A.A.); (A.A.S.); (M.A.R.); (K.Z.); (A.Z.)
| | - Sandra Dudley
- Bioengineering Research Centre, School of Engineering, London South Bank University, 103 Borough Road, London SE1 0AA, UK;
| | - Jon Arambarri
- Universidade Internacional do Cuanza, Cuito EN250, Angola; (J.A.); (Á.K.C.)
- Fundación Universitaria Internacional de Colombia, Bogotá 111321, Colombia
- Universidad Internacional Iberoamericana, Campeche 24560, Mexico
| | - Ángel Kuc Castilla
- Universidade Internacional do Cuanza, Cuito EN250, Angola; (J.A.); (Á.K.C.)
- Universidad de La Romana, La Romana 22000, Dominican Republic
- Universidad Europea del Atlántico, Isabel Torres 21, 39011 Santander, Spain
| | - Furqan Rustam
- School of Computing, National College of Ireland, Dublin D01 K6W2, Ireland
| |
Collapse
|
6
|
Moriya H, Machida A, Munakata T, Herai T, Tagai K. Relationships between subjective experience, electroencephalogram, and heart rate variability during a series of cosmetic behavior. Front Psychol 2024; 15:1225737. [PMID: 38807957 PMCID: PMC11130498 DOI: 10.3389/fpsyg.2024.1225737] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2023] [Accepted: 04/09/2024] [Indexed: 05/30/2024] Open
Abstract
Introduction Cosmetic behavior is an important daily activity, especially for women, because it increases visual attractiveness, self-confidence, and positive emotions. However, it is unknown whether a relationship exists between physiological measures and subjective experiences during the series of cosmetic behaviors. Methods Electroencephalograms (EEG) and electrocardiograms (ECG) from thirty female participants who were asked to look in a mirror after applying skincare, as well as base, eye, cheek, and lip makeup were recorded. The price range of cosmetic products was also considered. Subjective evaluations of the skin surface, emotions, and self-confidence were equally measured after looking in the mirror at each step of the cosmetic behavior. Linear mixed models were fitted to examine whether the subjective experience could be explained by the variety of cosmetic products and/or physiological responses. Results The subjective evaluation was summarized into the following three factors using a factor analysis: self-confidence, hedonic perception, and negative emotion. Each theta-band (4-6 Hz) power, alpha-band (8-13 Hz) power of the EEG, and heart rate variability measures were subjected to a principal component analysis separately. The linear mixed models indicated that the variation in the self-confidence score and the negative emotion score was explained only by the steps of cosmetic behaviors, that is, self-confidence increased while negative emotions decreased as the steps of cosmetic behaviors proceeded. On the other hand, the hedonic perception score was explained by the interaction of the steps of cosmetic behaviors and price, indicating that positive tactile perception and positive emotion were higher when luxury cosmetic products were applied than when affordable products were applied. Furthermore, the model indicated that the hedonic perception score was positively associated with the alpha-band power over occipital sites whereas sympathetic nervous system activity was negatively associated with the alpha-band power over lateral central sites. Discussion These results suggest that positive perceptual and emotional experiences are associated with greater attention to somatosensory information than to visual information and sympathetic autonomic nervous system activities. The current results also emphasize the possibility of using physiological measurements as objective measures of cosmetic behavior.
Collapse
Affiliation(s)
| | - Akiko Machida
- MIRAI Technology Institute, Shiseido Co., Ltd., Yokohama, Japan
| | - Taro Munakata
- MIRAI Technology Institute, Shiseido Co., Ltd., Yokohama, Japan
| | | | - Keiko Tagai
- MIRAI Technology Institute, Shiseido Co., Ltd., Yokohama, Japan
| |
Collapse
|
7
|
Nadalizadeh F, Rajabioun M, Feyzi A. Driving fatigue detection based on brain source activity and ARMA model. Med Biol Eng Comput 2024; 62:1017-1030. [PMID: 38117429 DOI: 10.1007/s11517-023-02983-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/30/2023] [Accepted: 11/28/2023] [Indexed: 12/21/2023]
Abstract
Fatigue among drivers is a significant issue in society, and according to organizational reports, it substantially contributes to accidents. So accurate fatigue detection in drivers plays a crucial role in reducing the number of people fatalities or injured resulting from accidents. Several methods are proposed for fatigue driver recognition among which electroencephalography (EEG) is one. This paper proposed a method for fatigue recognition by EEG signals with extracted features from source and sensor spaces. The proposed method starts with preprocessing by applying filtering and artifact rejection. Then source localization methods are applied to EEG signals for active source extraction. A multivariate autoregressive (MVAR) model is fitted to selected sources, and a dual Kalman filter is applied to estimate the source activity and their relationships. Then multivariate autoregressive moving average (ARMA) is fitted between EEG and source activity signals. Features are extracted from model parameters, source relationship matrix, and wavelet transform of EEG and source activity signals. The novelty of this approach is the use of ARMA model between source activities (as input) and EEG signals (as output) and feature extraction from source relations. Relevant features are selected using a combination of RelifF and neighborhood component analysis (NCA) methods. Three classifiers, namely k-nearest neighbor (KNN), support vector machine (SVM), and naive Bayesian (NB) classifiers, are employed to classify drivers. To improve performance, the final label for fatigue detection is calculated by combining these classifiers using the voting method. The results demonstrate that the proposed method accurately recognizes and classifies fatigued drivers with the ensemble classifiers in comparison with other methods.
Collapse
Affiliation(s)
- Fahimeh Nadalizadeh
- Faculty of Electrical and Computer Engineering, Tarbiat Modares University, Tehran, Iran
| | - Mehdi Rajabioun
- Department of Engineering, Mamaghan Branch, Islamic Azad University, Mamaghan, Iran.
| | - Amirreza Feyzi
- Department of Electrical and Computer Engineering, Tabriz University, Tabriz, Iran
| |
Collapse
|
8
|
Wang L, Song F, Zhou TH, Hao J, Ryu KH. EEG and ECG-Based Multi-Sensor Fusion Computing for Real-Time Fatigue Driving Recognition Based on Feedback Mechanism. SENSORS (BASEL, SWITZERLAND) 2023; 23:8386. [PMID: 37896480 PMCID: PMC10611368 DOI: 10.3390/s23208386] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/08/2023] [Revised: 10/07/2023] [Accepted: 10/10/2023] [Indexed: 10/29/2023]
Abstract
A variety of technologies that could enhance driving safety are being actively explored, with the aim of reducing traffic accidents by accurately recognizing the driver's state. In this field, three mainstream detection methods have been widely applied, namely visual monitoring, physiological indicator monitoring and vehicle behavior analysis. In order to achieve more accurate driver state recognition, we adopted a multi-sensor fusion approach. We monitored driver physiological signals, electroencephalogram (EEG) signals and electrocardiogram (ECG) signals to determine fatigue state, while an in-vehicle camera observed driver behavior and provided more information for driver state assessment. In addition, an outside camera was used to monitor vehicle position to determine whether there were any driving deviations due to distraction or fatigue. After a series of experimental validations, our research results showed that our multi-sensor approach exhibited good performance for driver state recognition. This study could provide a solid foundation and development direction for future in-depth driver state recognition research, which is expected to further improve road safety.
Collapse
Affiliation(s)
- Ling Wang
- Department of Computer Science and Technology, School of Computer Science, Northeast Electric Power University, Jilin 132013, China; (L.W.); (F.S.); (J.H.)
| | - Fangjie Song
- Department of Computer Science and Technology, School of Computer Science, Northeast Electric Power University, Jilin 132013, China; (L.W.); (F.S.); (J.H.)
| | - Tie Hua Zhou
- Department of Computer Science and Technology, School of Computer Science, Northeast Electric Power University, Jilin 132013, China; (L.W.); (F.S.); (J.H.)
| | - Jiayu Hao
- Department of Computer Science and Technology, School of Computer Science, Northeast Electric Power University, Jilin 132013, China; (L.W.); (F.S.); (J.H.)
| | - Keun Ho Ryu
- Data Science Laboratory, Faculty of Information Technology, Ton Duc Thang University, Ho Chi Minh City 700000, Vietnam;
- Biomedical Engineering Institute, Chiang Mai University, Chiang Mai 50200, Thailand
- Department of Computer Science, College of Electrical and Computer Engineering, Chungbuk National University, Cheongju 28644, Republic of Korea
| |
Collapse
|
9
|
Matheus A, Samer G, Lu W, Nengah H, Samantha W, Carl SY, Reena M. Machine learning polysomnographically-derived electroencephalography biomarkers predictive of epworth sleepiness scale. Sci Rep 2023; 13:9120. [PMID: 37277423 DOI: 10.1038/s41598-023-34716-5] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2022] [Accepted: 05/05/2023] [Indexed: 06/07/2023] Open
Abstract
Excessive daytime sleepiness (EDS) causes difficulty in concentrating and continuous fatigue during the day. In the clinical setting, the assessment and diagnosis of EDS rely mostly on subjective questionnaires and verbal reports, which compromises the reliability of clinical diagnosis and the ability to robustly discern candidacy for available therapies and track treatment response. In this study, we used a computational pipeline for the automated, rapid, high-throughput, and objective analysis of previously collected encephalography (EEG) data to identify surrogate biomarkers for EDS, thereby defining the quantitative EEG changes in individuals with high Epworth Sleepiness Scale (ESS) (n = 31), compared to a group of individuals with low ESS (n = 41) at the Cleveland Clinic. The epochs of EEG analyzed were extracted from a large overnight polysomnogram registry during the most proximate period of wakefulness. Signal processing of EEG showed significantly different EEG features in the low ESS group compared to high ESS, including enhanced power in the alpha and beta bands and attenuation in the delta and theta bands. Our machine learning (ML) algorithms trained on the binary classification of high vs. low ESS reached an accuracy of 80.2%, precision of 79.2%, recall of 73.8% and specificity of 85.3%. Moreover, we ruled out the effects of confounding clinical variables by evaluating the statistical contribution of these variables on our ML models. These results indicate that EEG data contain information in the form of rhythmic activity that could be leveraged for the quantitative assessment of EDS using ML.
Collapse
Affiliation(s)
- Araujo Matheus
- Sleep Disorders Center, Neurological Institute, Cleveland Clinic Foundation, Cleveland, OH, USA
| | - Ghosn Samer
- Department of Biomedical Engineering, Cleveland Clinic Foundation, Cleveland, OH, USA
| | - Wang Lu
- Quantitative Health Sciences, Lerner Research Institute, Cleveland Clinic Foundation, Cleveland, OH, USA
| | | | - Wells Samantha
- Sleep Disorders Center, Neurological Institute, Cleveland Clinic Foundation, Cleveland, OH, USA
| | - Saab Y Carl
- Department of Biomedical Engineering, Cleveland Clinic Foundation, Cleveland, OH, USA
- Department of Biomedical Engineering, Brown University, Providence, RI, USA
| | - Mehra Reena
- Sleep Disorders Center, Neurological Institute, Cleveland Clinic Foundation, Cleveland, OH, USA.
- Respiratory Institute, Cleveland Clinic Foundation, Cleveland, OH, USA.
- Cardiovascular and Metabolic Sciences, Lerner Research Institute, Cleveland Clinic Foundation, Cleveland, OH, USA.
- Heart and Vascular Institute, Cleveland Clinic Foundation, Cleveland, OH, USA.
| |
Collapse
|
10
|
Dogan S, Tuncer I, Baygin M, Tuncer T. A new hand-modeled learning framework for driving fatigue detection using EEG signals. Neural Comput Appl 2023. [DOI: 10.1007/s00521-023-08491-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/31/2023]
|
11
|
Rahman A, Hriday MBH, Khan R. Computer vision-based approach to detect fatigue driving and face mask for edge computing device. Heliyon 2022; 8:e11204. [PMID: 36325144 PMCID: PMC9619001 DOI: 10.1016/j.heliyon.2022.e11204] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/18/2022] [Revised: 07/10/2022] [Accepted: 10/18/2022] [Indexed: 11/06/2022] Open
Abstract
The fatality of road accidents in this era is alarming. According to WHO, approximately 1.30 million people die each year in road accidents. Road accidents result in significant socioeconomic losses for people, their families, and the country. The integration of modern technologies into automobiles can help to reduce the number of people killed or injured in road accidents. Most of the study and police reports claim that fatigued driving is one of the deadliest factors behind many road accidents. This paper presents a complete embedded system to detect fatigue driving using deep learning, computer vision, and heart rate monitoring with Nvidia Jetson Nano developer kit, Arduino Uno, and AD8232 heart rate module. The proposed system can monitor the driver's real-time situations, then analyze the situation to detect any fatigue conditions and act accordingly. The onboard camera module constantly monitors the driver. The frames are retrieved and analyzed by the core system that uses deep learning and computer vision techniques to verify the situation with Nvidia Jetson Nano. The driver's states are identified using eye and mouth localization approaches from 68 distinct facial landmarks. Experimentally driven threshold data is employed to classify the states. The onboard heart rate module constantly measures the heart rates and detects any fluctuation in BPM related to the drowsiness. This system uses a convolutional neural network-based deep learning framework to include additional face mask detection to cope with the current pandemic situation. The heart rate module works parallelly where the other modules work in a conditional sequential manner to ensure uninterrupted detection. It will detect any sign of drowsiness in real-time and generate the alarm. The system successfully passed the initial lab tests and some actual situation experiments with 97.44% accuracy in fatigue detection and 97.90% accuracy in face mask identification. The automatic device was able to analyze different situations of drivers (different distances of driver from the camera, various lighting conditions, wearing eyeglasses, oblique projection) more precisely and generate an alarm before the accident happened.
Collapse
|
12
|
Li R, Yang D, Fang F, Hong KS, Reiss AL, Zhang Y. Concurrent fNIRS and EEG for Brain Function Investigation: A Systematic, Methodology-Focused Review. SENSORS (BASEL, SWITZERLAND) 2022; 22:s22155865. [PMID: 35957421 PMCID: PMC9371171 DOI: 10.3390/s22155865] [Citation(s) in RCA: 76] [Impact Index Per Article: 25.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 06/20/2022] [Revised: 07/27/2022] [Accepted: 07/30/2022] [Indexed: 05/29/2023]
Abstract
Electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) stand as state-of-the-art techniques for non-invasive functional neuroimaging. On a unimodal basis, EEG has poor spatial resolution while presenting high temporal resolution. In contrast, fNIRS offers better spatial resolution, though it is constrained by its poor temporal resolution. One important merit shared by the EEG and fNIRS is that both modalities have favorable portability and could be integrated into a compatible experimental setup, providing a compelling ground for the development of a multimodal fNIRS-EEG integration analysis approach. Despite a growing number of studies using concurrent fNIRS-EEG designs reported in recent years, the methodological reference of past studies remains unclear. To fill this knowledge gap, this review critically summarizes the status of analysis methods currently used in concurrent fNIRS-EEG studies, providing an up-to-date overview and guideline for future projects to conduct concurrent fNIRS-EEG studies. A literature search was conducted using PubMed and Web of Science through 31 August 2021. After screening and qualification assessment, 92 studies involving concurrent fNIRS-EEG data recordings and analyses were included in the final methodological review. Specifically, three methodological categories of concurrent fNIRS-EEG data analyses, including EEG-informed fNIRS analyses, fNIRS-informed EEG analyses, and parallel fNIRS-EEG analyses, were identified and explained with detailed description. Finally, we highlighted current challenges and potential directions in concurrent fNIRS-EEG data analyses in future research.
Collapse
Affiliation(s)
- Rihui Li
- Center for Interdisciplinary Brain Sciences Research, Department of Psychiatry and Behavioral Sciences, Stanford University School of Medicine, Stanford, CA 94305, USA
- Department of Biomedical Engineering, University of Houston, Houston, TX 77004, USA
| | - Dalin Yang
- School of Mechanical Engineering, Pusan National University, Pusan 43241, Korea
- Mallinckrodt Institute of Radiology, Washington University School of Medicine in St. Louis, 4515 McKinley Avenue, St. Louis, MO 63110, USA
| | - Feng Fang
- Department of Biomedical Engineering, University of Houston, Houston, TX 77004, USA
| | - Keum-Shik Hong
- School of Mechanical Engineering, Pusan National University, Pusan 43241, Korea
| | - Allan L. Reiss
- Center for Interdisciplinary Brain Sciences Research, Department of Psychiatry and Behavioral Sciences, Stanford University School of Medicine, Stanford, CA 94305, USA
| | - Yingchun Zhang
- Department of Biomedical Engineering, University of Houston, Houston, TX 77004, USA
| |
Collapse
|
13
|
Diabetic Retinopathy Detection from Fundus Images of the Eye Using Hybrid Deep Learning Features. Diagnostics (Basel) 2022; 12:diagnostics12071607. [PMID: 35885512 PMCID: PMC9324358 DOI: 10.3390/diagnostics12071607] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/11/2022] [Revised: 06/25/2022] [Accepted: 06/28/2022] [Indexed: 12/03/2022] Open
Abstract
Diabetic Retinopathy (DR) is a medical condition present in patients suffering from long-term diabetes. If a diagnosis is not carried out at an early stage, it can lead to vision impairment. High blood sugar in diabetic patients is the main source of DR. This affects the blood vessels within the retina. Manual detection of DR is a difficult task since it can affect the retina, causing structural changes such as Microaneurysms (MAs), Exudates (EXs), Hemorrhages (HMs), and extra blood vessel growth. In this work, a hybrid technique for the detection and classification of Diabetic Retinopathy in fundus images of the eye is proposed. Transfer learning (TL) is used on pre-trained Convolutional Neural Network (CNN) models to extract features that are combined to generate a hybrid feature vector. This feature vector is passed on to various classifiers for binary and multiclass classification of fundus images. System performance is measured using various metrics and results are compared with recent approaches for DR detection. The proposed method provides significant performance improvement in DR detection for fundus images. For binary classification, the proposed modified method achieved the highest accuracy of 97.8% and 89.29% for multiclass classification.
Collapse
|
14
|
Guo W, Xu G, Wang Y. Horizontal and vertical features fusion network based on different brain regions for emotion recognition. Knowl Based Syst 2022. [DOI: 10.1016/j.knosys.2022.108819] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
|
15
|
Billones R, Liwang JK, Butler K, Graves L, Saligan LN. Dissecting the fatigue experience: A scoping review of fatigue definitions, dimensions, and measures in non-oncologic medical conditions. Brain Behav Immun Health 2021; 15:100266. [PMID: 34589772 PMCID: PMC8474156 DOI: 10.1016/j.bbih.2021.100266] [Citation(s) in RCA: 51] [Impact Index Per Article: 12.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/21/2021] [Revised: 04/29/2021] [Accepted: 05/03/2021] [Indexed: 11/30/2022] Open
Abstract
INTRODUCTION Fatigue is a prevalent and potentially debilitating symptom that impacts the health-related quality-of-life of individuals diagnosed with acute and chronic medical conditions. Yet, its etiologic mechanism is not fully understood. Additionally, the assessment and determination of the clinical meaning of fatigue and its multidimensionality may vary by medical condition. METHODS A scoping literature review was conducted to investigate how fatigue is defined and measured, including its dimensions, in non-oncologic medical conditions. The PubMed database was searched using keywords. RESULTS Overall, 8376 articles were screened at the title/abstract levels, where 293 articles were chosen for full-text review that mentioned fatigue or included fatigue measures. The review of the full text excluded 246 articles that did not assess at least one fatigue dimension using validated questionnaires and clinical tests. The final set included 47 articles. Physical fatigue was the most assessed fatigue dimension and the Multidimensional Fatigue Inventory was the most widely used questionnaire to assess fatigue in this review. LIMITATIONS This review was limited by including only English-language publications and using PubMed as the sole database for the search. CONCLUSIONS This review affirms that fatigue is a multidimensional construct, agnostic of medical condition, and that individual fatigue dimensions can be measured by validated clinical measures. Future research should focus on expanding the repertoire of clinical measures to assess specific fatigue dimensions.
Collapse
Affiliation(s)
| | | | - Kierra Butler
- National Institute of Nursing Research, National Institutes of Health, Bethesda, MD, USA
| | - Letitia Graves
- National Institute of Nursing Research, National Institutes of Health, Bethesda, MD, USA
| | - Leorey N. Saligan
- National Institute of Nursing Research, National Institutes of Health, Bethesda, MD, USA
| |
Collapse
|
16
|
Zhang Y, Zhu S. Study on the Effect of Driving Time on Fatigue of Grassland Road Based on EEG. JOURNAL OF HEALTHCARE ENGINEERING 2021; 2021:9957828. [PMID: 34306602 PMCID: PMC8285183 DOI: 10.1155/2021/9957828] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/23/2021] [Revised: 05/08/2021] [Accepted: 06/17/2021] [Indexed: 12/04/2022]
Abstract
In order to study the change law of the fatigue degree of grassland expressway drivers over time, this paper takes the semidesert grassland landscape of Xilinhot city as the experimental environment and takes the provincial highway S101 (K278-K424) as an example to design an actual driving test. Taking Urumqi, Inner Mongolia Autonomous Region, as the experimental section, combined with the Biopac MP150 multichannel physiological instrument and its auxiliary knowledge software and mathematical statistics methods, the relationship between EEG and time was studied. The test results show that the primary fatigue factor F 1 and the secondary fatigue factor F 2 can summarize the fatigue law characterized by 96.42% of EEG information. During 130 minutes of driving on the prairie highway, the periods of high fatigue were 105 minutes and 125 minutes, respectively. Driving fatigue can be divided into three stages over time: 5-65 min fatigue-free stage, 70-85 min fatigue transition stage, and 90-130 min fatigue stage. Fatigue changes over time. The law follows the Gaussian function and the sine function.
Collapse
Affiliation(s)
- Yule Zhang
- College of Energy and Transportation Engineering, Inner Mongolia Agricultural University, Hohhot 010000, China
| | - Shoulin Zhu
- College of Energy and Transportation Engineering, Inner Mongolia Agricultural University, Hohhot 010000, China
| |
Collapse
|