Driver Monitoring of Automated Vehicles by Classification of Driver Drowsiness Using a Deep Convolutional Neural Network Trained by Scalograms of ECG Signals

Cardiac indices of driver fatigue across in-lab and on-road studies

Driver fatigue is a major contributor to road accidents. Therefore, driver assistance systems (DAS) that would monitor drivers' states may contribute to road safety. Such monitoring can potentially be achieved with input from ECG indices (e.g., heart rate). We reviewed the empirical literature on responses of cardiac measures to driver fatigue and on detecting fatigue with cardiac indices and classification algorithms. We used meta-analytical methods to explore the pooled effect sizes of different cardiac indices of fatigue, their heterogeneity, and the consistency of their responses across studies. Our large pool of studies (N = 39) allowed us to stratify the results across on-road and simulator studies. We found that despite the large heterogeneity of the effect sizes between the studies, many indices had significant pooled effect sizes across the studies, and more frequently across the on-road studies. We also found that most indices showed consistent responses across both on-road and simulator studies. Regarding the detection accuracy, we found that even on-road classification could have been as accurate as 70% with only 2-min of data. However, we could only find two on-road studies that employed fatigue classification algorithms. Overall, our findings are encouraging with respect to the prospect of using cardiac measures for detecting driver fatigue. Yet, to fully explore this possibility, there is a need for additional on-road studies that would employ a similar set of cardiac indices and detection algorithms, a unified definition of fatigue, and additional levels of fatigue than the two fatigue vs alert states.

Detection of driver drowsiness level using a hybrid learning model based on ECG signals

OBJECTIVES

Fatigue has a considerable impact on the driver's vehicle and even the driver's own operating ability.

METHODS

An intelligent algorithm is proposed for the problem that it is difficult to classify the degree of drowsiness generated by the driver during the driving process. By studying the driver's electrocardiogram (ECG) during driving, two models were established to jointly classify the ECG signals as awake, stress, and fatigue or drowsiness states for drowsiness levels. Firstly, the deep learning method was used to establish the model_1 to predict the drowsiness of the original ECG, and model_2 was developed using the combination of principal component analysis (PCA) and weighted K-nearest neighbor (WKNN) algorithm to classify the heart rate variability characteristics. Then, the drowsiness prediction results of the two models were weighted according to certain rules, and the hybrid learning model combining dilated convolution and bidirectional long short-term memory network with PCA and WKNN algorithm was established, and the mixed model was denoted as DiCNN-BiLSTM and PCA-WKNN (DBPW). Finally, the validity of the DBPW model was verified by simulation of the public database.

RESULTS

The experimental results show that the average accuracy, sensitivity and F1 score of the test model in the dataset containing multiple drivers are 98.79, 98.81, and 98.79 % respectively, and the recognition accuracy for drowsiness or drowsiness state is 99.33 %.

CONCLUSIONS

Using the proposed algorithm, it is possible to identify driver anomalies and provide new ideas for the development of intelligent vehicles.

Driver Drowsiness Multi-Method Detection for Vehicles with Autonomous Driving Functions

The article outlines various approaches to developing a fuzzy decision algorithm designed for monitoring and issuing warnings about driver drowsiness. This algorithm is based on analyzing EOG (electrooculography) signals and eye state images with the aim of preventing accidents. The drowsiness warning system comprises key components that learn about, analyze and make decisions regarding the driver's alertness status. The outcomes of this analysis can then trigger warnings if the driver is identified as being in a drowsy state. Driver drowsiness is characterized by a gradual decline in attention to the road and traffic, diminishing driving skills and an increase in reaction time, all contributing to a higher risk of accidents. In cases where the driver does not respond to the warnings, the ADAS (advanced driver assistance systems) system should intervene, assuming control of the vehicle's commands.

Multi-Level Classification of Driver Drowsiness by Simultaneous Analysis of ECG and Respiration Signals Using Deep Neural Networks

The high number of fatal crashes caused by driver drowsiness highlights the need for developing reliable drowsiness detection methods. An ideal driver drowsiness detection system should estimate multiple levels of drowsiness accurately without intervening in the driving task. This paper proposes a multi-level drowsiness detection system by a deep neural network-based classification system using a combination of electrocardiogram and respiration signals. The proposed method is based on a combination of convolutional neural networks (CNNs) and long short-term memory (LSTM) networks for classifying drowsiness by concurrently using heart rate variability (HRV), power spectral density of HRV, and respiration rate signal as inputs. Two models, a CNN-based model and a hybrid CNN-LSTM-based model were used for multi-level classifications. The performance of the proposed method was evaluated on experimental data collected from 30 subjects in a simulated driving environment. The performance and the results of both models are presented and compared. The best performance for both three-level and five-level drowsiness classifications was achieved by the CNN-LSTM model. The results indicate that the three-level and five-level classifications of drowsiness can be achieved with 91 and 67% accuracy, respectively.

Advances in Automated Driving Systems

Electrification, automation of vehicle control, digitalization and new mobility are the mega trends in automotive engineering and they are strongly connected to each other [...]

Improved Perception of Motorcycles by Simulator-Based Driving Education

Research shows that about half of all motorcycle collisions with other vehicles were caused by the accident opponent, typically a passenger car. This study aimed to assess the effect of simulator training on improving car drivers’ perceptibility of motorcycles and thereby addressing this frequent type of motorcycle accident from the perspective of the initiator. For this purpose, a training program with different methods was conducted and tested in a driving simulator with 80 learner drivers aged between 15 and 27 years, assigned to a control group and three training groups: variable priority, equal priority, and equal priority with warning. The conflict scenarios were determined based on an analysis of motorcycle–car accidents. The variable priority training program resulted in better perceptibility of motorcycles as compared to the equal priority training program and equal priority with warning in two out of four test setups, i.e., urban roads with high contrast between motorcycle and the driving environment and on rural roads with a low contrast. Most participants rated each training method in the driving simulator as useful and would recommend it to other learner drivers. These results are important because they show that simulator training has a positive effect on the motorcycle detection performance of learner drivers. The early perception of motorcycles in car drivers is essential for preventing collisions between cars and motorcycles.

Effects of Automation and Fatigue on Drivers from Various Age Groups

This study explores how drivers are affected by automation when driving in rested and fatigued conditions. Eighty-nine drivers (45 females, 44 males) aged between 20 and 85 years attended driving experiments on separate days, once in a rested and once in a fatigued condition, in a counterbalanced order. The results show an overall effect of automation to significantly reduce drivers’ workload and effort. The automation had different effects, depending on the drivers’ conditions. Differences between the manual and automated mode were larger for the perceived time pressure and effort in the fatigued condition as compared to the rested condition. Frustration was higher during manual driving when fatigued, but also higher during automated driving when rested. Subjective fatigue and the percentage of eye closure (PERCLOS) were higher in the automated mode compared to manual driving mode. PERCLOS differences between the automated and manual mode were higher in the fatigued condition than in the rested condition. There was a significant interaction effect of age and automation on drivers’ PERCLOS. These results are important for the development of driver-centered automation because they show different benefits for drivers of different ages, depending on their condition (fatigued or rested).