A Comprehensive Survey of Driving Monitoring and Assistance Systems

Use of multiple indicators multiple causes (MIMIC) method to investigate quantitative inference in socioeconomic determinants on motorcyclist stress

Research has shown that driving-related stress plays a significant role in causing traffic accidents, either directly or indirectly. Motorcyclists often engage in risky driving behaviors due to elevated stress levels. This study investigates the influence of socioeconomic factors on driving stress among motorcyclists. Data were gathered from 50 participants, with heart rate (HR) recorded using the Polar Vantage V2 device. Heart rate variability (HRV) was analyzed in both time and frequency domains using Kubios HRV software. The study employed the Multiple Indicators Multiple Causes (MIMIC) model to explore the associations between socioeconomic factors and driving stress. The results indicate that variables such as age, gender, education level, occupation, income, driving experience, and travel purpose significantly affect stress levels across both HRV domains. These findings highlight the importance of addressing motorcyclist stress through targeted interventions, including educational programs and policy measures that regulate driving duration. Such strategies are particularly vital in developing countries to reduce stress and improve road safety. This research provides a foundation for developing practical solutions aimed at minimizing driving stress and enhancing the well-being of motorcyclists in high-risk environments.•A MIMIC model was applied to analyze the relationship between stress variables in the time and frequency domains based on HRV data.•The model identified significant causal relationships, emphasizing the pivotal role of socioeconomic factors in influencing motorcyclists' driving stress.•The model demonstrated strong statistical performance with key indicators: chi-square = 38.749, GFI = 0.958, CFI = 0.982, AGFI = 0.893, TLI = 0.961, and RMSEA = 0.057, confirming its robustness and reliability.

Development of a framework for risky driving scenario identification, individual risk assessment, and group risk differences estimation using naturalistic driving data from the i-DREAMS project

Driver-related factors, such as driving style and traffic offenses, are key contributors to road crashes, with driving risk varying substantially among individuals. Accurate assessment of individual driving risk and identification of high-risk driver characteristics are essential to reducing road crashes. Despite numerous studies on driving risk assessment, most rely solely on the frequency of single-threshold events, making them insufficiently comprehensive. Moreover, these studies neglect the repetitive nature of driving scenarios and differences in exposure, leading to imprecise assessments when using distance traveled as a measure of exposure. To address these shortcomings, we collected 18 weeks of naturalistic driving data from 100 participants (50 from the UK, 50 from Belgium) and developed a framework for assessing individual driving risk, consisting of three parts: (1) identification of risky driving scenarios, (2) assessment of individual driving risks, and (3) analysis of group risk differences to identify high-risk driver characteristics. Risky driving scenarios were characterized by critical events with high risk propensity and high heterogeneity among individual driving risks. Driving scenario indicators were developed that measure risk propensity and heterogeneity, enabling risk assessments based on the probability of critical events occurring in such scenarios. Individual driving risk was measured by the weighted probability of multi-threshold events (WPMTE) in risky driving scenarios and adjusted for differences in driving exposure. WPMTE provides a comprehensive and precise assessment of individual driving risks, aiding in the identification of high-risk drivers. Finally, statistical tests revealed significantly higher risks for young drivers (19-30) compared to middle-aged (46-60) and elderly drivers (61-79), as well as higher risks for Belgian drivers compared to UK drivers. These findings inform the development of tailored safety education and proactive interventions, promoting safer driving behaviors and reducing crash rates.

DUIncoder: Learning to Detect Driving Under the Influence Behaviors from Various Normal Driving Data

Driving Under the Influence (DUI) has emerged as a significant threat to public safety in recent years. Despite substantial efforts to effectively detect DUI, the inherent risks associated with acquiring DUI-related data pose challenges in meeting the data requirements for training. To address this issue, we propose DUIncoder, which is an unsupervised framework designed to learn exclusively from normal driving data across diverse scenarios to detect DUI behaviors and provide explanatory insights. DUIncoder aims to address the challenge of collecting DUI data by leveraging diverse normal driving data, which can be readily and continuously obtained from daily driving. Experiments on simulator data show that DUIncoder achieves detection performance superior to that of supervised learning methods which require additional DUI data. Moreover, its generalization capabilities and adaptability to incremental data demonstrate its potential for enhanced real-world applicability.

Effects of non-driving related postures on takeover performance during conditionally automated driving

In spite of the advancement in driving automation, driver's ability to resume manual control from a conditionally automated vehicle appears as a safety concern. Understanding the impact of various non-driving related tasks (NDRT) on takeover performance is crucial for the development of advanced driver assistance systems. The aim of this study was to investigate how the takeover performance was impacted by non-driving related postures when engaging in different NDRTs. A same takeover scenario with SAE automation level 3 requiring emergency braking was deployed for all test conditions on a static driving simulator under different time budgets. Reaction times, pedal movement and takeover quality were collected from 54 drivers (mean age 34.5 years, 27 females) taking over from two reference postures and 21 non-driving related postures. Results showed that drivers reacted faster given a shorter time budget. Non-driving related postures were found to prolong the takeover time and deteriorate the takeover quality. In particular, the postures with abnormal right foot positions, big trunk deviations and both hands occupation much lowered motoric readiness. Results also revealed that when driver's upper body was engaged in abnormal postures, driver's lower body would react slower, and vice versa. In addition, drivers' takeover performance was affected by their individual reaction capacity, which demonstrated a range of variation. Theoretical and practical implications of the findings are discussed.

Technologies for detecting and monitoring drivers' states: A systematic review

Driver fatigue or drowsiness detection techniques can significantly enhance road safety measures and reduce traffic accidents. These approaches used different sensor technologies to acquire the human physiological and behavioral characteristics to investigate the driver's vigilance state. Although the driver's vigilance detection technique has attracted significant interest recently, few studies have been conducted to review it systematically. These studies provide a thorough overview of the most advanced driver vigilance detection method available today in terms of sensor technology for scholars and specialists. This research is geared towards achieving three main objectives. Firstly, it aims to systematically gather, evaluate, and synthesize information from previous research published between 2014 and May 2024 on driver's state and driving sensors and their implementation on detection algorithms. It aims to provide a thorough review of the present state of research on wearable and unwearable sensor technology for driver fatigue detection, focusing on reporting experimental results in this field. This information will be necessary for experts and scientists seeking to advance their knowledge in this field. Lastly, the research aims to identify gaps in knowledge that require further investigation and recommend future research directions to help address these gaps. This way, it will contribute to the advancement of the field and provide beneficial insights for future researchers.

Driver Fatigue Detection Using Heart Rate Variability Features from 2-Minute Electrocardiogram Signals While Accounting for Sex Differences

Traffic accidents due to fatigue account for a large proportion of road fatalities. Based on simulated driving experiments with drivers recruited from college students, this paper investigates the use of heart rate variability (HRV) features to detect driver fatigue while considering sex differences. Sex-independent and sex-specific differences in HRV features between alert and fatigued states derived from 2 min electrocardiogram (ECG) signals were determined. Then, decision trees were used for driver fatigue detection using the HRV features of either all subjects or those of only males or females. Nineteen, eighteen, and thirteen HRV features were significantly different (Mann-Whitney U test, p < 0.01) between the two mental states for all subjects, males, and females, respectively. The fatigue detection models for all subjects, males, and females achieved classification accuracies of 86.3%, 94.8%, and 92.0%, respectively. In conclusion, sex differences in HRV features between drivers' mental states were found according to both the statistical analysis and classification results. By considering sex differences, precise HRV feature-based driver fatigue detection systems can be developed. Moreover, in contrast to conventional methods using HRV features from 5 min ECG signals, our method uses HRV features from 2 min ECG signals, thus enabling more rapid driver fatigue detection.

Integrating visual large language model and reasoning chain for driver behavior analysis and risk assessment

Driver behavior is a critical factor in driving safety, making the development of sophisticated distraction classification methods essential. Our study presents a Distracted Driving Classification (DDC) approach utilizing a visual Large Language Model (LLM), named the Distracted Driving Language Model (DDLM). The DDLM introduces whole-body human pose estimation to isolate and analyze key postural features-head, right hand, and left hand-for precise behavior classification and better interpretability. Recognizing the inherent limitations of LLMs, particularly their lack of logical reasoning abilities, we have integrated a reasoning chain framework within the DDLM, allowing it to generate clear, reasoned explanations for its assessments. Tailored specifically with relevant data, the DDLM demonstrates enhanced performance, providing detailed, context-aware evaluations of driver behaviors and corresponding risk levels. Notably outperforming standard models in both zero-shot and few-shot learning scenarios, as evidenced by tests on the 100-Driver dataset, the DDLM stands out as an advanced tool that promises significant contributions to driving safety by accurately detecting and analyzing driving distractions.

Object Detection, Recognition, and Tracking Algorithms for ADASs-A Study on Recent Trends

Advanced driver assistance systems (ADASs) are becoming increasingly common in modern-day vehicles, as they not only improve safety and reduce accidents but also aid in smoother and easier driving. ADASs rely on a variety of sensors such as cameras, radars, lidars, and a combination of sensors, to perceive their surroundings and identify and track objects on the road. The key components of ADASs are object detection, recognition, and tracking algorithms that allow vehicles to identify and track other objects on the road, such as other vehicles, pedestrians, cyclists, obstacles, traffic signs, traffic lights, etc. This information is then used to warn the driver of potential hazards or used by the ADAS itself to take corrective actions to avoid an accident. This paper provides a review of prominent state-of-the-art object detection, recognition, and tracking algorithms used in different functionalities of ADASs. The paper begins by introducing the history and fundamentals of ADASs followed by reviewing recent trends in various ADAS algorithms and their functionalities, along with the datasets employed. The paper concludes by discussing the future of object detection, recognition, and tracking algorithms for ADASs. The paper also discusses the need for more research on object detection, recognition, and tracking in challenging environments, such as those with low visibility or high traffic density.

Driver Drowsiness Detection: A Machine Learning Approach on Skin Conductance

The majority of car accidents worldwide are caused by drowsy drivers. Therefore, it is important to be able to detect when a driver is starting to feel drowsy in order to warn them before a serious accident occurs. Sometimes, drivers are not aware of their own drowsiness, but changes in their body signals can indicate that they are getting tired. Previous studies have used large and intrusive sensor systems that can be worn by the driver or placed in the vehicle to collect information about the driver's physical status from a variety of signals that are either physiological or vehicle-related. This study focuses on the use of a single wrist device that is comfortable for the driver to wear and appropriate signal processing to detect drowsiness by analyzing only the physiological skin conductance (SC) signal. To determine whether the driver is drowsy, the study tests three ensemble algorithms and finds that the Boosting algorithm is the most effective in detecting drowsiness with an accuracy of 89.4%. The results of this study show that it is possible to identify when a driver is drowsy using only signals from the skin on the wrist, and this encourages further research to develop a real-time warning system for early detection of drowsiness.

Vehicle and Driver Monitoring System Using On-Board and Remote Sensors

This paper presents an integrated monitoring system for the driver and the vehicle in a single case of study easy to configure and replicate. On-board vehicle sensors and remote sensors are combined to model algorithms for estimating polluting emissions, fuel consumption, driving style and driver's health. The main contribution of this paper is the analysis of interactions among the above monitored features highlighting the influence of the driver in the vehicle performance and vice versa. This analysis was carried out experimentally using one vehicle with different drivers and routes and implemented on a mobile application. Compared to commercial driver and vehicle monitoring systems, this approach is not customized, uses classical sensor measurements, and is based on simple algorithms that have been already proven but not in an interactive environment with other algorithms. In the procedure design of this global vehicle and driver monitoring system, a principal component analysis was carried out to reduce the variables used in the training/testing algorithms with objective to decrease the transfer data via Bluetooth between the used devices: a biometric wristband, a smartphone and the vehicle's central computer. Experimental results show that the proposed vehicle and driver monitoring system predicts correctly the fuel consumption index in 84%, the polluting emissions 89%, and the driving style 89%. Indeed, interesting correlation results between the driver's heart condition and vehicular traffic have been found in this analysis.

Correlation Analysis of In-Vehicle Sensors Data and Driver Signals in Identifying Driving and Driver Behaviors

Today's cars have dozens of sensors to monitor vehicle performance through different systems, most of which communicate via vehicular networks (CAN). Many of these sensors can be used for applications other than the original ones, such as improving the driver experience or creating new safety tools. An example is monitoring variables that describe the driver's behavior. Interactions with the pedals, speed, and steering wheel, among other signals, carry driving characteristics. However, not always all variables related to these interactions are available in all vehicles; for example, the excursion of the brake pedal. Using an acquisition module, data from the in-vehicle sensors were obtained from the CAN bus, the brake pedal (externally instrumented), and the driver's signals (instrumented with an inertial sensor and electromyography of their leg), to observe the driver and car information and evaluate the correlation hypothesis between these data, as well as the importance of the brake pedal signal not usually available in all car models. Different sets of sensors were evaluated to analyze the performance of three classifiers when analyzing the driver's driving mode. It was found that there are superior results in classifying identity or behavior when driver signals are included. When the vehicle and driver attributes were used, hits above 0.93 were obtained in the identification of behavior and 0.96 in the identification of the driver; without driver signals, accuracy was more significant than 0.80 in identifying behavior. The results show a good correlation between vehicle data and data obtained from the driver, suggesting that further studies may be promising to improve the accuracy of rates based exclusively on vehicle characteristics, both for behavior identification and driver identification, thus allowing practical applications in embedded systems for local signaling and/or storing information about the driving mode, which is important for logistics companies.

Risky-Driving-Image Recognition Based on Visual Attention Mechanism and Deep Learning

Risky driving behavior seriously affects the driver's ability to react, execute and judge, which is one of the major causes of traffic accidents. The timely and accurate identification of the driving status of drivers is particularly important, since drivers can quickly adjust their driving status to avoid safety accidents. In order to further improve the identification accuracy, this paper proposes a risky-driving image-recognition system based on the visual attention mechanism and deep-learning technology to identify four types of driving status images including normal driving, driving while smoking, driving while drinking and driving while talking. With reference to ResNet, we build four deep-learning models with different depths and embed the proposed visual attention blocks into the image-classification model. The experimental results indicate that the classification accuracy of the ResNet models with lower depth can exceed the ResNet models with higher depth by embedding the visual attention modules, while there is no significant change in model complexity, which could improve the model recognition accuracy without reducing the recognition efficiency.

Single Camera Face Position-Invariant Driver's Gaze Zone Classifier Based on Frame-Sequence Recognition Using 3D Convolutional Neural Networks

Estimating the driver's gaze in a natural real-world setting can be problematic for different challenging scenario conditions. For example, faces will undergo facial occlusions, illumination, or various face positions while driving. In this effort, we aim to reduce misclassifications in driving situations when the driver has different face distances regarding the camera. Three-dimensional Convolutional Neural Networks (CNN) models can make a spatio-temporal driver's representation that extracts features encoded in multiple adjacent frames that can describe motions. This characteristic may help ease the deficiencies of a per-frame recognition system due to the lack of context information. For example, the front, navigator, right window, left window, back mirror, and speed meter are part of the known common areas to be checked by drivers. Based on this, we implement and evaluate a model that is able to detect the head direction toward these regions having various distances from the camera. In our evaluation, the 2D CNN model had a mean average recall of 74.96% across the three models, whereas the 3D CNN model had a mean average recall of 87.02%. This result show that our proposed 3D CNN-based approach outperforms a 2D CNN per-frame recognition approach in driving situations when the driver's face has different distances from the camera.

How did COVID-19 impact driving behaviors and crash Severity? A multigroup structural equation modeling

Risky driving behaviors such as speeding and failing to signal have been witnessed more frequently during the COVID-19 pandemic, resulting in higher rates of severe crashes. This study aims to investigate how the COVID-19 pandemic impacts the likelihood of severe crashes via changing driving behaviors. Multigroup structural equation modeling (SEM) is used to capture the complex interrelationships between crash injury severity, the context of COVID-19, driving behaviors, and other risk factors for two different groups, i.e., highways and non-highways. The SEM constructs two latent variables, namely aggressiveness and inattentiveness, which are indicated by risk driving behaviors such as speeding, drunk driving, and distraction. One great advantage of SEM is that the measurement of latent variables and interrelationship modeling can be achieved simultaneously in one statistical estimation procedure. Group differences between highways and non-highways are tested using different equality constraints and multigroup SEM with equal regressions can deliver the augmented performance. The smaller severity threshold for the highway group indicates that it is more likely that a crash could involve severe injuries on highways as compared to those on non-highways. Results suggest that aggressiveness and inattentiveness of drivers increased significantly after the outbreak of COVID-19, leading to a higher likelihood of severe crashes. Failing to account for the indirect effect of COVID-19 via changing driving behaviors, the conventional probit model suggests an insignificant impact of COVID-19 on crash severity. Findings of this study provide insights into the effect of changing driving behaviors on safety during disruptive events like COVID-19.

Urethane-Foam-Embedded Silicon Pressure Sensors including Stress-Concentration Packaging Structure for Driver Posture Monitoring

We propose urethane-foam-embedded silicon pressure sensors, including a stress-concentration packaging structure, for integration into a car seat to monitor the driver’s cognitive state, posture, and driving behavior. The technical challenges of embedding silicon pressure sensors in urethane foam are low sensitivity due to stress dispersion of the urethane foam and non-linear sensor response caused by the non-uniform deformation of the foam. Thus, the proposed package structure includes a cover to concentrate the force applied over the urethane foam and frame to eliminate this non-linear stress because the outer edge of the cover receives large non-linear stress concentration caused by the geometric non-linearity of the uneven height of the sensor package and ground substrate. With this package structure, the pressure sensitivity of the sensors ranges from 0 to 10 kPa. The sensors also have high linearity with a root mean squared error of 0.049 N in the linear regression of the relationship between applied pressure and sensor output, and the optimal frame width is more than 2 mm. Finally, a prototype 3 × 3 sensor array included in the proposed package structure detects body movements, which will enable the development of sensor-integrated car seats.

In-Cabin Monitoring System for Autonomous Vehicles

In this paper, we have demonstrated a robust in-cabin monitoring system (IMS) for safety, security, surveillance, and monitoring, including privacy concerns for personal and shared autonomous vehicles (AVs). It consists of a set of monitoring cameras and an onboard device (OBD) equipped with artificial intelligence (AI). Hereafter, this combination of a camera and an OBD is referred to as the AI camera. We have investigated the issues for mobility services in higher levels of autonomous driving, what needs to be monitored, how to monitor, etc. Our proposed IMS is an on-device AI system that indigenously has improved the privacy of the users. Furthermore, we have enlisted the essential actions to be considered in an IMS and developed an appropriate database (DB). Our DB consists of multifaced scenarios important for monitoring the in-cabin of the higher-level AVs. Moreover, we have compared popular AI models applied for object and occupant recognition. In addition, our DB is available on request to support the research on the development of seamless monitoring of the in-cabin higher levels of autonomous driving for the assurance of safety and security.

Threats Detection during Human-Computer Interaction in Driver Monitoring Systems

This paper presents an approach and a case study for threat detection during human–computer interaction, using the example of driver–vehicle interaction. We analyzed a driver monitoring system and identified two types of users: the driver and the operator. The proposed approach detects possible threats for the driver. We present a method for threat detection during human–system interactions that generalizes potential threats, as well as approaches for their detection. The originality of the method is that we frame the problem of threat detection in a holistic way: we build on the driver–ITS system analysis and generalize existing methods for driver state analysis into a threat detection method covering the identified threats. The developed reference model of the operator–computer interaction interface shows how the driver monitoring process is organized, and what information can be processed automatically, and what information related to the driver behavior has to be processed manually. In addition, the interface reference model includes mechanisms for operator behavior monitoring. We present experiments that included 14 drivers, as a case study. The experiments illustrated how the operator monitors and processes the information from the driver monitoring system. Based on the case study, we clarified that when the driver monitoring system detected the threats in the cabin and notified drivers about them, the number of threats was significantly decreased.

[Design and validation of a computer-based task for the induction of a mental workload spectrum]

As part of the driver's cab 4.0 project funded by the BMBF, an adaptive human-machine interface for agricultural machinery, which detects the current level of mental workload by analysing physiological data is being developed. For this purpose, an experimental task is designed and evaluated, which can induce a mental workload spectrum from little to very strenuous in humans. In three laboratory studies, mental workload is generated by a monitoring activity, with varying difficulty levels. The complexity of the activity is increased by a visual and/or an auditory secondary task. Subjectively perceived mental workload is evaluated by using the Rating Scale Mental Effort, collecting reaction times and error rates. The studies with N = 17, N = 8 and N = 21 participants show that a dynamic combination of main and secondary tasks can induce significantly different degrees of workload (F (2.40) = 54,834, p < 0.001).Practical Relevance: The experimental task developed in this paper will be used to design a measuring system for mental workload based on physiological indicators for combine harvesters. In low-workload situations (e.g. automated harvesting) additional recommendations for action should be proposed by the system. During high workload sections excessive demands on the user should be avoided by only showing the information necessary to carry out the task at hand.

Design of an Estimator Using the Artificial Neural Network Technique to Characterise the Braking of a Motor Vehicle

Automatic systems are increasingly being applied in the automotive industry to improve driving safety and passenger comfort, reduce traffic and increase energy efficiency. The objective of this work is focused on improving the automatic brake assistance systems of motor vehicles trying to imitate human behaviour but correcting possible human errors such as distractions, lack of visibility or time reaction. The proposed system can optimise the intensity of the braking according to the available distance to carry out the manoeuvre and the vehicle speed to be as less aggressive as possible, thus giving priority to the comfort of the driver. A series of tests are carried out in this work with a vehicle instrumented with sensors that provide real-time information about the braking system. The data obtained experimentally during the dynamic tests are used to design an estimator using the Artificial Neural Network (ANN) technique. This information makes it possible to characterise all braking situations based on the pressure of the brake circuit, the type of manoeuvre and the test speed. Thanks to this ANN, it is possible to estimate the requirements of the braking system in real driving situations and carry out the manoeuvres automatically. Experiments and simulations verified the proposed method for the estimation of braking pressure in real deceleration scenarios.

Hybrid Interacting Multiple Model Filtering for Improving the Reliability of Radar-Based Forward Collision Warning Systems

Automotive forward collision warning (FCW) systems based on radar sensors attracted widespread attention in recent years. To achieve a reliable FCW, it is essential to accurately estimate the position and velocity of a preceding vehicle. To this end, this study proposed a novel estimation algorithm, which is a hybrid of interacting multiple model probabilistic data association (IMM-PDA) and finite impulse response (FIR) filters. Although the IMM-PDA filter is one of the most successful algorithm for tracking a maneuvering target in clutters, it sometimes exhibits divergence owing to modeling errors. In this study, the divergent IMM-PDA filter in the novel algorithm was reset and recovered using an assisting FIR filter. Consequently, this enabled reliable estimation for FCW. The improved reliability of the proposed algorithm was demonstrated through the simulation of preceding vehicle tracking using automotive radars.

Pose Estimation of Driver’s Head Panning Based on Interpolation and Motion Vectors under a Boosting Framework

Over the last decade, a driver’s distraction has gained popularity due to its increased significance and high impact on road accidents. Various factors, such as mood disorder, anxiety, nervousness, illness, loud music, and driver’s head rotation, contribute significantly to causing a distraction. Many solutions have been proposed to address this problem; however, various aspects of it are still unresolved. The study proposes novel geometric and spatial scale-invariant features under a boosting framework for detecting a driver’s distraction due to the driver’s head panning. These features are calculated using facial landmark detection algorithms, including the Active Shape Model (ASM) and Boosted Regression with Markov Networks (BoRMaN). The proposed approach is compared with six existing state-of-the-art approaches using four benchmark datasets, including DrivFace dataset, Boston University (BU) dataset, FT-UMT dataset, and Pointing’04 dataset. The proposed approach outperforms the existing approaches achieving an accuracy of 94.43%, 92.08%, 96.63%, and 83.25% on standard datasets.

MLife: a lite framework for machine learning lifecycle initialization

Machine learning (ML) lifecycle is a cyclic process to build an efficient ML system. Though a lot of commercial and community (non-commercial) frameworks have been proposed to streamline the major stages in the ML lifecycle, they are normally overqualified and insufficient for an ML system in its nascent phase. Driven by real-world experience in building and maintaining ML systems, we find that it is more efficient to initialize the major stages of ML lifecycle first for trial and error, followed by the extension of specific stages to acclimatize towards more complex scenarios. For this, we introduce a simple yet flexible framework, MLife, for fast ML lifecycle initialization. This is built on the fact that data flow in MLife is in a closed loop driven by bad cases, especially those which impact ML model performance the most but also provide the most value for further ML model development—a key factor towards enabling enterprises to fast track their ML capabilities. Better yet, MLife is also flexible enough to be easily extensible to more complex scenarios for future maintenance. For this, we introduce two real-world use cases to demonstrate that MLife is particularly suitable for ML systems in their early phases.

Evaluation of a Fatigue Detector Using Eye Closure-Associated Indicators Acquired from Truck Drivers in a Simulator Study

This paper presents a camera-based prototype sensor for detecting fatigue and drowsiness in drivers, which are common causes of road accidents. The evaluation of the detector operation involved eight professional truck drivers, who drove the truck simulator twice—i.e., when they were rested and drowsy. The Fatigue Symptoms Scales (FSS) questionnaire was used to assess subjectively perceived levels of fatigue, whereas the percentage of eye closure time (PERCLOS), eye closure duration (ECD), and frequency of eye closure (FEC) were selected as eye closure-associated fatigue indicators, determined from the images of drivers’ faces captured by the sensor. Three alternative models for subjective fatigue were used to analyse the relationship between the raw score of the FSS questionnaire, and the eye closure-associated indicators were estimated. The results revealed that, in relation to the subjective assessment of fatigue, PERCLOS is a significant predictor of the changes observed in individual subjects during the performance of tasks, while ECD reflects the individual differences in subjective fatigue occurred both between drivers and in individual drivers between the ‘rested’ and ‘drowsy’ experimental conditions well. No relationship between the FEC index and the FSS state scale was found.

Survey and Synthesis of State of the Art in Driver Monitoring

Road vehicle accidents are mostly due to human errors, and many such accidents could be avoided by continuously monitoring the driver. Driver monitoring (DM) is a topic of growing interest in the automotive industry, and it will remain relevant for all vehicles that are not fully autonomous, and thus for decades for the average vehicle owner. The present paper focuses on the first step of DM, which consists of characterizing the state of the driver. Since DM will be increasingly linked to driving automation (DA), this paper presents a clear view of the role of DM at each of the six SAE levels of DA. This paper surveys the state of the art of DM, and then synthesizes it, providing a unique, structured, polychotomous view of the many characterization techniques of DM. Informed by the survey, the paper characterizes the driver state along the five main dimensions—called here “(sub)states”—of drowsiness, mental workload, distraction, emotions, and under the influence. The polychotomous view of DM is presented through a pair of interlocked tables that relate these states to their indicators (e.g., the eye-blink rate) and the sensors that can access each of these indicators (e.g., a camera). The tables factor in not only the effects linked directly to the driver, but also those linked to the (driven) vehicle and the (driving) environment. They show, at a glance, to concerned researchers, equipment providers, and vehicle manufacturers (1) most of the options they have to implement various forms of advanced DM systems, and (2) fruitful areas for further research and innovation.

Exploration of Driver Posture Monitoring Using Pressure Sensors with Lower Resolution

Pressure sensors are good candidates for measuring driver postural information, which is indicative for identifying driver’s intention and seating posture. However, monitoring systems based on pressure sensors must overcome the price barriers in order to be practically feasible. This study, therefore, was dedicated to explore the possibility of using pressure sensors with lower resolution for driver posture monitoring. We proposed pressure features including center of pressure, contact area proportion, and pressure ratios to recognize five typical trunk postures, two typical left foot postures, and three typical right foot postures. The features from lower-resolution mapping were compared with those from high-resolution Xsensor pressure mats on the backrest and seat pan. We applied five different supervised machine-learning techniques to recognize the postures of each body part and used leave-one-out cross-validation to evaluate their performance. A uniform sampling method was used to reduce number of pressure sensors, and five new layouts were tested by using the best classifier. Results showed that the random forest classifier outperformed the other classifiers with an average classification accuracy of 86% using the original pressure mats and 85% when only 8% of the pressure sensors were available. This study demonstrates the feasibility of using fewer pressure sensors for driver posture monitoring and suggests research directions for better sensor designs.

The Role of 5G Technologies in a Smart City: The Case for Intelligent Transportation System

A smart city is an urban area that collects data using various electronic methods and sensors. Smart cities rely on Information and Communication Technologies (ICT) and aim to improve the quality of services by managing public resources and focusing on comfort, maintenance, and sustainability. The fifth generation (5G) of wireless mobile communication enables a new kind of communication network to connect everyone and everything. 5G will profoundly impact economies and societies as it will provide the necessary communication infrastructure required by various smart city applications. Intelligent Transporting System (ITS) is one of the many smart city applications that can be realized via 5G technology. The paper aims to discuss the impact and implications of 5G on ITS from various dimensions. Before this, the paper presents an overview of the technological context and the economic benefits of the 5G and how key vertical industries will be affected in a smart city, i.e., energy, healthcare, manufacturing, entertainment, and automotive and public transport. Afterward, 5G for ITS is introduced in more detail.

On the Benefits of Using Object-Oriented Programming for the Objective Evaluation of Vehicle Dynamic Performance in Concurrent Simulations

Assessing passenger cars’ dynamic performance is a critical aspect for car industries, due to its impact on the overall vehicle safety evaluation and the subjective nature of the involved handling and comfort metrics. Accordingly, ISO standards, such as ISO 4138 and ISO 3888, define several specific driving tests to assess vehicle dynamics performance objectively. Consequently, proper evaluation of the dynamic behaviour requires measuring several physical quantities, including accelerations, speed, and linear and angular displacements obtained after instrumenting a vehicle with multiple sensors. This experimental activity is highly demanding in terms of hardware costs, and it is also significantly time-consuming. Several approaches can be considered for reducing vehicle development time. In particular, simulation software can be exploited to predict the approximate behaviour of a vehicle using virtual scenarios. Moreover, motion platforms and detail-scalable numerical vehicle models are widely implemented for the purpose. This paper focuses on a customized simulation environment developed in C++, which exploits the advantages of object-oriented programming. The presented framework strives to perform concurrent simulations of vehicles with different characteristics such as mass, tyres, engine, suspension, and transmission systems. Within the proposed simulation framework, we adopted a hierarchical and modular representation. Vehicles are modelled by a 14 degree-of-freedom (DOF) full-vehicle model, capable of capturing the dynamics and complemented by a set of scalable-detail models for the remaining sub-systems such as tyre, engine, and steering system. Furthermore, this paper proposes the usage of autonomous virtual drivers for a more objective evaluation of vehicle dynamic performances. Moreover, to further evaluate our simulator architecture’s efficiency and assess the achieved level of concurrency, we designed a benchmark able to analyse the scaling of the performances with respect to the number of different vehicles during the same simulation. Finally, the paper reports the proposed simulation environment’s scalability resulting from a set of different and varying driving scenarios.

The Role of Eye Tracking Technology in Assessing Older Driver Safety

A growing body of literature is focused on the use of eye tracking (ET) technology to understand the association between objective visual parameters and higher order brain processes such as cognition. One of the settings where this principle has found practical utility is in the area of driving safety.

METHODS

We reviewed the literature to identify the changes in ET parameters with older adults and neurodegenerative disease.

RESULTS

This narrative review provides a brief overview of oculomotor system anatomy and physiology, defines common eye movements and tracking variables that are typically studied, explains the most common methods of eye tracking measurements during driving in simulation and in naturalistic settings, and examines the association of impairment in ET parameters with advanced age and neurodegenerative disease.

CONCLUSION

ET technology is becoming less expensive, more portable, easier to use, and readily applicable in a variety of clinical settings. Older adults and especially those with neurodegenerative disease may have impairments in visual search parameters, placing them at risk for motor vehicle crashes. Advanced driver assessment systems are becoming more ubiquitous in newer cars and may significantly reduce crashes related to impaired visual search, distraction, and/or fatigue.

Intelligent Driving Assistant Based on Road Accident Risk Map Analysis and Vehicle Telemetry

Through the application of intelligent systems in driver assistance systems, the experience of traveling by road has become much more comfortable and safe. In this sense, this paper then reports the development of an intelligent driving assistant, based on vehicle telemetry and road accident risk map analysis, whose responsibility is to alert the driver in order to avoid risky situations that may cause traffic accidents. In performance evaluations using real cars in a real environment, the on-board intelligent assistant reproduced real-time audio-visual alerts according to information obtained from both telemetry and road accident risk map analysis. As a result, an intelligent assistance agent based on fuzzy reasoning was obtained, which supported the driver correctly in real-time according to the telemetry data, the vehicle environment and the principles of secure driving practices and transportation regulation laws. Experimental results and conclusions emphasizing the advantages of the proposed intelligent driving assistant in the improvement of the driving task are presented.

Gaze and Eye Tracking: Techniques and Applications in ADAS

Tracking drivers’ eyes and gazes is a topic of great interest in the research of advanced driving assistance systems (ADAS). It is especially a matter of serious discussion among the road safety researchers’ community, as visual distraction is considered among the major causes of road accidents. In this paper, techniques for eye and gaze tracking are first comprehensively reviewed while discussing their major categories. The advantages and limitations of each category are explained with respect to their requirements and practical uses. In another section of the paper, the applications of eyes and gaze tracking systems in ADAS are discussed. The process of acquisition of driver’s eyes and gaze data and the algorithms used to process this data are explained. It is explained how the data related to a driver’s eyes and gaze can be used in ADAS to reduce the losses associated with road accidents occurring due to visual distraction of the driver. A discussion on the required features of current and future eye and gaze trackers is also presented.

Moving Object Detection Based on Optical Flow Estimation and a Gaussian Mixture Model for Advanced Driver Assistance Systems

Most approaches for moving object detection (MOD) based on computer vision are limited to stationary camera environments. In advanced driver assistance systems (ADAS), however, ego-motion is added to image frames owing to the use of a moving camera. This results in mixed motion in the image frames and makes it difficult to classify target objects and background. In this paper, we propose an efficient MOD algorithm that can cope with moving camera environments. In addition, we present a hardware design and implementation results for the real-time processing of the proposed algorithm. The proposed moving object detector was designed using hardware description language (HDL) and its real-time performance was evaluated using an FPGA based test system. Experimental results demonstrate that our design achieves better detection performance than existing MOD systems. The proposed moving object detector was implemented with 13.2K logic slices, 104 DSP48s, and 163 BRAM and can support real-time processing of 30 fps at an operating frequency of 200 MHz.