Dangerous intersections? A review of studies of fatigue and distraction in the automated vehicle

Driver behavior while using Level 2 vehicle automation: a hybrid naturalistic study

Vehicle automation is becoming more prevalent. Understanding how drivers use this technology and its safety implications is crucial. In a 6-8 week naturalistic study, we leveraged a hybrid naturalistic driving research design to evaluate driver behavior with Level 2 vehicle automation, incorporating unique naturalistic and experimental control conditions. Our investigation covered four main areas: automation usage, system warnings, driving demand, and driver arousal, as well as secondary task engagement. While on the interstate, drivers were advised to engage Level 2 automation whenever they deemed it safe, and they complied by using it over 70% of the time. Interestingly, the frequency of system warnings increased with prolonged use, suggesting an evolving relationship between drivers and the automation features. Our data also revealed that drivers were discerning in their use of automation, opting for manual control under high driving demand conditions. Contrary to common safety concerns, our data indicated no significant rise in driver fatigue or fidgeting when using automation, compared to a control condition. Additionally, observed patterns of engagement in secondary tasks like radio listening and text messaging challenge existing assumptions about automation leading to dangerous driver distraction. Overall, our findings provide new insights into the conditions under which drivers opt to use automation and reveal a nuanced behavioral profile that emerges when automation is in use.

Psychophysiological models of hypovigilance detection: A scoping review

Hypovigilance represents a major contributor to accidents. In operational contexts, the burden of monitoring/managing vigilance often rests on operators. Recent advances in sensing technologies allow for the development of psychophysiology-based (hypo)vigilance prediction models. Still, these models remain scarcely applied to operational situations and need better understanding. The current scoping review provides a state of knowledge regarding psychophysiological models of hypovigilance detection. Records evaluating vigilance measuring tools with gold standard comparisons and hypovigilance prediction performances were extracted from MEDLINE, PsychInfo, and Inspec. Exclusion criteria comprised aspects related to language, non-empirical papers, and sleep studies. The Quality Assessment tool for Diagnostic Accuracy Studies (QUADAS) and the Prediction model Risk Of Bias ASsessment Tool (PROBAST) were used for bias evaluation. Twenty-one records were reviewed. They were mainly characterized by participant selection and analysis biases. Papers predominantly focused on driving and employed several common psychophysiological techniques. Yet, prediction methods and gold standards varied widely. Overall, we outline the main strategies used to assess hypovigilance, their principal limitations, and we discuss applications of these models.

Fatigue and Secondary Media Impacts in the Automated Vehicle: A Multidimensional State Perspective

Safety researchers increasingly recognize the impacts of task-induced fatigue on vehicle driving behavior. The current study (N = 180) explored the use of a multidimensional fatigue measure, the Driver Fatigue Questionnaire (DFQ), to test the impacts of vehicle automation, secondary media use, and driver personality on fatigue states and performance in a driving simulator. Secondary media included a trivia game and a cellphone conversation. Simulated driving induced large-magnitude fatigue states in participants, including tiredness, confusion, coping through self-comforting, and muscular symptoms. Consistent with previous laboratory and field studies, dispositional fatigue proneness predicted increases in state fatigue during the drive, especially tiredness, irrespective of automation level and secondary media. Similar to previous studies, automation slowed braking response to the emergency event following takeover but did not affect fatigue. Secondary media use relieved subjective fatigue and improved lateral control but did not affect emergency braking. Confusion was, surprisingly, associated with faster braking, and tiredness was associated with impaired control of lateral position of the vehicle. These associations were not moderated by the experimental factors. Overall, data support the use of multidimensional assessments of both fatigue symptoms and information-processing components for evaluating safety impacts of interventions for fatigue.

Understanding the domain of driving distraction with knowledge graphs

This paper aims to provide insight into the driving distraction domain systematically on the basis of scientific knowledge graphs. For this purpose, 3,790 documents were taken into consideration after retrieving from Web of Science Core Collection and screening, and two types of knowledge graphs were constructed to demonstrate bibliometric information and domain-specific research content respectively. In terms of bibliometric analysis, the evolution of publication and citation numbers reveals the accelerated development of this domain, and trends of multidisciplinary and global participation could be identified according to knowledge graphs from Vosviewer. In terms of research content analysis, a new framework consisting of five dimensions was clarified, including "objective factors", "human factors", "research methods", "data" and "data science". The main entities of this domain were identified and relations between entities were extracted using Natural Language Processing methods with Python 3.9. In addition to the knowledge graph composed of all the keywords and relationships, entities and relations under each dimension were visualized, and relations between relevant dimensions were demonstrated in the form of heat maps. Furthermore, the trend and significance of driving distraction research were discussed, and special attention was given to future directions of this domain.

Mental workload accumulation effect of mobile phone distraction in L2 autopilot mode

As automated vehicles become more common, there is a need for precise measurement and definition of when and in what ways a driver can use a mobile phone in L2 autonomous driving mode, for how long it can be used, the complexity of the call content, and the accumulated mental workload. This study uses a 2 (driving mode) × 2 (call content complexity) × 6 (driving stage) three-factor mixed experimental design to investigate the effect of these factors on the driver's mental workload by measuring the driver's performance on Detection response tasks, pupil diameter, and EEG components in various brain regions in the alpha band. The results showed that drivers' mental workload levels converge between manual and automatic driving modes as the duration of driving increases, regardless of the level of complexity of the mobile phone conversation. This suggests that mobile phone conversations can also disrupt the driver's cognitive resource balance in L2 automatic driving mode, as it increases mental workload while also impairing the normal functioning of brain functions such as cognitive control, problem solving, and judgment, thereby compromising driving safety.

The moderation effects of task attributes and mental fatigue on post-interruption task performance in a concurrent multitasking environment

In a concurrent multitasking environment, performing many types of tasks increases task complexity, and working long hours makes a person susceptible to mental fatigue. Emerging technologies may lead to more task interruptions. This study examines the effects of task attributes and mental fatigue on interrupted task performance in a concurrent multitasking environment. Thirty-four participants performed the MATB-Ⅱ under eight conditions (two-level task interruption, two-level task complexity, two-level fatigue). The results revealed the significant interaction effects of interruption × task complexity and of interruption × fatigue state. The findings show that more time is required to return to a complex primary task, and there are differences among subtask types. Mental fatigue negatively affects primary task performance, workload, and the resumption lag after an interruption. The findings are explained by the increasing information cues needed to resume complex tasks and the negative effect of fatigue on memory activation.

How drowsiness and distraction can interfere with take-over performance: A systematic and meta-analysis review

Drowsiness and distraction are major factors of road crashes and responsible of>35% of road fatalities. Automated driving could solve or minimize their impact, yet it is also in itself a way to promote them. Previous literature reviews and meta-analysis regarding take-overs during automated driving primarily focused on distraction rather than drowsiness. We thus present a systematic and meta-analysis literature review focused on the effect of distraction and drowsiness on take-over performance. From an initial selection of 1896 articles from databases, we obtained by applying systematic review methodology a total of 58 articles with 42 articles dedicated to distraction and 17 articles related to drowsiness. According to our analysis, we demonstrated that distraction and drowsiness increased the take-over request reaction time (TOR-RT), which could also lead to a reduction of the quality of take-overs. In addition, this longer reaction time was even more important in the case of handheld non-driving related tasks, which is important to consider as phone use is among the most frequent tasks done during automated driving. On a more methodological aspect, we also demonstrated that take-over time budget had a significant effect on TOR-RT. However, it is difficult to estimate to what extend distraction and drowsiness could impact the take-over quality, even if several elements supported safety issues. We underpinned several limits of the current methodologies applied in the study of distraction and drowsiness such as (i) the lack of additional measures related to the take-over quality (e.g., accelerations, collision rate), (ii) the many different methodologies applied to the determination of the TOR-RT (e.g., deactivation by the steering wheel, pedals, button), (iii) the high frequency of take-over requests which can lead to habituation effects, (iv) the lack of control conditions, (v) the fact that the level of drowsiness was relatively low in most studies. We thus highlighted recommendations for a better estimation of the effect of distraction and drowsiness on take-over performance. Further studies should adopt more standardized measures of TOR-RT and additional take-over quality measures, try minimizing the number of take-over requests, and carefully consider the time budget available for the use case since it influences the TOR-RT. Regarding distraction, researchers should consider the impact of tasks requiring handholding items. Concerning drowsiness, further protocols should consider the non-linearity of drowsiness and presence of micro sleeps and favor take-over requests based on drowsiness level protocols rather than on fixed duration protocols.

Effects of partially automated driving on the development of driver sleepiness

The objective of this study was to compare the development of sleepiness during manual driving versus level 2 partially automated driving, when driving on a motorway in Sweden. The hypothesis was that partially automated driving will lead to higher levels of fatigue due to underload. Eighty-nine drivers were included in the study using a 2 × 2 design with the conditions manual versus partially automated driving and daytime (full sleep) versus night-time (sleep deprived). The results showed that night-time driving led to markedly increased levels of sleepiness in terms of subjective sleepiness ratings, blink durations, PERCLOS, pupil diameter and heart rate. Partially automated driving led to slightly higher subjective sleepiness ratings, longer blink durations, decreased pupil diameter, slower heart rate, and higher EEG alpha and theta activity. However, elevated levels of sleepiness mainly arose from the night-time drives when the sleep pressure was high. During daytime, when the drivers were alert, partially automated driving had little or no detrimental effects on driver fatigue. Whether the negative effects of increased sleepiness during partially automated driving can be compensated by the positive effects of lateral and longitudinal driving support needs to be investigated in further studies.

The ANT Executive Control Index: No Evidence for Temporal Decrement

OBJECTIVE

This study tested whether indices of executive control, alertness, and orienting measured with Attention Network Test (ANT) are vulnerable to temporal decrement in performance.

BACKGROUND

Developing the resource theory of sustained attention requires identifying neurocognitive processes vulnerable to decrement. Executive control processes may be prone to impairment in fatigue states. Such processes are also highlighted in alternative theories. Determining the role of executive control in vigilance can both advance theory and contribute to practical countermeasures for decrement in human factors contexts.

METHOD

In Study 1, 80 participants performed the standard ANT for an extended duration of about 55 to 60 min. Study 2 (160 participants) introduced manipulations of trial blocking and stimulus degradation intended to increase resource depletion. Reaction time and accuracy measures were analyzed. Subjective stress and workload were assessed in both studies.

RESULTS

In both studies, the ANT induced levels of subjective workload and task disengagement consistent with previous sustained attention studies. No systematic decrement in any performance measure was observed.

CONCLUSION

Executive control assessed by the ANT is not highly vulnerable to temporal decrement, even when task demands are elevated. Future work should differentiate executive control processes; proactive control may be more implicated in sustained attention decrement than in reactive control.

APPLICATION

Designing systems and interfaces to reduce executive control demands may be generally beneficial but will not directly mitigate temporal performance decrement. Enhancing design guidelines and neuroergonomic methods for monitoring operator attention requires further work to identify key neurocognitive processes for decrement.

A comparative study of state-of-the-art driving strategies for autonomous vehicles

The autonomous vehicle is regarded as a promising technology with the potential to reshape mobility and solve many traffic issues, such as accessibility, efficiency, convenience, and especially safety. Many previous studies on driving strategies mainly focused on the low-level detailed driving behaviors or specific traffic scenarios but lacked the high-level driving strategy studies. Though researchers showed increasing interest in driving strategies, there still has no comprehensive answer on how to proactively implement safe driving. After analyzing several representative driving strategies, we propose three characteristic dimensions that are important to measure driving strategies: preferred objective, risk appetite, and collaborative manner. According to these three characteristic dimensions, we categorize existing driving strategies of autonomous vehicles into four kinds: defensive driving strategies, competitive driving strategies, negotiated driving strategies, and cooperative driving strategies. This paper provides a timely comparative review of these four strategies and highlights the possible directions for improving the high-level driving strategy design.

Driver Fatigue Detection Systems Using Multi-Sensors, Smartphone, and Cloud-Based Computing Platforms: A Comparative Analysis

Internet of things (IoT) cloud-based applications deliver advanced solutions for smart cities to decrease traffic accidents caused by driver fatigue while driving on the road. Environmental conditions or driver behavior can ultimately lead to serious roadside accidents. In recent years, the authors have developed many low-cost, computerized, driver fatigue detection systems (DFDs) to help drivers, by using multi-sensors, and mobile and cloud-based computing architecture. To promote safe driving, these are the most current emerging platforms that were introduced in the past. In this paper, we reviewed state-of-the-art approaches for predicting unsafe driving styles using three common IoT-based architectures. The novelty of this article is to show major differences among multi-sensors, smartphone-based, and cloud-based architectures in multimodal feature processing. We discussed all of the problems that machine learning techniques faced in recent years, particularly the deep learning (DL) model, to predict driver hypovigilance, especially in terms of these three IoT-based architectures. Moreover, we performed state-of-the-art comparisons by using driving simulators to incorporate multimodal features of the driver. We also mention online data sources in this article to test and train network architecture in the field of DFDs on public available multimodal datasets. These comparisons assist other authors to continue future research in this domain. To evaluate the performance, we mention the major problems in these three architectures to help researchers use the best IoT-based architecture for detecting DFDs in a real-time environment. Moreover, the important factors of Multi-Access Edge Computing (MEC) and 5th generation (5G) networks are analyzed in the context of deep learning architecture to improve the response time of DFD systems. Lastly, it is concluded that there is a research gap when it comes to implementing the DFD systems on MEC and 5G technologies by using multimodal features and DL architecture.

Driver Monitoring for a Driver-Centered Design and Assessment of a Merging Assistance System Based on V2V Communications

Merging is one of the most critical scenarios that can be found in road transport. In this maneuver, the driver is subjected to a high mental load due to the large amount of information he handles, while making decisions becomes a crucial issue for their safety and those in adjacent vehicles. In previous works, it was studied how the merging maneuver affected the cognitive load required for driving by means of an eye tracking system, justifying the proposal of a driver assistance system for the merging maneuver on highways. This paper presents a merging assistance system based on communications between vehicles, which allows vehicles to share internal variables of position and speed and is implemented on a mobile device located inside the vehicle. The system algorithm decides where and when the vehicle can start the merging maneuver in safe conditions and provides the appropriate information to the driver. Parameters and driving simulator tests are used for the interface definition to develop the less intrusive and demanding one. Afterward, the system prototype was installed in a real passenger car and tests in real scenarios were conducted with several drivers to assess usability and mental load. Comparisons among alternative solutions are shown and effectiveness is assessed.

Effects of a Motion Seat System on Driver's Passive Task-Related Fatigue: An On-Road Driving Study

Passive task-related (TR) fatigue caused by monotonous driving can negatively affect driving safety by impairing driver alertness and performance. This study aims to evaluate the effectiveness of a motion seat system on the driver’s passive TR fatigue in terms of driving performance, physiological response, and subjective fatigue by using automotive and physiological sensors those applicable to on-road driving environment. Twenty drivers (5 females and 15 males; age = 38.5 ± 12.2) with more than two years of driving experience participated in an on-road experiment with two driving conditions: driving in the static seat condition during the first half of the driving session and then in the static (static–static, SS) or motion seat (static–motion, SM) condition during the second half. The SM condition showed significantly lower passive TR fatigue by 4.4~56.5% compared to the SS condition in terms of the standard deviation of velocity, percentage of eyelid closure rate (PERCLOS), and the ratio of low- to high-frequency power (LF/HF) of electrocardiography signals. The drivers rated significantly lower subjective state changes of overall fatigue, mental fatigue, passive TR fatigue, drowsiness, and decreased concentration in the SM condition than those in the SS condition. The findings of the study support the use of a motion seat system can be an effective countermeasure to reduce passive TR fatigue.

Challenges to Human Drivers in Increasingly Automated Vehicles

OBJECTIVE

We examine the relationships between contemporary progress in on-road vehicle automation and its coherence with an envisioned "autopia" (automobile utopia) whereby the vehicle operation task is removed from all direct human control.

BACKGROUND

The progressive automation of on-road vehicles toward a completely driverless state is determined by the integration of technological advances into the private automobile market; improvements in transportation infrastructure and systems efficiencies; and the vision of future driving as a crash-free enterprise. While there are many challenges to address with respect to automated vehicles concerning the remaining driver role, a considerable amount of technology is already present in vehicles and is advancing rapidly.

METHODS

A multidisciplinary team of experts met to discuss the most critical challenges in the changing role of the driver, and associated safety issues, during the transitional phase of vehicle automation where human drivers continue to have an important but truncated role in monitoring and supervising vehicle operations.

RESULTS

The group endorsed that vehicle automation is an important application of information technology, not only because of its impact on transportation efficiency, but also because road transport is a life critical system in which failures result in deaths and injuries. Five critical challenges were identified: driver independence and mobility, driver acceptance and trust, failure management, third-party testing, and political support.

CONCLUSION

Vehicle automation is not technical innovation alone, but is a social as much as a technological revolution consisting of both attendant costs and concomitant benefits.