Effects of scheduled manual driving on drowsiness and response to take over request: A simulator study towards understanding drivers in automated driving

Age-related differences in takeover performance: A comparative analysis of older and younger drivers in prolonged partially automated driving

OBJECTIVE

Vehicle automation technologies have the potential to address the mobility needs of older adults. However, age-related cognitive declines may pose new challenges for older drivers when they are required to take back or "takeover" control of their automated vehicle. This study aims to explore the impact of age on takeover performance under partially automated driving conditions and the interaction effect between age and voluntary non-driving-related tasks (NDRTs) on takeover performance.

METHOD

A total of 42 older drivers (M = 65.5 years, SD = 4.4) and 40 younger drivers (M = 37.2 years, SD = 4.5) participated in this mixed-design driving simulation experiment (between subjects: age [older drivers vs. younger drivers] and NDRT engagement [road monitoring vs. voluntary NDRTs]; within subjects: hazardous event occurrence time [7.5th min vs. 38.5th min]).

RESULTS

Older drivers exhibited poorer visual exploration performance (i.e., longer fixation point duration and smaller saccade amplitude), lower use of advanced driving assistance systems (ADAS; e.g., lower percentage of time adaptive cruise control activated [ACCA]) and poorer takeover performance (e.g., longer takeover time, larger maximum resulting acceleration, and larger standard deviation of lane position) compared to younger drivers. Furthermore, older drivers were less likely to experience driving drowsiness (e.g., lower percentage of time the eyes are fully closed and Karolinska Sleepiness Scale levels); however, this advantage did not compensate for the differences in takeover performance with younger drivers. Older drivers had lower NDRT engagement (i.e., lower percentage of fixation time on NDRTs), and NDRTs did not significantly affect their drowsiness but impaired takeover performance (e.g., higher collision rate, longer takeover time, and larger maximum resulting acceleration).

CONCLUSIONS

These findings indicate the necessity of addressing the impaired takeover performance due to cognitive decline in older drivers and discourage them from engaging in inappropriate NDRTs, thereby reducing their crash risk during automated driving.

Exploring the Timing of Disengagement From Nondriving Related Tasks in Scheduled Takeovers With Pre-Alerts: An Analysis of Takeover-Related Measures

OBJECTIVES

This study aimed to investigate drivers' disengagement from nondriving related tasks (NDRT) during scheduled takeovers and to evaluate its impact on takeover performance.

BACKGROUND

During scheduled takeovers, drivers typically have sufficient time to prepare. However, inadequate disengagement from NDRTs can introduce safety risks.

METHOD

Participants experienced scheduled takeovers using a driving simulator, undergoing two conditions, with and without an NDRT. We assessed their takeover performance and monitored their NDRT disengagement from visual, cognitive, and physical perspectives.

RESULTS

The study examined three NDRT disengagement timings (DTs): DT1 (disengaged before the takeover request), DT2 (disengaged after the request but before taking over), and DT3 (not disengaged). The impact of NDRT on takeover performance varied depending on DTs. Specifically, DT1 demonstrated no adverse effects; DT2 impaired takeover time, while DT3 impaired both takeover time and quality. Additionally, participants who displayed DT1 exhibited longer eye-off-NDRT duration and a higher eye-off-NDRT count during the prewarning stage compared to those with DT2 and DT3.

CONCLUSION

Drivers can benefit from earlier disengagement from NDRTs, demonstrating resilience to the adverse effects of NDRTs on takeover performance. The disengagement of cognition is often delayed compared to that of eyes and hands, potentially leading to DT3. Moreover, visual disengagement from NDRTs during the prewarning stage could distinguish DT1 from the other two.

APPLICATION

Our study emphasizes considering NDRT disengagement in designing systems for scheduled takeovers. Measures should be taken to promote early disengagement, facilitate cognitive disengagement, and employ visual disengagement during the prewarning period as predictive indicators of DTs.

Is driving experience all that matters? Drivers' takeover performance in conditionally automated driving

INTRODUCTION

In conditionally automated driving, drivers are allowed to engage in non-driving related tasks (NDRTs) and are occasionally requested to take over vehicle control in situations that the automation system cannot handle. Drivers may not be able to adequately perform such requests if they have limited driving experience. This study investigates the influence of driving experience on takeover performance in conditionally automated driving.

METHOD

Nineteen subjects participated in this driving simulator study. The NDRTs consisted of three tasks: writing business emails (working condition), watching videos (entertaining condition), and taking a break with eyes closed (resting condition). These three NDRTs require drivers to invest high, moderate, and low levels of mental workload, respectively. The duration of engagement in each NDRT before a takeover request (TOR) was either 5 minutes (short interval) or 30 minutes (long interval).

RESULTS

Drivers' driving experience and performance during the control period are highly correlated with their TOR performance. Furthermore, the type and duration of NDRT influence TOR performance, and inexperienced drivers exhibit poorer TOR performance than experienced drivers.

CONCLUSIONS AND PRACTICAL APPLICATIONS

These findings have relevance for the types of NDRTs that ought to be permitted during automated driving, the design of automated driving systems, and the formulation of regulations regarding the responsible use of automated vehicles.

The impact of non-driving related tasks on the development of driver sleepiness and takeover performances in prolonged automated driving

INTRODUCTION

Vehicle automation is thought to improve road safety since numerous accidents are caused by human error. However, the lack of active involvement and monotonous driving environments due to automation may contribute to drivers' passive fatigue and sleepiness. Previous research indicated that non-driving related tasks (NDRTs) were beneficial in maintaining drivers' arousal levels but detrimental to takeover performance.

METHOD

A 3·2 mixed design (between subjects: driving condition; within subjects: takeover orders) simulator experiment was conducted to explore the development of driver sleepiness in prolonged automated driving context and the effect of NDRTs on driver sleepiness development, and to further evaluate the impact of driver sleepiness and NDRTs on takeover performance. Sixty-three participants were randomly assigned to three driving conditions, each lasting 60 min: automated driving while performing driving environment monitoring task; visual NDRTs task; and visual NDRTs with scheduled driving environment monitoring task. Two hazardous events occurring at about the 5th and 55th min needed to be handled during the respective driving.

RESULTS

Drivers performing monitoring tasks had a faster development of driver sleepiness than drivers in the other two conditions in terms of both subjective and objective indicators. Takeover performance of drivers performing monitoring task were undermined due to driver sleepiness in terms of braking and steering reaction times, the time between saccade latency and braking or steering reaction times, and so forth. Additionally, NDRTs impaired the drivers' takeover ability in terms of saccade latency, max braking pedal input, max steering velocity, minimum time to collision, and so forth. This study shows that NDRTs with scheduled road environment monitoring task improve takeover performance during prolonged automated driving by helping to maintain driver alertness.

PRACTICAL APPLICATIONS

Findings from this work provide some technical assistance in the development of driver sleepiness monitoring systems for conditionally automated vehicles.

EOG-Based Human–Computer Interface: 2000–2020 Review

Electro-oculography (EOG)-based brain–computer interface (BCI) is a relevant technology influencing physical medicine, daily life, gaming and even the aeronautics field. EOG-based BCI systems record activity related to users’ intention, perception and motor decisions. It converts the bio-physiological signals into commands for external hardware, and it executes the operation expected by the user through the output device. EOG signal is used for identifying and classifying eye movements through active or passive interaction. Both types of interaction have the potential for controlling the output device by performing the user’s communication with the environment. In the aeronautical field, investigations of EOG-BCI systems are being explored as a relevant tool to replace the manual command and as a communicative tool dedicated to accelerating the user’s intention. This paper reviews the last two decades of EOG-based BCI studies and provides a structured design space with a large set of representative papers. Our purpose is to introduce the existing BCI systems based on EOG signals and to inspire the design of new ones. First, we highlight the basic components of EOG-based BCI studies, including EOG signal acquisition, EOG device particularity, extracted features, translation algorithms, and interaction commands. Second, we provide an overview of EOG-based BCI applications in the real and virtual environment along with the aeronautical application. We conclude with a discussion of the actual limits of EOG devices regarding existing systems. Finally, we provide suggestions to gain insight for future design inquiries.

How drowsiness and distraction can interfere with take-over performance: A systematic and meta-analysis review

Drowsiness and distraction are major factors of road crashes and responsible of>35% of road fatalities. Automated driving could solve or minimize their impact, yet it is also in itself a way to promote them. Previous literature reviews and meta-analysis regarding take-overs during automated driving primarily focused on distraction rather than drowsiness. We thus present a systematic and meta-analysis literature review focused on the effect of distraction and drowsiness on take-over performance. From an initial selection of 1896 articles from databases, we obtained by applying systematic review methodology a total of 58 articles with 42 articles dedicated to distraction and 17 articles related to drowsiness. According to our analysis, we demonstrated that distraction and drowsiness increased the take-over request reaction time (TOR-RT), which could also lead to a reduction of the quality of take-overs. In addition, this longer reaction time was even more important in the case of handheld non-driving related tasks, which is important to consider as phone use is among the most frequent tasks done during automated driving. On a more methodological aspect, we also demonstrated that take-over time budget had a significant effect on TOR-RT. However, it is difficult to estimate to what extend distraction and drowsiness could impact the take-over quality, even if several elements supported safety issues. We underpinned several limits of the current methodologies applied in the study of distraction and drowsiness such as (i) the lack of additional measures related to the take-over quality (e.g., accelerations, collision rate), (ii) the many different methodologies applied to the determination of the TOR-RT (e.g., deactivation by the steering wheel, pedals, button), (iii) the high frequency of take-over requests which can lead to habituation effects, (iv) the lack of control conditions, (v) the fact that the level of drowsiness was relatively low in most studies. We thus highlighted recommendations for a better estimation of the effect of distraction and drowsiness on take-over performance. Further studies should adopt more standardized measures of TOR-RT and additional take-over quality measures, try minimizing the number of take-over requests, and carefully consider the time budget available for the use case since it influences the TOR-RT. Regarding distraction, researchers should consider the impact of tasks requiring handholding items. Concerning drowsiness, further protocols should consider the non-linearity of drowsiness and presence of micro sleeps and favor take-over requests based on drowsiness level protocols rather than on fixed duration protocols.

Age-related effects of executive function on takeover performance in automated driving

The development of highly automated vehicles can meet elderly drivers' mobility needs; however, worse driving performance after a takeover request (TOR) is frequently found, especially regarding non-driving related tasks (NDRTs). This study aims to detect the correlation between takeover performance and underlying cognitive factors comprising a set of higher order cognitive processes including executive functions. Thirty-five young and 35 elderly participants were tested by computerized cognitive tasks and simulated driving tasks to evaluate their executive functions and takeover performance. Performance of n-back tasks, Simon tasks, and task switching were used to evaluate updating, inhibition, and shifting components of executive functions by principal component analysis. The performance of lane changing after TOR was measured using the standard deviation of the steering wheel angle and minimum time-to-collision (TTC). Differences between age groups and NDRT engagement were assessed by two-way mixed analysis of variance. Older participants had significantly lower executive function ability and were less stable and more conservative when engaged in NDRT. Furthermore, a significant correlation between executive function and lateral driving stability was found. These findings highlight the interaction between age-related differences in executive functions and takeover performance; thus, provide implications for designing driver screening tests or human-machine interfaces.

Effects of gender, age, experience, and practice on driver reaction and acceptance of traffic jam chauffeur systems

This study conducted a driving simulation experiment to compare four automated driving systems (ADS) designs during lane change demanding traffic situations on highways while accounting for the drivers’ gender, age, experience, and practice. A lane-change maneuver was required when the automated vehicle approaches traffic congestion on the left-hand lane. ADS-1 can only reduce the speed to synchronize with the congestion. ADS-2 reduces the speed and issues an optional request to intervene, advising the driver to change lanes manually. ADS-3 offers to overtake the congestion autonomously if the driver approves it. ADS-4 overtakes the congestion autonomously without the driver’s approval. Results of drivers’ reaction, acceptance, and trust indicated that differences between ADS designs increase when considering the combined effect of drivers’ demographic factors more than the individual effect of each factor. However, the more ADS seems to have driver-like capacities, the more impact of demographic factors is expected. While preliminary, these findings may help us understand how ADS users’ behavior can differ based on the interaction between human demographic factors and system design.

Context–aware assistive driving: an overview of techniques for mitigating the risks of driver in real-time driving environment

Purpose

This study aims to analyze driver risks in the driving environment. A complete analysis of context aware assistive driving techniques. Context awareness in assistive driving by probabilistic modeling techniques. Advanced techniques using Spatio-temporal techniques, computer vision and deep learning techniques.

Design/methodology/approach

Autonomous vehicles have been aimed to increase driver safety by introducing vehicle control from the driver to Advanced Driver Assistance Systems (ADAS). The core objective of these systems is to cut down on road accidents by helping the user in various ways. Early anticipation of a particular action would give a prior benefit to the driver to successfully handle the dangers on the road. In this paper, the advancements that have taken place in the use of multi-modal machine learning for assistive driving systems are surveyed. The aim is to help elucidate the recent progress and techniques in the field while also identifying the scope for further research and improvement. The authors take an overview of context-aware driver assistance systems that alert drivers in case of maneuvers by taking advantage of multi-modal human processing to better safety and drivability.

Findings

There has been a huge improvement and investment in ADAS being a key concept for road safety. In such applications, data is processed and information is extracted from multiple data sources, thus requiring training of machine learning algorithms in a multi-modal style. The domain is fast gaining traction owing to its applications across multiple disciplines with crucial gains.

Research limitations/implications

The research is focused on deep learning and computer vision-based techniques to generate a context for assistive driving and it would definitely adopt by the ADAS manufacturers.

Social implications

As context-aware assistive driving would work in real-time and it would save the lives of many drivers, pedestrians.

Originality/value

This paper provides an understanding of context-aware deep learning frameworks for assistive driving. The research is mainly focused on deep learning and computer vision-based techniques to generate a context for assistive driving. It incorporates the latest state-of-the-art techniques using suitable driving context and the driver is alerted. Many automobile manufacturing companies and researchers would refer to this study for their enhancements.

No Difference in Arousal or Cognitive Demands Between Manual and Partially Automated Driving: A Multi-Method On-Road Study

INTRODUCTION

Partial driving automation is not always reliable and requires that drivers maintain readiness to take over control and manually operate the vehicle. Little is known about differences in drivers' arousal and cognitive demands under partial automation and how it may make it difficult for drivers to transition from automated to manual modes. This research examined whether there are differences in drivers' arousal and cognitive demands during manual versus partial automation driving.

METHOD

We compared arousal (using heart rate) and cognitive demands (using the root mean square of successive differences in normal heartbeats; RMSSD, and Detection Response Task; DRT) while 39 younger (M = 28.82 years) and 32 late-middle-aged (M = 52.72 years) participants drove four partially automated vehicles (Cadillac, Nissan Rogue, Tesla, and Volvo) on interstate highways. If compared to manual driving, drivers' arousal and cognitive demands were different under partial automation, then corresponding differences in heart rate, RMSSD, and DRT would be expected. Alternatively, if drivers' arousal and cognitive demands were similar in manual and partially automated driving, no difference in the two driving modes would be expected.

RESULTS

Results suggest no significant differences in heart rate, RMSSD, or DRT reaction time performance between manual and partially automated modes of driving for either younger or late-middle-aged adults across the four test vehicles. A Bayes Factor analysis suggested that heart rate, RMSSD, and DRT data showed extreme evidence in favor of the null hypothesis.

CONCLUSION

This novel study conducted on real roads with a representative sample provides important evidence of no difference in arousal and cognitive demands. Younger and late-middle-aged motorists who are new to partial automation are able to maintain arousal and cognitive demands comparable to manual driving while using the partially automated technology. Drivers who are more experienced with partially automated technology may respond differently than those with limited prior experience.

Applications of brain imaging methods in driving behaviour research

Applications of neuroimaging methods have substantially contributed to the scientific understanding of human factors during driving by providing a deeper insight into the neuro-cognitive aspects of driver brain. This has been achieved by conducting simulated (and occasionally, field) driving experiments while collecting driver brain signals of various types. Here, this sector of studies is comprehensively reviewed at both macro and micro scales. At the macro scale, bibliometric aspects of these studies are analysed. At the micro scale, different themes of neuroimaging driving behaviour research are identified and the findings within each theme are synthesised. The surveyed literature has reported on applications of four major brain imaging methods. These include Functional Magnetic Resonance Imaging (fMRI), Electroencephalography (EEG), Functional Near-Infrared Spectroscopy (fNIRS) and Magnetoencephalography (MEG), with the first two being the most common methods in this domain. While collecting driver fMRI signal has been particularly instrumental in studying neural correlates of intoxicated driving (e.g. alcohol or cannabis) or distracted driving, the EEG method has been predominantly utilised in relation to the efforts aiming at development of automatic fatigue/drowsiness detection systems, a topic to which the literature on neuro-ergonomics of driving particularly has shown a spike of interest within the last few years. The survey also reveals that topics such as driver brain activity in semi-automated settings or neural activity of drivers with brain injuries or chronic neurological conditions have by contrast been investigated to a very limited extent. Potential topics in driving behaviour research are identified that could benefit from the adoption of neuroimaging methods in future studies. In terms of practicality, while fMRI and MEG experiments have proven rather invasive and technologically challenging for adoption in driving behaviour research, EEG and fNIRS applications have been more diverse. They have even been tested beyond simulated driving settings, in field driving experiments. Advantages and limitations of each of these four neuroimaging methods in the context of driving behaviour experiments are outlined in the paper.

Exploring the effectiveness of a digital voice assistant to maintain driver alertness in partially automated vehicles

OBJECTIVE

Vehicle automation shifts the driver's role from active operator to passive observer at the potential cost of degrading their alertness. This study investigated the role of an in-vehicle voice-based assistant (VA; conversing about traffic/road environment) to counter the disengaging and fatiguing effects of automation.

METHOD

Twenty-four participants undertook two drives- with and without VA in a partially automated vehicle. Participants were subsequently categorized into high and low participation groups (based on their proportion of vocal exchanges with VA). The effectiveness of VA was assessed based on driver alertness measured using Karolinska Sleepiness Scale (KSS), eye-based sleepiness indicators and glance behavior, NASA-TLX workload rating and time to gain motor readiness in response to take-over request and performance rating made by the drivers.

RESULTS

Paired samples t-tests comparison of alertness measures across the two drives were conducted. Lower KSS rating, larger pupil diameter, higher glances (rear-mirror, roadside vehicles and signals in the drive with VA) and higher feedback ratings of VA indicated the efficiency of VA in improving driver alertness during automation. However, there was no significant difference in alertness or glance behavior between the driver groups (high and low-PR), although the time to resume steering control was significantly lower in the higher engagement group.

CONCLUSION

The study successfully demonstrated the advantages of using a voice assistant (VA) to counter these effects of passive fatigue, for example, by reducing the time to gain motor-readiness following a TOR. The findings show that despite the low engagement in spoken conversation, active listening also positively influenced driver alertness and awareness during the drive in an automated vehicle.

Effects of partially automated driving on the development of driver sleepiness

The objective of this study was to compare the development of sleepiness during manual driving versus level 2 partially automated driving, when driving on a motorway in Sweden. The hypothesis was that partially automated driving will lead to higher levels of fatigue due to underload. Eighty-nine drivers were included in the study using a 2 × 2 design with the conditions manual versus partially automated driving and daytime (full sleep) versus night-time (sleep deprived). The results showed that night-time driving led to markedly increased levels of sleepiness in terms of subjective sleepiness ratings, blink durations, PERCLOS, pupil diameter and heart rate. Partially automated driving led to slightly higher subjective sleepiness ratings, longer blink durations, decreased pupil diameter, slower heart rate, and higher EEG alpha and theta activity. However, elevated levels of sleepiness mainly arose from the night-time drives when the sleep pressure was high. During daytime, when the drivers were alert, partially automated driving had little or no detrimental effects on driver fatigue. Whether the negative effects of increased sleepiness during partially automated driving can be compensated by the positive effects of lateral and longitudinal driving support needs to be investigated in further studies.

Integration of automated vehicles in mixed traffic: Evaluating changes in performance of following human-driven vehicles

The introduction of Automated Vehicles (AVs) into the transportation network is expected to improve system performance, but the impacts of AVs in mixed traffic streams have not been clearly studied. As AV's market penetration increases, the interactions between conventional vehicles and AVs are inevitable but by no means clear. This study aims to create new knowledge by quantifying the behavioral changes caused when conventional human-driven vehicles follow AVs and investigating the impact of these changes (if any) on safety and the environment. This study analyzes data obtained from a field experiment by Texas A&M University to evaluate the effects of AVs on the behavior of a following human-driver. The dataset is comprised of nine drivers that attempted to follow 5 speed-profiles, with two scenarios per profile. In scenario one, a human-driven vehicle follows an AV that implements a human driver speed profile (base). In scenario two, the human-driven vehicle follows an AV that executes an AV speed profile. In order to evaluate safety, these scenarios are compared using time-to-collision (TTC) and several other driving volatility measures. Likewise, fuel consumption and emissions are used to investigate environmental impacts. Overall, the results show that AVs in mixed traffic streams can induce behavioral changes in conventional vehicle drivers, with some beneficial effects on safety and the environment. On average, a driver that follows an AV exhibits lower driving volatility in terms of speed and acceleration, which represents more stable traffic flow behavior and lower crash risk. The analysis showed a remarkable improvement in TTC as a result of the notably better speed adjustments of the following vehicle (i.e., lower differences in speeds between the lead and following vehicles) in the second scenario. Furthermore, human-driven vehicles were found to consume less fuel and produce fewer emissions on average when following an AV.

Effects of one-pedal automobile operation on the driver's emotional state and cognitive workload

A one-pedal system for operating an electric vehicle allows drivers to flexibly accelerate and decelerate (and even stop) by using just an accelerator pedal. Based on previous findings, one-pedal operation is considered to have the potential to increase positive emotions and decrease cognitive workload. To test this possibility, the present study compared the emotional state and cognitive workload between one-pedal and conventional two-pedal operation. Participants drove a vehicle on public roads, and driving enjoyment (i.e., pleasure and immersion) and the cognitive workload (i.e., ease and effortlessness) were assessed by means of questionnaires. In addition, physiological variations associated with driving pleasure and difficulty were assessed by electroencephalography (EEG). Both the questionnaire and EEG results revealed an increase in driving enjoyment in one-pedal operation. On the other hand, only the EEG results suggested a decrease in the cognitive workload in one-pedal operation; the questionnaire results did not show a significant difference between the pedal conditions. These findings support the notion that one-pedal operation has a positive influence on the driver's mental state, though its influence on the cognitive workload will require further investigation. We discuss future directions toward a better understanding of the effects of one-pedal operation on the driver's mental state.

Age-related differences in effects of non-driving related tasks on takeover performance in automated driving

INTRODUCTION

During SAE level 3 automated driving, the driver's role changes from active driver to fallback-ready driver. Drowsiness is one of the factors that may degrade driver's takeover performance. This study aimed to investigate effects of non-driving related tasks (NDRTs) to counter driver's drowsiness with a Level 3 system activated and to improve successive takeover performance in a critical situation. A special focus was placed on age-related differences in the effects.

METHOD

Participants of three age groups (younger, middle-aged, older) drove the Level 3 system implemented in a high-fidelity motion-based driving simulator for about 30 min under three experiment conditions: without NDRT, while watching a video clip, and while switching between watching a video clip and playing a game. The Karolinska Sleepiness Scale and eyeblink duration measured driver drowsiness. At the end of the drive, the drivers had to take over control of the vehicle and manually change the lane to avoid a collision. Reaction time and steering angle variability were measured to evaluate the two aspects of driving performance.

RESULTS

For younger drivers, both single and multiple NDRT engagements countered the development of driver drowsiness during automated driving, and their takeover performance was equivalent to or better than their performance without NDRT engagement. For older drivers, NDRT engagement did not affect the development of drowsiness but degraded takeover performance especially under the multiple NDRT engagement condition. The results for middle-aged drivers fell at an intermediate level between those for younger and older drivers. Practical Applications: The present findings do not support general recommendations of NDRT engagement to counter drowsiness during automated driving. This study is especially relevant to the automotive industry's search for options that will ensure the safest interfaces between human drivers and automation systems.