The interaction of cognitive load and attention-directing cues in driving

Study on the Effect of Judgment Excitation Mode to Relieve Driving Fatigue Based on MF-DFA

Driving fatigue refers to a phenomenon in which a driver’s physiological and psychological functions become unbalanced after a long period of continuous driving, and their driving skills decline objectively. The hidden dangers of driving fatigue to traffic safety should not be underestimated. In this work, we propose a judgment excitation mode (JEM), which adds secondary cognitive tasks to driving behavior through dual-channel human–computer interaction, so as to delay the occurrence of driving fatigue. We used multifractal detrended fluctuation analysis (MF-DFA) to study the dynamic properties of subjects’ EEG, and analyzed the effect of JEM on fatigue retardation by Hurst exponent value and multifractal spectrum width value. The results show that the multifractal properties of the two driving modes (normal driving mode and JEM) are significantly different. The JEM we propose can effectively delay the occurrence of driving fatigue, and has good prospects for future practical applications.

Effects of verbal tasks on driving simulator performance

We report results from a driving simulator paradigm we developed to test the fine temporal effects of verbal tasks on simultaneous tracking performance. A total of 74 undergraduate students participated in two experiments in which they controlled a cursor using the steering wheel to track a moving target and where the dependent measure was overall deviation from target. Experiment 1 tested tracking performance during slow and fast target speeds under conditions involving either no verbal input or output, passive listening to spoken prompts via headphones, or responding to spoken prompts. Experiment 2 was similar except that participants read written prompts overlain on the simulator screen instead of listening to spoken prompts. Performance in both experiments was worse during fast speeds and worst overall during responding conditions. Most significantly, fine scale time-course analysis revealed deteriorating tracking performance as participants prepared and began speaking and steadily improving performance while speaking. Additionally, post-block survey data revealed that conversation recall was best in responding conditions, and perceived difficulty increased with task complexity. Our study is the first to track temporal changes in interference at high resolution during the first hundreds of milliseconds of verbal production and comprehension. Our results are consistent with load-based theories of multitasking performance and show that language production, and, to a lesser extent, language comprehension tap resources also used for tracking. More generally, our paradigm provides a useful tool for measuring dynamical changes in tracking performance during verbal tasks due to the rapidly changing resource requirements of language production and comprehension.

Eye-head coordination and dynamic visual scanning as indicators of visuo-cognitive demands in driving simulator

Driving is an everyday task involving a complex interaction between visual and cognitive processes. As such, an increase in the cognitive and/or visual demands can lead to a mental overload which can be detrimental for driving safety. Compiling evidence suggest that eye and head movements are relevant indicators of visuo-cognitive demands and attention allocation. This study aims to investigate the effects of visual degradation on eye-head coordination as well as visual scanning behavior during a highly demanding task in a driving simulator. A total of 21 emmetropic participants (21 to 34 years old) performed dual-task driving in which they were asked to maintain a constant speed on a highway while completing a visual search and detection task on a navigation device. Participants did the experiment with optimal vision and with contact lenses that introduced a visual perturbation (myopic defocus). The results indicate modifications of eye-head coordination and the dynamics of visual scanning in response to the visual perturbation induced. More specifically, the head was more involved in horizontal gaze shifts when the visual needs were not met. Furthermore, the evaluation of visual scanning dynamics, based on time-based entropy which measures the complexity and randomness of scanpaths, revealed that eye and gaze movements became less explorative and more stereotyped when vision was not optimal. These results provide evidence for a reorganization of both eye and head movements in response to increasing visual-cognitive demands during a driving task. Altogether, these findings suggest that eye and head movements can provide relevant information about visuo-cognitive demands associated with complex tasks. Ultimately, eye-head coordination and visual scanning dynamics may be good candidates to estimate drivers' workload and better characterize risky driving behavior.

Age, cognitive load, and multimodal effects on driver response to directional warning

Inattention can be considered a primary cause of vehicular accidents or crashes, and in-car warning signals are applied to alert the driver to take action even in automated vehicles. Because of age related decline of the older driver's abilities, in-car warning signals may need adjustment to the older driver. We therefore investigated the effects of uni-, bi- and trimodal directional warnings (i.e., light, sound, vibration) on young and older drivers' responses in a driving simulator. A young group of 15 drivers (20-25 years of age) and an older group of 16 drivers (65-79 years of age) participated. In the simulations, warning signal was presented at the left, the center, or the right in front of the participant. With a warning at the left, the center, and the right the correct response was to steer to the right, brake, and steer to the left, respectively. The main results showed the older drivers' responses were slower for each type of warning compared with the young drivers' responses. Overall, the responses were slower with an added cognitively loading task. The only multimodal type of warning inducing overall faster response than its constituent warning types was the vibration-sound, and only for the older drivers. Additionally, with the groups' responses collapsed, such a true multimodal effect on response time also showed for the center vibration-sound warning (i.e., braking response). The only multimodal warning showing clear reduction in response errors compared with its constituent warning types was the vibration-sound for the older drivers during extra cognitive load. The main conclusion is that older drivers can benefit from bimodal warning, as compared with unimodal, in terms of faster and more accurate response. The potential superiority of trimodal warning is nevertheless argued.

Visual advisory warnings about hidden dangers: Effects of specific symbols and spatial referencing on necessary and unnecessary warnings

Augmented reality has the potential to improve the effectiveness of collision warnings in vehicles because they inherently convey spatial information about the hazard and can guide the attention of the driver towards it. For future warning systems, which can detect sight obstructed dangers, related work already revealed some advantages. In a driving simulator study with 80 participants, we investigated the effects of three corresponding design parameters which are commonly integrated at augmented reality warnings. This study analyzes the individual contribution of specific warning symbols, warning animation, and spatial referencing. Part one of the study concentrates on the effectiveness of necessary warnings and part two on the drivers' compliance despite false alarms. Compared to the control condition with static unspecific warning symbols, static specific warning symbols depicting the type and motion direction of the hazard led to several but inconsistent advantages. The scaling animation only improved subjective evaluation. However, spatial referencing of an (unspecific) warning symbol consistently improved drivers' reactions to as well as evaluations of necessary and unnecessary warnings. The results emphasize the potential of spatial referencing, particularly for in-vehicle warnings of future collision avoidance systems.

Measuring working memory load effects on electrophysiological markers of attention orienting during a simulated drive

Intersection accidents result in a significant proportion of road fatalities, and attention allocation likely plays a role. Attention allocation may depend on (limited) working memory (WM) capacity. Driving is often combined with tasks increasing WM load, consequently impairing attention orienting. This study (n = 22) investigated WM load effects on event-related potentials (ERPs) related to attention orienting. A simulated driving environment allowed continuous lane-keeping measurement. Participants were asked to orient attention covertly towards the side indicated by an arrow, and to respond only to moving cars appearing on the attended side by pressing a button. WM load was manipulated using a concurrent memory task. ERPs showed typical attentional modulation (cue: contralateral negativity, LDAP; car: N1, P1, SN and P3) under low and high load conditions. With increased WM load, lane-keeping performance improved, while dual task performance degraded (memory task: increased error rate; orienting task: increased false alarms, smaller P3). Practitioner Summary: Intersection driver-support systems aim to improve traffic safety and flow. However, in-vehicle systems induce WM load, increasing the tendency to yield. Traffic flow reduces if drivers stop at inappropriate times, reducing the effectiveness of systems. Consequently, driver-support systems could include WM load measurement during driving in the development phase.

Augmented reality warnings in vehicles: Effects of modality and specificity on effectiveness

In the future, vehicles will be able to warn drivers of hidden dangers before they are visible. Specific warning information about these hazards could improve drivers' reactions and the warning effectiveness, but could also impair them, for example, by additional cognitive-processing costs. In a driving simulator study with 88 participants, we investigated the effects of modality (auditory vs. visual) and specificity (low vs. high) on warning effectiveness. For the specific warnings, we used augmented reality as an advanced technology to display the additional auditory or visual warning information. Part one of the study concentrates on the effectiveness of necessary warnings and part two on the drivers' compliance despite false alarms. For the first warning scenario, we found several positive main effects of specificity. However, subsequent effects of specificity were moderated by the modality of the warnings. The specific visual warnings were observed to have advantages over the three other warning designs concerning gaze and braking reaction times, passing speeds and collision rates. Besides the true alarms, braking reaction times as well as subjective evaluation after these warnings were still improved despite false alarms. The specific auditory warnings were revealed to have only a few advantages, but also several disadvantages. The results further indicate that the exact coding of additional information, beyond its mere amount and modality, plays an important role. Moreover, the observed advantages of the specific visual warnings highlight the potential benefit of augmented reality coding to improve future collision warnings.

Effect of working memory load on electrophysiological markers of visuospatial orienting in a spatial cueing task simulating a traffic situation

Visuospatial attentional orienting has typically been studied in abstract tasks with low ecological validity. However, real-life tasks such as driving require allocation of working memory (WM) resources to several subtasks over and above orienting in a complex sensory environment. The aims of this study were twofold: firstly, to establish whether electrophysiological signatures of attentional orienting commonly observed under simplified task conditions generalize to a more naturalistic task situation with realistic-looking stimuli, and, secondly, to assess how these signatures are affected by increased WM load under such conditions. Sixteen healthy participants performed a dual task consisting of a spatial cueing paradigm and a concurrent verbal memory task that simulated aspects of an actual traffic situation. Behaviorally, we observed a load-induced detriment of sensitivity to targets. In the EEG, we replicated orienting-related alpha lateralization, the lateralized ERPs ADAN, EDAN, and LDAP, and the P1-N1 attention effect. When WM load was high (i.e., WM resources were reduced), lateralization of oscillatory activity in the lower alpha band was delayed. In the ERPs, we found that ADAN was also delayed, while EDAN was absent. Later ERP correlates were unaffected by load. Our results show that the findings in highly controlled artificial tasks can be generalized to spatial orienting in ecologically more valid tasks, and further suggest that the initiation of spatial orienting is delayed when WM demands of an unrelated secondary task are high.

Tactile warning signals for in-vehicle systems

The last few years have seen growing interest in the design of tactile warning signals to direct driver attention to potentially dangerous road situations (e.g. an impending crash) so that they can initiate an avoidance maneuver in a timely manner. In this review, we highlight the potential uses of such warning signals for future collision warning systems and compare them with more traditional visual and auditory warnings. Basic tactile warning signals are capable of promoting driver alertness, which has been demonstrated to be beneficial for forward collision avoidance (when compared to a no warning baseline condition). However, beyond their basic alerting function, directional tactile warning signals are now increasingly being utilized to shift the attention of the driver toward locations of interest, and thus to further facilitate their speeded responses to potential collision events. Currently, many researchers are focusing their efforts on the development of meaningful (iconic) tactile warning signals. For instance, dynamic tactile warnings (varying in their intensity and/or location) can potentially be used to convey meaningful information to drivers. Finally, we highlight the future research that will be needed in order to explore how to present multiple directional warnings using dynamic tactile cues, thus forming an integrated collision avoidance system for future in-vehicle use.

A parametric duration model of the reaction times of drivers distracted by mobile phone conversations

The use of mobile phones while driving is more prevalent among young drivers-a less experienced cohort with elevated crash risk. The objective of this study was to examine and better understand the reaction times of young drivers to a traffic event originating in their peripheral vision whilst engaged in a mobile phone conversation. The CARRS-Q advanced driving simulator was used to test a sample of young drivers on various simulated driving tasks, including an event that originated within the driver's peripheral vision, whereby a pedestrian enters a zebra crossing from a sidewalk. Thirty-two licensed drivers drove the simulator in three phone conditions: baseline (no phone conversation), hands-free and handheld. In addition to driving the simulator each participant completed questionnaires related to driver demographics, driving history, usage of mobile phones while driving, and general mobile phone usage history. The participants were 21-26 years old and split evenly by gender. Drivers' reaction times to a pedestrian in the zebra crossing were modelled using a parametric accelerated failure time (AFT) duration model with a Weibull distribution. Also tested where two different model specifications to account for the structured heterogeneity arising from the repeated measures experimental design. The Weibull AFT model with gamma heterogeneity was found to be the best fitting model and identified four significant variables influencing the reaction times, including phone condition, driver's age, license type (provisional license holder or not), and self-reported frequency of usage of handheld phones while driving. The reaction times of drivers were more than 40% longer in the distracted condition compared to baseline (not distracted). Moreover, the impairment of reaction times due to mobile phone conversations was almost double for provisional compared to open license holders. A reduction in the ability to detect traffic events in the periphery whilst distracted presents a significant and measurable safety concern that will undoubtedly persist unless mitigated.

How to present collision warnings at intersections?--a comparison of different approaches

Analyses of intersection accidents have shown that in most cases drivers overlook or see other road users with the right of way too late that they cannot react in time. Appropriate visual warning signals which support drivers in their attention allocation and driving behavior could be useful to improve drivers' reaction. In a driving simulator study, two warning strategies varying in their timing: (1) top-down warning while approaching a critical intersection and (2) bottom-up warning directly before the critical incident were investigated in a T-intersection situation. For the bottom-up warning, two warning signal designs were compared. 48 subjects (M=27.3 years, SD=7.4 years) participated in the study. Driving data as well as subjective evaluation of the three warning signals (one early and two late warning signals) were analyzed. The early warning signal which was given while approaching the intersection showed a positive effect. Here, most collisions could be avoided due to drivers' adaptation of their driving behavior toward safer driving. They waited longer at the intersection before turning and finally turned with a lower velocity. In addition, drivers evaluated the early warning signal as very "useful". With regard to the late warnings a much smaller effect was found. From these results, requirements can be derived for the design of effective warning strategies when driving at critical intersections.

Attentional effects on gaze preference for salient loci in traffic scenes

Alerting drivers for self-regulation of attention might decrease crash risks attributable to absent-minded driving. However, no reliable method exists for monitoring driver attention. Therefore, we examined attentional effects on gaze preference for salient loci (GPS) in traffic scenes. In an active viewing (AV) condition requiring endogenous attention for traffic scene comprehension, participants identified appropriate speeds for driving in presented traffic scene images. In a passive viewing (PV) condition requiring no endogenous attention, participants passively viewed traffic scene images. GPS was quantified by the mean saliency value averaged across fixation locations. Results show that GPS was less during AV than during PV. Additionally, gaze dwell time on signboards was shorter for AV than for PV. These results suggest that, in the absence of endogenous attention for traffic scene comprehension, gaze tends to concentrate on irrelevant salient loci in a traffic environment. Therefore, increased GPS can indicate absent-minded driving.

PRACTITIONER SUMMARY

The present study demonstrated that, without endogenous attention for traffic scene comprehension, gaze tends to concentrate on irrelevant salient loci in a traffic environment. This result suggests that increased gaze preference for salient loci indicates absent-minded driving, which is otherwise difficult to detect.

Auditory Displays for In-Vehicle Technologies

Modern vehicle cockpits have begun to incorporate a number of information-rich techno-logies, including systems to enhance and improve driving and navigation performance and also driving-irrelevant information systems. The visually intensive nature of the driving task requires these systems to adopt primarily nonvisual means of information display, and the auditory modality represents an obvious alternative to vision for interacting with in-vehicle technologies (IVTs). Although the literature on auditory displays has grown tremendously in recent decades, to date, few guidelines or recommendations exist to aid in the design of effective auditory displays for IVTs. This chapter provides an overview of the current state of research and practice with auditory displays for IVTs. The role of basic auditory capabilities and limitations as they relate to in-vehicle auditory display design are discussed. Extant systems and prototypes are reviewed, and when possible, design recommendations are made. Finally, research needs and an iterative design process to meet those needs are discussed.

Pop-out without awareness: unseen feature singletons capture attention only when top-down attention is available

Visual pop-out occurs when a unique visual target (e.g., a feature singleton) is present in a set of homogeneous distractors. However, the role of visual awareness in this process remains unclear. In the experiments reported here, we showed that even though subjects were not aware of a suppressed pop-out display, their subsequent performance on an orientation-discrimination task was significantly better at the pop-out location than at a control location. These results indicate that conscious visual awareness of a feature singleton is not necessary for it to attract attention. Furthermore, the subliminal pop-out effect disappeared when subjects diverted their attention toward a rapid sequential visual presentation task while presented with the same subliminal pop-out display. These results suggest that top-down attention is necessary for the subliminal pop-out effect and that the cognitive processes underlying attention and awareness are somewhat independent.

A selective review of simulated driving studies: Combining naturalistic and hybrid paradigms, analysis approaches, and future directions

Naturalistic paradigms such as movie watching or simulated driving that mimic closely real-world complex activities are becoming more widely used in functional magnetic resonance imaging (fMRI) studies both because of their ability to robustly stimulate brain connectivity and the availability of analysis methods which are able to capitalize on connectivity within and among intrinsic brain networks identified both during a task and in resting fMRI data. In this paper we review over a decade of work from our group and others on the use of simulated driving paradigms to study both the healthy brain as well as the effects of acute alcohol administration on functional connectivity during such paradigms. We briefly review our initial work focused on the configuration of the driving simulator and the analysis strategies. We then describe in more detail several recent studies from our group including a hybrid study examining distracted driving and compare resulting data with those from a separate visual oddball task (Fig. 6). The analysis of these data was performed primarily using a combination of group independent component analysis (ICA) and the general linear model (GLM) and in the various studies we highlight novel findings which result from an analysis of either 1) within-network connectivity, 2) inter-network connectivity, also called functional network connectivity, or 3) the degree to which the modulation of the various intrinsic networks were associated with the alcohol administration and the task context. Despite the fact that the behavioral effects of alcohol intoxication are relatively well known, there is still much to discover on how acute alcohol exposure modulates brain function in a selective manner, associated with behavioral alterations. Through the above studies, we have learned more regarding the impact of acute alcohol intoxication on organization of the brain's intrinsic connectivity networks during performance of a complex, real-world cognitive operation. Lessons learned from the above studies have broader applicability to designing ecologically valid, complex, functional MRI cognitive paradigms and incorporating pharmacologic challenges into such studies. Overall, the use of hybrid driving studies is a particularly promising area of neuroscience investigation.

Mind wandering behind the wheel: performance and oculomotor correlates

OBJECTIVE

An experiment studied the frequency and correlates of driver mind wandering.

BACKGROUND

Driver mind wandering is associated with risk for crash involvement. The present experiment examined the performance and attentional changes by which this effect might occur.

METHOD

Participants performed a car-following task in a high-fidelity driving simulator and were asked to report any time they caught themselves mind wandering. Vehicle control and eye movement data were recorded.

RESULTS

As compared with their attentive performance, participants showed few deficits in vehicle control while mind wandering but tended to focus visual attention narrowly on the road ahead.

CONCLUSION

Data suggest that mind wandering can engender a failure to monitor the environment while driving.

APPLICATION

Results identify behavioral correlates and potential risks of mind wandering that might enable efforts to detect and mitigate driver inattention.