Augmented reality for supporting the interaction between pedestrians and automated vehicles: an experimental outdoor study

Introduction: Communication from automated vehicles (AVs) to pedestrians using augmented reality (AR) could positively contribute to traffic safety. However, previous AR research for pedestrians was mainly conducted through online questionnaires or experiments in virtual environments instead of real ones. Methods: In this study, 28 participants conducted trials outdoors with an approaching AV and were supported by four different AR interfaces. The AR experience was created by having participants wear a Varjo XR-3 headset with see-through functionality, with the AV and AR elements virtually overlaid onto the real environment. The AR interfaces were vehicle-locked (Planes on vehicle), world-locked (Fixed pedestrian lights, Virtual fence), or head-locked (Pedestrian lights HUD). Participants had to hold down a button when they felt it was safe to cross, and their opinions were obtained through rating scales, interviews, and a questionnaire. Results: The results showed that participants had a subjective preference for AR interfaces over no AR interface. Furthermore, the Pedestrian lights HUD was more effective than no AR interface in a statistically significant manner, as it led to participants more frequently keeping the button pressed. The Fixed pedestrian lights scored lower than the other interfaces, presumably due to low saliency and the fact that participants had to visually identify both this AR interface and the AV. Discussion: In conclusion, while users favour AR in AV-pedestrian interactions over no AR, its effectiveness depends on design factors like location, visibility, and visual attention demands. In conclusion, this work provides important insights into the use of AR outdoors. The findings illustrate that, in these circumstances, a clear and easily interpretable AR interface is of key importance.

The effect of an occlusion-induced delay on braking behavior in critical situations: A driving simulator study

OBJECTIVE

To share results of an experiment that used visual occlusion for a new purpose: inducing a waiting time.

BACKGROUND

Senders was a leading figure in human factors. In his research on the visual demands of driving, he used occlusion techniques.

METHODS

In a simulator experiment, we examined how drivers brake for different levels of urgency and different visual conditions. In three blocks (1 = brake lights, 2 = no brake lights, 3 = occlusion), drivers followed a vehicle at 13.4 or 33.4 m distance. At certain moments, the lead vehicle decelerated moderately (1.7 m/s2) or strongly (6.5 m/s2). In the occlusion condition, the screens blanked for 0.4 s (if 6.5 m/s2) or 2.0 s (if 1.7 m/s2) when the lead vehicle started to decelerate. Participants were instructed to brake only after the occlusion ended.

RESULTS

The lack of brake lights caused a delayed response. In the occlusion condition, drivers adapted to the instructed late braking by braking harder. However, adaptation was not always possible: In the most urgent condition, most participants collided with the lead vehicle because the ego-vehicle's deceleration limits were reached. In non-urgent conditions, some drivers braked unnecessarily hard. Furthermore, while waiting until the occlusion cleared, some drivers lightly touched the brake pedal.

CONCLUSION

This experimental design demonstrates how drivers (sometimes fail to) adjust their braking behavior to the criticality of the situation.

APPLICATION

The phenomena of biomechanical readiness and (inappropriate) dosing of the brake pedal may be relevant to safety, traffic flow, and ADAS design.

Responses to Raven matrices: Governed by visual complexity and centrality

Raven matrices are widely considered a pure test of cognitive abilities. Previous research has examined the extent to which cognitive strategies are predictive of the number of correct responses to Raven items. This study examined whether response times can be explained directly from the centrality and visual complexity of the matrix cells (edge density and perceived complexity). A total of 159 participants completed a 12-item version of the Raven Advanced Progressive Matrices. In addition to item number (an index of item difficulty), the findings demonstrated a positive correlation between the visual complexity of Raven items and both the mean response time and the number of fixations on the matrix (a strong correlate of response time). Moreover, more centrally placed cells as well as more complex cells received more fixations. It is concluded that response times on Raven matrices are impacted by low-level stimulus attributes, namely, visual complexity and eccentricity.

How do people distribute their attention while observing The Night Watch?

This study explored how people look at The Night Watch (1642), Rembrandt's masterpiece. Twenty-one participants each stood in front of the painting for 5 min, while their eyes were recorded with a mobile eye-tracker and their thoughts were verbalized with a think-aloud method. We computed a heatmap of the participants’ attentional distribution using a novel markerless mapping method. The results showed that the participants’ attention was mainly directed at the faces of the two central figures, the bright mascot girl in the painting, and detailed elements such as the apparel of the key figures. The eye-movement analysis and think-aloud data also showed that participants’ attention shifted from the faces of the key figures to other elements of the scene over the course of the 5 min. Our analyses are consistent with the theory that Rembrandt used light and texture to capture the viewer's attention. Finally, the robustness of the eye-tracking method was demonstrated by replicating the study on a smaller replica.

Redesigning Today's Driving Automation Toward Adaptive Backup Control With Context-Based and Invisible Interfaces

OBJECTIVE

We investigated a driver monitoring system (DMS) designed to adaptively back up distracted drivers with automated driving.

BACKGROUND

Humans are likely inadequate for supervising today's on-road driving automation. Conversely, backup concepts can use eye-tracker DMS to retain the human as the primary driver and use computerized control only if needed. A distraction DMS where perceived false alarms are minimized and the status of the backup is unannounced might reduce problems of distrust and overreliance, respectively. Experimental research is needed to assess the viability of such designs.

METHODS

In a driving simulator, 91 participants either supervised driving automation (auto-hand-on-wheel vs. auto-hands-off-wheel), drove with different forms of DMS-induced backup control (eyes-only-backup vs. eyes-plus-context-backup; visible-backup vs. invisible-backup), or drove without any automation. All participants performed a visual N-back task throughout.

RESULTS

Supervised driving automation increased visual distraction and hazard non-responses compared to backup and conventional driving. Auto-hand-on-wheel improved response generation compared to auto-hands-off-wheel. Across entire driving trials, the backup improved lateral performance compared to conventional driving. Without negatively impacting safety, the eyes-plus-context-backup DMS reduced unnecessary automated control compared to the eyes-only-backup DMS conditions. Eyes-only-backup produced low satisfaction ratings, whereas eyes-plus-context-backup satisfaction was on par with automated driving. There were no appreciable negative consequences attributable to the invisible-backup driving automation.

CONCLUSIONS

We have demonstrated preliminary feasibility of DMS designs that incorporate driving context information for distraction assessment and suppress their status indication.

APPLICATION

An appropriately designed DMS can enable benefits for automated driving as a backup.

Emergency braking at intersections: A motion-base motorcycle simulator study

Powered two-wheeler riders are frequently involved in crashes at intersections because an approaching car driver fails to give right of way. This simulator study aimed to investigate how riders perform an emergency braking maneuver in response to an oncoming car and, second, whether longitudinal motion cues provided by a motion platform influence riders' braking performance. Twelve riders approached a four-way intersection at the same time as an oncoming car. We manipulated the car's direction of travel, speed profile, and its indicator light. The results showed that the more dangerous the situation (safe, near-miss, impending-crash), the more likely riders were to initiate braking. Although riders braked in the majority of trials when the car crossed their path, they were often unsuccessful in avoiding a collision with the car. No statistically significant differences were found in riders' initiation of braking and braking style between the motion and no-motion simulator configurations.

What determines drivers' speed? A replication of three behavioural adaptation experiments in a single driving simulator study

We conceptually replicated three highly cited experiments on speed adaptation, by measuring drivers' experienced risk (galvanic skin response; GSR), experienced task difficulty (self-reported task effort; SRTE) and safety margins (time-to-line-crossing; TLC) in a single experiment. The three measures were compared using a nonparametric index that captures the criteria of constancy during self-paced driving and sensitivity during forced-paced driving. In a driving simulator, 24 participants completed two forced-paced and one self-paced run. Each run held four different lane width conditions. Results showed that participants drove faster on wider lanes, thus confirming the expected speed adaptation. None of the three measures offered persuasive evidence for speed adaptation because they failed either the sensitivity criterion (GSR) or the constancy criterion (TLC, SRTE). An additional measure, steering reversal rate, outperformed the other three measures regarding sensitivity and constancy, prompting a further evaluation of the role of control activity in speed adaptation. Practitioner Summary: Results from a driving simulator experiment suggest that it is not experienced risk, experienced effort or safety margins that govern drivers' choice of speed. Rather, our findings suggest that steering reversal rate has an explanatory role in speed adaptation.

Differences between racing and non-racing drivers: A simulator study using eye-tracking

Motorsport has developed into a professional international competition. However, limited research is available on the perceptual and cognitive skills of racing drivers. By means of a racing simulator, we compared the driving performance of seven racing drivers with ten non-racing drivers. Participants were tasked to drive the fastest possible lap time. Additionally, both groups completed a choice reaction time task and a tracking task. Results from the simulator showed faster lap times, higher steering activity, and a more optimal racing line for the racing drivers than for the non-racing drivers. The non-racing drivers’ gaze behavior corresponded to the tangent point model, whereas racing drivers showed a more variable gaze behavior combined with larger head rotations while cornering. Results from the choice reaction time task and tracking task showed no statistically significant difference between the two groups. Our results are consistent with the current consensus in sports sciences in that task-specific differences exist between experts and novices while there are no major differences in general cognitive and motor abilities.

Effects of platooning on signal-detection performance, workload, and stress: A driving simulator study

Platooning, whereby automated vehicles travel closely together in a group, is attractive in terms of safety and efficiency. However, concerns exist about the psychological state of the platooning driver, who is exempted from direct control, yet remains responsible for monitoring the outside environment to detect potential threats. By means of a driving simulator experiment, we investigated the effects on recorded and self-reported measures of workload and stress for three task-instruction conditions: (1) No Task, in which participants had to monitor the road, (2) Voluntary Task, in which participants could do whatever they wanted, and (3) Detection Task, in which participants had to detect red cars. Twenty-two participants performed three 40-min runs in a constant-speed platoon, one condition per run in counterbalanced order. Contrary to some classic literature suggesting that humans are poor monitors, in the Detection Task condition participants attained a high mean detection rate (94.7%) and a low mean false alarm rate (0.8%). Results of the Dundee Stress State Questionnaire indicated that automated platooning was less distressing in the Voluntary Task than in the Detection Task and No Task conditions. In terms of heart rate variability, the Voluntary Task condition yielded a lower power in the low-frequency range relative to the high-frequency range (LF/HF ratio) than the Detection Task condition. Moreover, a strong time-on-task effect was found, whereby the mean heart rate dropped from the first to the third run. In conclusion, participants are able to remain attentive for a prolonged platooning drive, and the type of monitoring task has effects on the driver's psychological state.

The Effect of Haptic Support Systems on Driver Performance: A Literature Survey

A large number of haptic driver support systems have been described in the scientific literature. However, there is little consensus regarding the design, evaluation methods, and effectiveness of these systems. This literature survey aimed to investigate: (1) what haptic systems (in terms of function, haptic signal, channel, and supported task) have been experimentally tested, (2) how these haptic systems have been evaluated, and (3) their reported effects on driver performance and behaviour. We reviewed empirical research in which participants had to drive a vehicle in a real or simulated environment, were able to control the heading and/or speed of the vehicle, and a haptic signal was provided to them. The results indicated that a clear distinction can be made between warning systems (using vibrations) and guidance systems (using continuous forces). Studies typically used reaction time measures for evaluating warning systems and vehicle-centred performance measures for evaluating guidance systems. In general, haptic warning systems reduced the reaction time of a driver compared to no warnings, although these systems may cause annoyance. Guidance systems generally improved the performance of drivers compared to non-aided driving, but these systems may suffer from after-effects. Longitudinal research is needed to investigate the transfer and retention of effects caused by haptic support systems.

A quarter of a century of the DBQ: some supplementary notes on its validity with regard to accidents

This article synthesises the latest information on the relationship between the Driver Behaviour Questionnaire (DBQ) and accidents. We show by means of computer simulation that correlations with accidents are necessarily small because accidents are rare events. An updated meta-analysis on the zero-order correlations between the DBQ and self-reported accidents yielded an overall r of .13 (fixed-effect and random-effects models) for violations (57,480 participants; 67 samples) and .09 (fixed-effect and random-effects models) for errors (66,028 participants; 56 samples). An analysis of a previously published DBQ dataset (975 participants) showed that by aggregating across four measurement occasions, the correlation coefficient with self-reported accidents increased from .14 to .24 for violations and from .11 to .19 for errors. Our meta-analysis also showed that DBQ violations (r = .24; 6353 participants; 20 samples) but not DBQ errors (r = - .08; 1086 participants; 16 samples) correlated with recorded vehicle speed. Practitioner Summary: The DBQ is probably the most widely used self-report questionnaire in driver behaviour research. This study shows that DBQ violations and errors correlate moderately with self-reported traffic accidents.

Effects of visual fidelity on curve negotiation, gaze behaviour and simulator discomfort

Technological developments have led to increased visual fidelity of driving simulators. However, simplified visuals have potential advantages, such as improved experimental control, reduced simulator discomfort and increased generalisability of results. In this driving simulator study, we evaluated the effects of visual fidelity on driving performance, gaze behaviour and subjective discomfort ratings. Twenty-four participants drove a track with 90° corners in (1) a high fidelity, textured environment, (2) a medium fidelity, non-textured environment without scenery objects and (3) a low-fidelity monochrome environment that only showed lane markers. The high fidelity level resulted in higher steering activity on straight road segments, higher driving speeds and higher gaze variance than the lower fidelity levels. No differences were found between the two lower fidelity levels. In conclusion, textures and objects were found to affect steering activity and driving performance; however, gaze behaviour during curve negotiation and self-reported simulator discomfort were unaffected.

PRACTITIONER SUMMARY

In a driving simulator study, three levels of visual fidelity were evaluated. The results indicate that the highest fidelity level, characterised by a textured environment, resulted in higher steering activity, higher driving speeds and higher variance of horizontal gaze than the two lower fidelity levels without textures.

Should drivers be operating within an automation-free bandwidth? Evaluating haptic steering support systems with different levels of authority

OBJECTIVE

The aim of this study was to compare continuous versus bandwidth haptic steering guidance in terms of lane-keeping behavior, aftereffects, and satisfaction.

BACKGROUND

An important human factors question is whether operators should be supported continuously or only when tolerance limits are exceeded. We aimed to clarify this issue for haptic steering guidance by investigating costs and benefits of both approaches in a driving simulator.

METHODS

Thirty-two participants drove five trials, each with a different level of haptic support: no guidance (Manual); guidance outside a 0.5-m bandwidth (Band1); a hysteresis version of Band1, which guided back to the lane center once triggered (Band2); continuous guidance (Cont); and Cont with double feedback gain (ContS). Participants performed a reaction time task while driving. Toward the end of each trial, the guidance was unexpectedly disabled to investigate aftereffects.

RESULTS

All four guidance systems prevented large lateral errors (>0.7 m). Cont and especially ContS yielded smaller lateral errors and higher time to line crossing than Manual, Band1, and Band2. Cont and ContS yielded short-lasting aftereffects, whereas Band1 and Band2 did not. Cont yielded higher self-reported satisfaction and faster reaction times than Band1.

CONCLUSIONS

Continuous and bandwidth guidance both prevent large driver errors. Continuous guidance yields improved performance and satisfaction over bandwidth guidance at the cost of aftereffects and variability in driver torque (indicating human-automation conflicts).

APPLICATION

The presented results are useful for designers of haptic guidance systems and support critical thinking about the costs and benefits of automation support systems.

Obstacle avoidance, visual detection performance, and eye-scanning behavior of glaucoma patients in a driving simulator: a preliminary study

The objective of this study was to evaluate differences in driving performance, visual detection performance, and eye-scanning behavior between glaucoma patients and control participants without glaucoma. Glaucoma patients (n = 23) and control participants (n = 12) completed four 5-min driving sessions in a simulator. The participants were instructed to maintain the car in the right lane of a two-lane highway while their speed was automatically maintained at 100 km/h. Additional tasks per session were: Session 1: none, Session 2: verbalization of projected letters, Session 3: avoidance of static obstacles, and Session 4: combined letter verbalization and avoidance of static obstacles. Eye-scanning behavior was recorded with an eye-tracker. Results showed no statistically significant differences between patients and control participants for lane keeping, obstacle avoidance, and eye-scanning behavior. Steering activity, number of missed letters, and letter reaction time were significantly higher for glaucoma patients than for control participants. In conclusion, glaucoma patients were able to avoid objects and maintain a nominal lane keeping performance, but applied more steering input than control participants, and were more likely than control participants to miss peripherally projected stimuli. The eye-tracking results suggest that glaucoma patients did not use extra visual search to compensate for their visual field loss. Limitations of the study, such as small sample size, are discussed.

The effect of concurrent bandwidth feedback on learning the lane-keeping task in a driving simulator

OBJECTIVE

The aim of this study was to investigate whether concurrent bandwidth feedback improves learning of the lane-keeping task in a driving simulator.

BACKGROUND

Previous research suggests that bandwidth feedback improves learning and that off-target feedback is superior to on-target feedback. This study aimed to extend these findings for the lane-keeping task.

METHOD

Participants without a driver's license drove five 8-min lane-keeping sessions in a driver training simulator: three practice sessions, an immediate retention session, and a delayed retention session I day later. There were four experimental groups (n=15 per group): (a) on-target, receiving seat vibrations when the center of the car was within 0.5 m of the lane center; (b) off-target, receiving seat vibrations when the center of the car was more than 0.5 m away from the lane center; (c) control, receiving no vibrations; and (d) realistic, receiving seat vibrations depending on engine speed. During retention, all groups were provided with the realistic vibrations.

RESULTS

During practice, on-target and off-target groups had better lane-keeping performance than the nonaugmented groups, but this difference diminished in the retention phase. Furthermore, during late practice and retention, the off-target group outperformed the on-target group.The off-target group had a higher rate of steering reversal and higher steering entropy than the nonaugmented groups, whereas no clear group differences were found regarding mean speed, mental workload, or self-reported measures.

CONCLUSION

Off-target feedback is superior to on-target feedback for learning the lane-keeping task.

APPLICATION

This research provides knowledge to researchers and designers of training systems about the value of feedback in simulator-based training of vehicular control.

Objective classification of residents based on their psychomotor laparoscopic skills

Background

From the clinical point of view, it is important to recognize residents’ level of expertise with regard to basic psychomotor skills. For that reason, surgeons and surgical organizations (e.g., Acreditation Council for Graduate Medical Education, ACGME) are calling for assessment tools that credential residents as technically competent. Currently, no method is universally accepted or recommended for classifying residents as “experienced,” “intermediates,” or “novices” according to their technical abilities. This study introduces a classification method for recognizing residents’ level of experience in laparoscopic surgery based on psychomotor laparoscopic skills alone.

Methods

For this study, 10 experienced residents (>100 laparoscopic procedures performed), 10 intermediates (10–100 procedures performed), and 11 novices (no experience) performed four tasks in a box trainer. The movements of the laparoscopic instruments were recorded with the TrEndo tracking system and analyzed using six motion analysis parameters (MAPs). The MAPs of all participants were submitted to principal component analysis (PCA), a data reduction technique. The scores of the first principal components were used to perform linear discriminant analysis (LDA), a classification method. Performance of the LDA was examined using a leave-one-out cross-validation.

Results

Of 31 participants, 23 were classified correctly with the proposed method, with 7 categorized as experienced, 7 as intermediates, and 9 as novices.

Conclusions

The proposed method provides a means to classify residents objectively as experienced, intermediate, or novice surgeons according to their basic laparoscopic skills. Due to the simplicity and generalizability of the introduced classification method, it is easy to implement in existing trainers.