External Human-Machine Interfaces on Automated Vehicles: Effects on Pedestrian Crossing Decisions

Deciding when to cross in front of an autonomous vehicle: How child and adult pedestrians respond to eHMI timing and vehicle kinematics

How autonomous vehicles (AVs) communicate their intentions to vulnerable road users (e.g., pedestrians) is a concern given the rapid growth and adoption of this technology. At present, little is known about how children respond to external Human Machine Interface (eHMI) signals from AVs. The current study examined how adults and children respond to the combination of explicit (eHMI signals) and implicit information (vehicle deceleration) to guide their road-crossing decisions. Children (8- to 12-year-olds) and adults made decisions about when to cross in front of a driverless car in an immersive virtual environment. The car sometimes stopped, either abruptly or gradually (manipulated within subjects), to allow participants to cross. When yielding, the car communicated its intent via a dome light that changed from red to green and varied in its timing onset (manipulated between subjects): early eHMI onset, late eHMI onset, or control (no eHMI). As expected, we found that both children and adults waited longer to enter the roadway when vehicles decelerated abruptly than gradually. However, adults responded to the early eHMI signal by crossing sooner when the cars decelerated either gradually or abruptly compared to the control condition. Children were heavily influenced by the late eHMI signal, crossing later when the eHMI signal appeared late and the vehicle decelerated either gradually or abruptly compared to the control condition. Unlike adults, children in the control condition behaved similarly to children in the early eHMI condition by crossing before the yielding vehicle came to a stop. Together, these findings suggest that early eHMI onset may lead to riskier behavior (initiating crossing well before a gradually decelerating vehicle comes to a stop), whereas late eHMI onset may lead to safer behavior (waiting for the eHMI signal to appear before initiating crossing). Without an eHMI signal, children show a concerning overreliance on gradual vehicle deceleration to judge yielding intent.

Effects of a color gradient and emoji in AR-HUD warning interfaces in autonomous vehicles on takeover performance and driver emotions

OBJECTIVE

This study examined the effects of color gradients and emojis in an augmented reality-head-up display (AR-HUD) warning interface on driver emotions and takeover performance.

METHODS

A total of 48 participants were grouped into four different warning interfaces for a simulated self-driving takeover experiment. Two-way analysis of variance and the Kruskal-Wallis test was used to analyze takeover time, mood, task load, and system availability.

RESULTS

Takeover efficiency and task load did not significantly differ among the interfaces, but the interfaces with a color gradient and emoji positively affected drivers' emotions. Emojis also positively affected emotional valence, and the color gradient had a high emotional arousal effect. Both the color gradient and the emoji interfaces had an inhibitory effect on negative emotions. The emoji interface was easier to learn, reducing driver learning costs.

CONCLUSIONS

These findings offer valuable insights for designing safer and more user-friendly AR-HUD interfaces for self-driving cars.

Augmented reality for supporting the interaction between pedestrians and automated vehicles: an experimental outdoor study

Introduction: Communication from automated vehicles (AVs) to pedestrians using augmented reality (AR) could positively contribute to traffic safety. However, previous AR research for pedestrians was mainly conducted through online questionnaires or experiments in virtual environments instead of real ones. Methods: In this study, 28 participants conducted trials outdoors with an approaching AV and were supported by four different AR interfaces. The AR experience was created by having participants wear a Varjo XR-3 headset with see-through functionality, with the AV and AR elements virtually overlaid onto the real environment. The AR interfaces were vehicle-locked (Planes on vehicle), world-locked (Fixed pedestrian lights, Virtual fence), or head-locked (Pedestrian lights HUD). Participants had to hold down a button when they felt it was safe to cross, and their opinions were obtained through rating scales, interviews, and a questionnaire. Results: The results showed that participants had a subjective preference for AR interfaces over no AR interface. Furthermore, the Pedestrian lights HUD was more effective than no AR interface in a statistically significant manner, as it led to participants more frequently keeping the button pressed. The Fixed pedestrian lights scored lower than the other interfaces, presumably due to low saliency and the fact that participants had to visually identify both this AR interface and the AV. Discussion: In conclusion, while users favour AR in AV-pedestrian interactions over no AR, its effectiveness depends on design factors like location, visibility, and visual attention demands. In conclusion, this work provides important insights into the use of AR outdoors. The findings illustrate that, in these circumstances, a clear and easily interpretable AR interface is of key importance.

The 'invisible gorilla' during pedestrian-AV interaction: Effects of secondary tasks on pedestrians' reaction to eHMIs

In road traffic, mental overload often leads to a failure to notice new and distinctive stimuli. Such phenomenon is known as 'inattentional blindness'. Safe and efficient interaction between automated vehicles (AVs) and pedestrians is expected to rely heavily on external human-machine interfaces (eHMIs), a tool AVs are equipped with to communicate their intentions to pedestrians. This study seeks to explore the phenomenon of 'inattentional blindness' in the context of pedestrian-AV interactions. Specifically, the aim is to understand the effects of a warning eHMI on pedestrians' crossing decisions when they are engaged in a secondary task. In an experiment study with videos of pedestrian crossing scenarios filmed from the perspective of the crossing pedestrian, participants had to decide the latest point at which they would be willing to cross the road in front of an AV with an eHMI vs. an AV without an eHMI. Participants were also asked to predict the future behavior of the AV. 125 female and 9 male participants aged between 18 and 25 completed the experiment and a follow-up questionnaire. It was found that the presence of a warning eHMI on AVs contributes to a clearer understanding of pedestrians' inferences about the intention of AVs and helps deter late and dangerous crossing decisions made by pedestrians. However, the eHMI fail to help pedestrians avoid such decisions when they face a high mental workload induced by secondary task engagement.

Effect of vehicle external acceleration signal light on pedestrian-vehicle interaction

The number of casualties resulting from collisions between pedestrians and motor vehicles continues to rise. A significant factor is the misunderstanding of vehicle behavior intentions by pedestrians. This is especially true with the continuous development of vehicle automation technology, which has reduced direct interaction between drivers and the outside world. Therefore, accurate communication of vehicle behavior intentions is becoming increasingly important. The purpose of this study is to investigate the impact of external vehicle acceleration signal light on the interaction experience between pedestrians and vehicles. The differences between the use and nonuse of acceleration signal light are compared through controlled test track experiments in real scenarios and in videos.The results show that acceleration signal light help pedestrians understand vehicle behavior intentions more quickly and make safer crossing decisions as well as improving their perception of safety when crossing the street and their trust in vehicle behavior.

Deceleration parameters as implicit communication signals for pedestrians' crossing decisions and estimations of automated vehicle behaviour

Society greatly expects the widespread deployment of automated vehicles (AVs). However, the absence of a driver role results in unresolved communication issues between pedestrians and AVs. Research has shown the crucial role of implicit communication signals in this context. Nonetheless, it remains unclear how pedestrians subjectively estimate vehicle behaviour and whether they incorporate these estimations as part of their crossing decisions. For the first time, this study explores the impact of implicit communication signals on pedestrians' subjective estimations of approaching vehicle behaviour across a wide range of experimental traffic scenarios and on their crossing decisions in the same scenarios through a comprehensive analysis. Two simulator tasks, namely a natural road crossing task and a vehicle behaviour estimation task, were designed with controlled time to collision, vehicle speed, and deceleration behaviour. A novel finding is that the correlation between crossing decisions and vehicle behaviour estimations depends on the traffic scenario. Pedestrians' recognition of different deceleration behaviour aligned with their crossing decisions, supporting the notion that they actively estimate vehicle behaviour as part of their decision-making process. However, if the traffic gap was long enough, the effects of vehicle speed were the opposite between crossing decisions and estimations, suggesting that vehicle behaviour estimation may not directly impact crossing decisions when the time gap to the vehicle is large. We also found that pedestrians crossed the street earlier and estimated yielding behaviour more accurately in early-onset braking scenarios than in late-onset braking scenarios. Interestingly, vehicle speed significantly affected pedestrians' estimations, with pedestrians tending to perceive low vehicle speed as yielding behaviour regardless of whether the vehicle yielded. Finally, we demonstrated that visual cue τ̇ is a practical indicator for controlling the vehicle deceleration evidence in the experiment. In conclusion, these findings reveal in detail the role of deceleration parameters as implicit communication signals between pedestrians and AVs, with implications for road crossing safety and the development of AVs.

Pedestrians' road-crossing behavior towards eHMI-equipped autonomous vehicles driving in segregated and mixed traffic conditions

Pedestrians' road-crossing behavior can be influenced by eHMIs (external Human-Machine Interfaces) on autonomous vehicles (AVs). In this research, we developed a novel eHMI concept that aimed to support pedestrians' risk evaluation by displaying predicted real-time risk levels. In a virtual reality environment, we measured pedestrians' road-crossing behavior when they encountered AVs with this eHMI and manual-driven vehicles (MVs) in the same lane. Results showed that pedestrians exhibited typical crossing behaviors based on gap size for both vehicle types. In segregated traffic conditions, compared to MVs, eHMI-equipped AVs made pedestrians more sensitive to the changes in gap size by rejecting more small gaps and accepting more large gaps. Pedestrians also walked faster and kept larger safety margins for smaller gaps. Similar results were observed for AVs in mixed traffic conditions. However, in mixed traffic conditions, pedestrians faced more challenges when interacting with MVs as they tended to accept smaller gaps, walk more slowly, and maintain smaller safety margins. These findings indicate that dynamic risk information could be conducive to pedestrians' road-crossing behavior, but the use of eHMIs on AVs might disrupt pedestrians' interactions with MVs in complex traffic conditions. This potential risk shift among vehicles also poses the question of whether AVs should use segregated lanes to reduce their indirect impacts on pedestrian-MV interactions.

External Human-Machine Interfaces for Automated Vehicles in Shared Spaces: A Review of the Human-Computer Interaction Literature

Given the rise of automated vehicles from an engineering and technical perspective, there has been increased research interest concerning the Human and Computer Interactions (HCI) between vulnerable road users (VRUs, such as cyclists and pedestrians) and automated vehicles. As with all HCI challenges, clear communication and a common understanding-in this application of shared road usage-is critical in order to reduce conflicts and crashes between the VRUs and automated vehicles. In an effort to solve this communication challenge, various external human-machine interface (eHMI) solutions have been developed and tested across the world. This paper presents a timely critical review of the literature on the communication between automated vehicles and VRUs in shared spaces. Recent developments will be explored and studies analyzing their effectiveness will be presented, including the innovative use of Virtual Reality (VR) for user assessments. This paper provides insight into several gaps in the eHMI literature and directions for future research, including the need to further research eHMI effects on cyclists, investigate the negative effects of eHMIs, and address the technical challenges of eHMI implementation. Furthermore, it has been underlined that there is a lack of research into the use of eHMIs in shared spaces, where the communication and interaction needs differ from conventional roads.

Ranking Crossing Scenario Complexity for eHMIs Testing: A Virtual Reality Study

External human–machine interfaces (eHMIs) have the potential to benefit AV–pedestrian interactions. The majority of studies investigating eHMIs have used relatively simple traffic environments, i.e., a single pedestrian crossing in front of a single eHMI on a one-lane straight road. While this approach has proved to be efficient in providing an initial understanding of how pedestrians respond to eHMIs, it over-simplifies interactions which will be substantially more complex in real-life circumstances. A process is illustrated in a small-scale study (N = 10) to rank different crossing scenarios by level of complexity. Traffic scenarios were first developed for varying traffic density, visual complexity of the road scene, road geometry, weather and visibility conditions, and presence of distractions. These factors have been previously shown to increase difficulty and riskiness of the crossing task. The scenarios were then tested in a motion-based, virtual reality environment. Pedestrians’ perceived workload and objective crossing behaviour were measured as indirect indicators of the level of complexity of the crossing scenario. Sense of presence and simulator sickness were also recorded as a measure of the ecological validity of the virtual environment. The results indicated that some crossing scenarios were more taxing for pedestrians than others, such as those with road geometries where traffic approached from multiple directions. Further, the presence scores showed that the virtual environments experienced were found to be realistic. This paper concludes by proposing a “complex” environment to test eHMIs under more challenging crossing circumstances.

Smiles and Angry Faces vs. Nods and Head Shakes: Facial Expressions at the Service of Autonomous Vehicles

When deciding whether to cross the street or not, pedestrians take into consideration information provided by both vehicle kinematics and the driver of an approaching vehicle. It will not be long, however, before drivers of autonomous vehicles (AVs) will be unable to communicate their intention to pedestrians, as they will be engaged in activities unrelated to driving. External human–machine interfaces (eHMIs) have been developed to fill the communication gap that will result by offering information to pedestrians about the situational awareness and intention of an AV. Several anthropomorphic eHMI concepts have employed facial expressions to communicate vehicle intention. The aim of the present study was to evaluate the efficiency of emotional (smile; angry expression) and conversational (nod; head shake) facial expressions in communicating vehicle intention (yielding; non-yielding). Participants completed a crossing intention task where they were tasked with deciding appropriately whether to cross the street or not. Emotional expressions communicated vehicle intention more efficiently than conversational expressions, as evidenced by the lower latency in the emotional expression condition compared to the conversational expression condition. The implications of our findings for the development of anthropomorphic eHMIs that employ facial expressions to communicate vehicle intention are discussed.

Communication between Autonomous Vehicles and Pedestrians: An Experimental Study Using Virtual Reality

One of the major challenges of autonomous vehicles (AV) is their interaction with pedestrians. Unofficial interactions such as gestures, eye contact, waving, and flashing lights are very common behavioral patterns for drivers to express their intent to give priority. In our research we composed a virtual reality experiment for a pedestrian crossing in an urban environment in order to test pedestrians' reactions on an LED light display mounted on a virtual AV. Our main research interest was to investigate whether communication patterns influence the decision making of pedestrians when crossing the road. In a VR environment, four scenarios were created with a vehicle approaching a pedestrian crossing with different speeds and displaying a special red/green sign to pedestrians. Here, 51 persons participating in the experiment had to decide when crossing is safe. Results show that the majority of people indicated they would cross in the time windows when it was actually safe to cross. Male subjects made their decision to cross slightly faster but no significant differences were found in the decision making by gender. It was found that age is not an influencing factor, either. Overall, a quick learning process was experienced proving that explicit communication patterns are self-explaining.

Differences in Pedestrian Behavior at Crosswalk between Communicating with Conventional Vehicle and Automated Vehicle in Real Traffic Environment

In this study, we examine the differences in pedestrian behavior at crosswalks between communicating with conventional vehicles (CVs) and automated vehicles (AVs). To analyze pedestrian behavior statistically, we record the pedestrian’s position (x- and y-coordinates) every 0.5 s and perform a hot spot analysis. A Toyota Prius (ZVW30) is used as the CV and AV, and the vehicle behavior is controlled using the Wizard of Oz method. An experiment is conducted on a public road in Odaiba, Tokyo, Japan, where 38 participants are recruited for each experiment involving a CV and an AV. The participants cross the road after communicating with the CV or AV. The results show that the pedestrians can cross earlier when communicating with the CV as compared with the AV. The hot spot analysis shows that pedestrians who communicate with the CV decide to cross the road before the CV stops; however, pedestrians who communicate with the AVs decide to cross the road after the AV stops. It is discovered that perceived safety does not significantly affect pedestrian behavior; therefore, earlier perceived safety by drivers’ communication and external human–machine interface is more important than higher perceived safety for achieving efficient communication.

How to improve pedestrians' trust in automated vehicles: new road infrastructure, external human-machine interface with anthropomorphism, or conventional road signaling?

Introduction

Automated vehicles need to gain the trust of all road users in order to be accepted. To make technology trustworthy, automated vehicles must transmit crucial information to pedestrians through a human-machine interface, allowing pedestrians to accurately predict and act on their next behavior. However, the unsolved core issue in the field of vehicle automation is to know how to successfully communicate with pedestrians in a way that is efficient, comfortable, and easy to understand. This study investigated the impact of three human-machine interfaces specifically designed for pedestrians' trust during the street crossing in front of an automated vehicle. The interfaces used different communication channels to interact with pedestrians, i.e., through a new road infrastructure, an external human-machine interface with anthropomorphism, or with conventional road signaling.

Methods

Mentally projected in standard and non-standard use cases of human-machine interfaces, 731 participants reported their feelings and behavior through an online survey.

Results

Results showed that human-machine interfaces were efficient to improve trust and willingness to cross the street in front of automated vehicles. Among external human-machine interfaces, anthropomorphic features showed significant advantages in comparison with conventional road signals to induce pedestrians' trust and safer crossing behaviors. More than the external human-machine interfaces, findings highlighted the efficiency of the trust-based road infrastructure on the global street crossing experience of pedestrians with automated vehicles.

Discussion

All of these findings support trust-centered design to anticipate and build safe and satisfying human-machine interactions.

Effect of Signal Design of Autonomous Vehicle Intention Presentation on Pedestrians' Cognition

In this study, a method is devised that allows the intentions of autonomous vehicles to be effectively communicated to pedestrians and passengers via an efficient interactive interface. Visual and auditory factors are used as variables to investigate the effects of different autonomous vehicle signal factors on the judgment of pedestrians and to determine the main factors such that the best combination can be proposed. Two visual dimensions (i.e., color and flashing) and three auditory dimensions (i.e., rhythm, frequency, and melody) are used as the experimental signal variables. In addition, deceleration and waiting-to-restart scenarios are investigated. Multiple-choice questions and a subjective cognition scale are used for evaluation. The results show that the combination of green and slow rhythm can be used for the road-user-first case, whereas the combination of red and fast rhythm can be used for the vehicle-first case. Under the same intention, factors of color, flashing, rhythm, and melody are highly similar in terms of the combination mode, except for the frequency. In the deceleration and waiting-to-restart scenarios, the frequencies of the best signal are high and low frequencies, respectively. The results of this study can be used as a reference for the signal design of autonomous vehicles in the future and provide ideas for the interactions between autonomous vehicles and pedestrians.

Stopping by looking: A driver-pedestrian interaction study in a coupled simulator using head-mounted displays with eye-tracking

Automated vehicles (AVs) can perform low-level control tasks but are not always capable of proper decision-making. This paper presents a concept of eye-based maneuver control for AV-pedestrian interaction. Previously, it was unknown whether the AV should conduct a stopping maneuver when the driver looks at the pedestrian or looks away from the pedestrian. A two-agent experiment was conducted using two head-mounted displays with integrated eye-tracking. Seventeen pairs of participants (pedestrian and driver) each interacted in a road crossing scenario. The pedestrians' task was to hold a button when they felt safe to cross the road, and the drivers' task was to direct their gaze according to instructions. Participants completed three 16-trial blocks: (1) Baseline, in which the AV was pre-programmed to yield or not yield, (2) Look to Yield (LTY), in which the AV yielded when the driver looked at the pedestrian, and (3) Look Away to Yield (LATY), in which the AV yielded when the driver did not look at the pedestrian. The driver's eye movements in the LTY and LATY conditions were visualized using a virtual light beam. Crossing performance was assessed based on whether the pedestrian held the button when the AV yielded and released the button when the AV did not yield. Furthermore, the pedestrians' and drivers' acceptance of the mappings was measured through a questionnaire. The results showed that the LTY and LATY mappings yielded better crossing performance than Baseline. Furthermore, the LTY condition was best accepted by drivers and pedestrians. Eye-tracking analyses indicated that the LTY and LATY mappings attracted the pedestrian's attention, while pedestrians still distributed their attention between the AV and a second vehicle approaching from the other direction. In conclusion, LTY control may be a promising means of AV control at intersections before full automation is technologically feasible.

Crossing the street in front of an autonomous vehicle: An investigation of eye contact between drivengers and vulnerable road users

Communication between road users is a major key to coordinate movement and increase roadway safety. The aim of this work was to grasp how pedestrians (Experiment A), cyclists (Experiment B), and kick scooter users (Experiment C) sought to visually communicate with drivengers when they would face autonomous vehicles (AVs). In each experiment, participants (n = 462, n = 279, and n = 202, respectively) were asked to imagine themselves in described situations of encounters between a specific type of vulnerable road user (e.g., pedestrian) and a human driver in an approaching car. The human driver state and the communicative means of the approaching car through an external Human-Machine Interface (eHMI) were manipulated between the scenarios. The participants were prompted to rate from “never” to “always” (6-point Likert scale) the frequency with which they would seek eye contact with the human driver either in order to express their willingness to cross or to make their effective decision to cross. Our findings revealed that a passive human driver in an AV with no visual checking on the road triggered a decline in vulnerable road users’ desire to communicate by eye contact (Experiments A–C). Moreover, the results of Experiment C demonstrated that the speed screen, the text message screen, and the vibrating mobile app eHMI signals diminished kick scooter users’ desire to communicate visually with the human driver, with some age-based differences. This suggested a better comprehension of the approaching car’s intentions by the kick scooter users, driven by the features of the eHMI.

External communication of automated shuttles: Results, experiences, and lessons learned from three European long-term research projects

Automated shuttles are already seeing deployment in many places across the world and have the potential to transform public mobility to be safer and more accessible. During the current transition phase from fully manual vehicles toward higher degrees of automation and resulting mixed traffic, there is a heightened need for additional communication or external indicators to comprehend automated vehicle actions for other road users. In this work, we present and discuss the results from seven studies (three preparatory and four main studies) conducted in three European countries aimed at investigating and providing a variety of such external communication solutions to facilitate the exchange of information between automated shuttles and other motorized and non-motorized road users.

Ghost on the Windshield: Employing a Virtual Human Character to Communicate Pedestrian Acknowledgement and Vehicle Intention

Pedestrians base their street-crossing decisions on vehicle-centric as well as driver-centric cues. In the future, however, drivers of autonomous vehicles will be preoccupied with non-driving related activities and will thus be unable to provide pedestrians with relevant communicative cues. External human–machine interfaces (eHMIs) hold promise for filling the expected communication gap by providing information about a vehicle’s situational awareness and intention. In this paper, we present an eHMI concept that employs a virtual human character (VHC) to communicate pedestrian acknowledgement and vehicle intention (non-yielding; cruising; yielding). Pedestrian acknowledgement is communicated via gaze direction while vehicle intention is communicated via facial expression. The effectiveness of the proposed anthropomorphic eHMI concept was evaluated in the context of a monitor-based laboratory experiment where the participants performed a crossing intention task (self-paced, two-alternative forced choice) and their accuracy in making appropriate street-crossing decisions was measured. In each trial, they were first presented with a 3D animated sequence of a VHC (male; female) that either looked directly at them or clearly to their right while producing either an emotional (smile; angry expression; surprised expression), a conversational (nod; head shake), or a neutral (neutral expression; cheek puff) facial expression. Then, the participants were asked to imagine they were pedestrians intending to cross a one-way street at a random uncontrolled location when they saw an autonomous vehicle equipped with the eHMI approaching from the right and indicate via mouse click whether they would cross the street in front of the oncoming vehicle or not. An implementation of the proposed concept where non-yielding intention is communicated via the VHC producing either an angry expression, a surprised expression, or a head shake; cruising intention is communicated via the VHC puffing its cheeks; and yielding intention is communicated via the VHC nodding, was shown to be highly effective in ensuring the safety of a single pedestrian or even two co-located pedestrians without compromising traffic flow in either case. The implications for the development of intuitive, culture-transcending eHMIs that can support multiple pedestrians in parallel are discussed.

Macro-level literature analysis on pedestrian safety: Bibliometric overview, conceptual frames, and trends

Due to the high volume of documents in the pedestrian safety field, the current study conducts a systematic bibliometric analysis on the researches published before October 3, 2021, based on the science-mapping approach. Science mapping enables us to present a broad picture and comprehensive review of a significant number of documents using co-citation, bibliographic coupling, collaboration, and co-word analysis. To this end, a dataset of 6311 pedestrian safety papers was collected from the Web of Science Core Collection database. First, a descriptive analysis was carried out, covering whole yearly publications, most-cited papers, and most-productive authors, as well as sources, affiliations, and countries. In the next steps, science mapping was implemented to clarify the social, intellectual, and conceptual structures of pedestrian-safety research using the VOSviewer and Bibliometrix R-package tools. Remarkably, based on intellectual structure, pedestrian safety demonstrated an association with seven research areas: "Pedestrian crash frequency models", "Pedestrian injury severity crash models", "Traffic engineering measures in pedestrians' safety", "Global reports around pedestrian accident epidemiology", "Effect of age and gender on pedestrians' behavior", "Distraction of pedestrians", and "Pedestrian crowd dynamics and evacuation". Moreover, according to conceptual structure, five major research fronts were found to be relevant, namely "Collision avoidance and intelligent transportation systems (ITS)", "Epidemiological studies of pedestrian injury and prevention", "Pedestrian road crossing and behavioral factors", "Pedestrian flow simulation", and "Walkable environment and pedestrian safety". Finally, "autonomous vehicle", "pedestrian detection", and "collision avoidance" themes were identified as having the greatest centrality and development degrees in recent years.

External Human-Machine Interfaces Can Be Misleading: An Examination of Trust Development and Misuse in a CAVE-Based Pedestrian Simulation Environment

OBJECTIVE

To investigate pedestrians' misuse of an automated vehicle (AV) equipped with an external human-machine interface (eHMI). Misuse occurs when a pedestrian enters the road because of uncritically following the eHMI's message.

BACKGROUND

Human factors research indicates that automation misuse is a concern. However, there is no consensus regarding misuse of eHMIs.

METHODS

Sixty participants each experienced 50 crossing trials in a Cave Automatic Virtual Environment (CAVE) simulator. The three independent variables were as follows: (1) behavior of the approaching AV (within-subject: yielding at 33 or 43 m distance, no yielding), (2) eHMI presence (within-subject: eHMI on upon yielding, off), and (3) eHMI onset timing (between-subjects: eHMI turned on 1 s before or 1 s after the vehicle started to decelerate). Two failure trials were included where the eHMI turned on, yet the AV did not yield. Dependent measures were the moment of entering the road and perceived risk, comprehension, and trust.

RESULTS

Trust was higher with eHMI than without, and the -1 Group crossed earlier than the +1 Group. In the failure trials, perceived risk increased to high levels, whereas trust and comprehension decreased. Thirty-five percent of the participants in the -1 and +1 Groups walked onto the road when the eHMI failed for the first time, but there were no significant differences between the two groups.

CONCLUSION

eHMIs that provide anticipatory information stimulate early crossing. eHMIs may cause people to over-rely on the eHMI and under-rely on the vehicle-intrinsic cues.

APPLICATION

eHMI have adverse consequences, and education of eHMI capability is required.

A Study of Bibliometric Trends in Automotive Human–Machine Interfaces

With the development of autonomous driving technology and the internet, automotive human–machine interface (HMI) technology has become an important part of contemporary automotive design. Currently, global automakers are designing a variety of innovative in-car HMIs that illustrate the direction of automotive design in the new era from the perspective of technological aesthetics and experience design. However, sleek designs and innovative experience methods must be built on the basis of safety. Therefore, it is necessary to summarize existing research in the field of automotive HMI and construct a literature review of automotive design research. In this paper, literature on automotive HMI from the Scopus database was analyzed using bibliometric methods such as descriptive analysis, keyword co-occurrence, and literature co-citation network analysis. The final mapping analysis revealed that the current automotive HMI research literature primarily focuses on user research, interface research, external environment research, and technology implementation research related to automotive HMI. The three main stages of automotive HMI research include conceptual construction, system and technology refinement, and user perception research from the perspective of driver assistance and information recognition. Additionally, burst detection suggests that future research should focus on driver assistance, trust levels, and e-HMI information communication.

Toward a Holistic Communication Approach to an Automated Vehicle's Communication With Pedestrians: Combining Vehicle Kinematics With External Human-Machine Interfaces for Differently Sized Automated Vehicles

Future automated vehicles (AVs) of different sizes will share the same space with other road users, e. g., pedestrians. For a safe interaction, successful communication needs to be ensured, in particular, with vulnerable road users, such as pedestrians. Two possible communication means exist for AVs: vehicle kinematics for implicit communication and external human-machine interfaces (eHMIs) for explicit communication. However, the exact interplay is not sufficiently studied yet for pedestrians' interactions with AVs. Additionally, very few other studies focused on the interplay of vehicle kinematics and eHMI for pedestrians' interaction with differently sized AVs, although the precise coordination is decisive to support the communication with pedestrians. Therefore, this study focused on how the interplay of vehicle kinematics and eHMI affects pedestrians' willingness to cross, trust and perceived safety for the interaction with two differently sized AVs (smaller AV vs. larger AV). In this experimental online study (N = 149), the participants interacted with the AVs in a shared space. Both AVs were equipped with a 360° LED light-band eHMI attached to the outer vehicle body. Three eHMI statuses (no eHMI, static eHMI, and dynamic eHMI) were displayed. The vehicle kinematics were varied at two levels (non-yielding vs. yielding). Moreover, “non-matching” conditions were included for both AVs in which the dynamic eHMI falsely communicated a yielding intent although the vehicle did not yield. Overall, results showed that pedestrians' willingness to cross was significantly higher for the smaller AV compared to the larger AV. Regarding the interplay of vehicle kinematics and eHMI, results indicated that a dynamic eHMI increased pedestrians' perceived safety when the vehicle yielded. When the vehicle did not yield, pedestrians' perceived safety still increased for the dynamic eHMI compared to the static eHMI and no eHMI. The findings of this study demonstrated possible negative effects of eHMIs when they did not match the vehicle kinematics. Further implications for a holistic communication strategy for differently sized AVs will be discussed.

Deviant Behavior of Pedestrians: A Risk Gamble or Just Against Automated Vehicles? How About Social Control?

Recent evidence suggests that the assumed conflict-avoidant programming of autonomous vehicles will incentivize pedestrians to bully them. However, this frequent argument disregards the embedded nature of social interaction. Rule violations are socially sanctioned by different forms of social control, which could moderate the rational incentive to abuse risk-avoidant vehicles. Drawing on a gamified virtual reality (VR) experiment (n = 36) of urban traffic scenarios, we tested how vehicle type, different forms of social control, and monetary benefit of rule violations affect pedestrians’ decision to jaywalk. In a second step, we also tested whether differences in those effects exist when controlling for the risk of crashes in conventional vehicles. We find that individuals do indeed jaywalk more frequently when faced with an automated vehicle (AV), and this effect largely depends on the associated risk and not their automated nature. We further show that social control, especially in the form of formal traffic rules and norm enforcement, can reduce jaywalking behavior for any vehicle. Our study sheds light on the interaction dynamics between humans and AVs and how this is influenced by different forms of social control. It also contributes to the small gamification literature in this human–computer interaction.

Methods of Generating Emotional Movements and Methods of Transmitting Behavioral Intentions: A Perspective on Human-Coexistence Robots

The purpose of this paper is to introduce and discuss the following two functions that are considered to be important in human-coexistence robots and human-symbiotic robots: the method of generating emotional movements, and the method of transmitting behavioral intentions. The generation of emotional movements is to design the bodily movements of robots so that humans can feel specific emotions. Specifically, the application of Laban movement analysis, the development from the circumplex model of affect, and the imitation of human movements are discussed. However, a general technique has not yet been established to modify any robot movement so that it contains a specific emotion. The transmission of behavioral intentions is about allowing the surrounding humans to understand the behavioral intentions of robots. Specifically, informative motions in arm manipulation and the transmission of the movement intentions of robots are discussed. In the former, the target position in the reaching motion, the physical characteristics in the handover motion, and the landing distance in the throwing motion are examined, but there are still few research cases. In the latter, no groundbreaking method has been proposed that is fundamentally different from earlier studies. Further research and development are expected in the near future.

Go Ahead, Please!—Evaluation of External Human—Machine Interfaces in a Real-World Crossing Scenario

In the future, automated vehicles (AVs) without a human driver will potentially have to manage communication with vulnerable road users, such as pedestrians, in everyday traffic interaction situations. The aim of this work is to investigate pedestrian reactions to external communication concepts in a controlled, but real-world crossing scenario. The focus is to investigate which properties of external human–machine interfaces (eHMIs) promote the comprehensibility of vehicle intention (yielding for the pedestrian) and therefore lead to faster and, at the same time, safer crossing decisions of pedestrians. For this purpose, three different eHMI concepts (intention-based light-band, perception-based light-band, and the combination of light-band and signal lamp) were examined and compared to a baseline (no eHMI). In a Wizard-of-Oz experiment, participants (n = 30) encountered a test vehicle equipped with the eHMIs in a real-world crossing scenario. The crossing initiation time in seconds and the participant's intention recognition were measured. Furthermore, the influence of the eHMIs on acceptance and perceived safety was evaluated. It was shown that the presence of the intention-based light-band, and the combination of light-band and signal lamp led to an earlier crossing decision compared to baseline with no eHMI. In summary, the results indicate that the intention-based light-band has a positive effect on the comprehensibility of the vehicle's intention. All concepts were evaluated positively regarding acceptance and perceived safety, and did not differ significantly from each other.

Team Citizenship Pressure: How Does It Relate to OCB and Citizenship Fatigue

The aim of this research is to shed light on the phenomenon of citizenship pressure as a team-level construct. Building on the conservation of resources theory, the study used a moderated-mediation model to explore whether team organizational citizenship behavior (OCB) mediates the relationship between team citizenship pressure and team citizenship fatigue and whether this mediation is moderated by perceived supervisor support. Results from a study of 91 professional teams in the educational system indicate that team citizenship pressure had a significant and positive relationship with team OCB, as well as with team citizenship fatigue. The results also support the overall moderated-mediation model, but contrary to the hypothesized pattern of interaction, we found that team citizenship pressure was significantly and positively correlated with OCB when perceived supervisor support was low, but not when it was high. Limitations and implications for future research are discussed.

One Solution Fits All? Evaluating Different Communication Strategies of a Light-based External Human-Machine Interface for Differently Sized Automated Vehicles from a Pedestrian's Perspective

Differently sized automated vehicles (AVs) will enter the roads of tomorrow and will interact with other road users. Pedestrians as vulnerable road users heavily rely on the communication with other road users, especially for the interaction with larger vehicles, as miscommunication pose a high risk. Therefore, AVs need to provide communication abilities to safely interact with pedestrians. This study's focus was on the explicit communication which is highly relevant in low-speed and low-distance traffic scenarios to clarify misunderstandings before they result in accidents. External human-machine interfaces (eHMIs) placed on the outside of AVs can be used as a communication tool to explicitly inform the surrounding traffic environment. Although research manifested effects of vehicle size on pedestrians' perceived safety and crossing behavior, little research about the eHMI design for differently sized AVs exists. This experimental online study (N = 155) aimed at investigating the application of a light-based eHMI on two differently sized AVs (car, bus) by focusing on the overall goal of ensuring traffic safety in future traffic. The light-based eHMI showed different communication strategies, i.e., a static eHMI and three dynamic eHMIs. The results revealed that an automated car was perceived as safer and affectively rated as more positive compared to an automated bus. Nevertheless, no significant differences were found between the two AVs in terms of the eHMI communication. A dynamic eHMI was perceived as safer and evaluated affectively as more positive compared to a static eHMI or no eHMI for both AVs. In conclusion, the use of a light-based eHMI had a positive effect on pedestrians' interaction with an automated car and an automated bus and, therefore, could contribute to the overall traffic safety in this study. Implications for the design of eHMIs for differently sized AVs were discussed.

Language of Driving for Autonomous Vehicles

Environmental awareness and technological advancements for self-driving cars are close to making autonomous vehicles (AV) a reality in everyday scenarios and a part of smart cities’ transportation systems. The perception of safety and trust towards AVs of passengers and other agents in the urban scenario, being pedestrians, cyclists, scooter drivers or car drivers, is of primary importance and the theme of investigation of many research groups. Driver-to-driver communication channels as much as car-to-driver human–machine interfaces (HMI) are well established and part of normal training and experience. The situation is different when users must cope with driverless and autonomous vehicles, both as passengers and as agents sharing the same urban domain. This research focuses on the new challenges of connected driverless vehicles, investigating an emerging topic, namely the language of driving (LoD) between these machines and humans participating in traffic scenarios. This work presents the results of a field study conducted at Tallinn University Technology campus with the ISEAUTO autonomous driving shuttle, including interviews with 176 subjects communicating using LoD. Furthermore, this study combines expert focus group interviews to build a joint base of needs and requirements for AVs in public spaces. Based on previous studies and questionnaire results, we established the hypotheses that we can enhance physical survey results using experimental scenarios with VR/AR tools to allow the fast prototyping of different external and internal HMIs, facilitating the assessment of communication efficacy, evaluation of usability, and impact on the users. The aim is to point out how we can enhance AV design and LoD communications using XR tools. The scenarios were chosen to be inclusive and support the needs of different demographics while at the same time determining the limitations of surveys and real-world experimental scenarios in LoD testing and design for future pilots.

Designing Interactions With Shared AVs in Complex Urban Mobility Scenarios

In this article, we report on the design and evaluation of an external human-machine interface (eHMI) for a real autonomous vehicle (AV), developed to operate as a shared transport pod in a pedestrianized urban space. We present insights about our human-centered design process, which included testing initial concepts through a tangible toolkit and evaluating 360-degree recordings of a staged pick-up scenario in virtual reality. Our results indicate that in complex mobility scenarios, participants filter for critical eHMI messages; further, we found that implicit cues (i.e., pick-up manoeuvre and proximity to the rider) influence participants' experience and trust, while at the same time more explicit interaction modes are desired. This highlights the importance of considering interactions with shared AVs as a service more holistically, in order to develop knowledge about AV-pedestrian interactions in complex mobility scenarios that complements more targeted eHMI evaluations.

A Video-Based, Eye-Tracking Study to Investigate the Effect of eHMI Modalities and Locations on Pedestrian–Automated Vehicle Interaction

Numerous studies have emerged on the external human–machine interface (eHMI) to facilitate the communication between automated vehicles (AVs) and other road users. However, it remains to be determined which eHMI modality and location are proper for the pedestrian–AV interaction. Therefore, a video-based, eye-tracking study was performed to investigate how pedestrians responded to AVs with eHMIs in different modalities (flashing text, smiley, light band, sweeping pedestrian icon, arrow, and light bar) and locations (grill, windshield, and roof). Moreover, the effects of pedestrian-related factors (e.g., gender, sensation-seeking level, and traffic accident involvement) were also included and evaluated. The dependent variables included pedestrians’ clarity-rating scores towards these eHMI concepts, road-crossing decision time, and gaze-based metrics (e.g., fixation counts, dwell time, and first fixation duration). The results showed that the text, icon, and arrow-based eHMIs resulted in the shortest decision time, highest clarity scores, and centralized visual attention. The light strip-based eHMIs yielded no significant decrease in decision time yet longer fixation time, indicating difficulties in comprehension of their meaning without learning. The eHMI location had no effect on pedestrians’ decision time but a substantial influence on their visual searching strategy, with a roof eHMI contradicting pedestrians’ inherent scanning pattern. These findings provide implications for the standardized design of future eHMIs.

The Effect of eHMI Malfunctions on Younger and Elderly Pedestrians' Trust and Acceptance of Automated Vehicle Communication Signals

To ensure traffic flow and road safety in automated driving, external human-machine interfaces (eHMIs) could prospectively support the interaction between automated vehicles (AVs; SAE Level 3 or higher) and pedestrians if implicit communication is insufficient. Particularly elderly pedestrians (≥65 years) who are notably vulnerable in terms of traffic safety might benefit of the advantages of additional signals provided by eHMIs. Previous research showed that eHMIs were assessed as useful means of communication in AVs and were preferred over exclusively implicit communication signals. However, the attitudes of elderly users regarding technology usage and acceptance are ambiguous (i.e., less intention to use technology vs. a tendency toward overreliance on technology compared to younger users). Considering potential eHMI malfunctions, an appropriate level of trust in eHMIs is required to ensure traffic safety. So far, little research respected the impact of multiple eHMI malfunctions on participants' assessment of the system. Moreover, age effects were rarely investigated in eHMIs. In the current monitor-based study, N = 36 participants (19 younger, 17 elderly) repeatedly assessed an eHMI: During an initial measurement, when encountering a valid system and after experiencing eHMI malfunctions. Participants indicated their trust and acceptance in the eHMI, feeling of safety during the interaction and vigilance toward the eHMI. The results showed a positive effect of interacting with a valid system that acted consistently to the vehicle's movements compared to an initial assessment of the system. After experiencing eHMI malfunctions, participants' assessment of the system declined significantly. Moreover, elderly participants assessed the eHMI more positive across all conditions than younger participants did. The findings imply that participants considered the vehicle's movements as implicit communication cues in addition to the provided eHMI signals during the encounters. To support traffic safety and smooth interactions, eHMI signals are required to be in line with vehicle's movements as implicit communication cues. Moreover, the results underline the importance of calibrating an appropriate level of trust in eHMI signals. An adequate understanding of eHMI signals needs to be developed. Thereby, the requirements of different user groups should be specifically considered.

External Human-Machine Interfaces for Autonomous Vehicles from Pedestrians' Perspective: A Survey Study

With the increasing number of automated vehicles (AVs) being tested and operating on roads, external Human–Machine Interfaces (eHMIs) are proposed to facilitate interactions between AVs and other road users. Considering the need to protect vulnerable road users, this paper addresses the issue by providing research evidence on various designs of eHMIs. Ninety participants took part in this experiment. Six sets of eHMI prototypes—Text, Arrowed (Dynamic), Text and Symbol, Symbol only, Tick and Cross and Traffic Lights, including two sub-designs (Cross and Do Not Cross)—were designed. The results showed that 65.1% of participants agreed that external communication would have a positive effect on pedestrians’ crossing decisions. Among all the prototypes, Text, and Text and Symbol, eHMIs were the most widely accepted. In particular, for elderly people and those unfamiliar with traffic rules, Text, and Text and Symbol, eHMIs would lead to faster comprehension. The results confirmed that 68.5% of participants would feel safer crossing if the eHMI had the following features: ‘Green’, ‘Text’, ‘Symbol’, or ‘Dynamic’. These features are suggested in the design of future systems. This research concluded that eHMIs have a positive effect on V2X communication and that textual eHMIs were clear to pedestrians.

Analyzing Pedestrian Behavior at Unsignalized Crosswalks from the Drivers’ Perspective: A Qualitative Study

This study investigated drivers’ perceptions of pedestrian crossing behavior at unsignalized crosswalks, which was less fruitful in quantitative and qualitative traffic research. Subjective and snow-ball sampling were used to conduct semi-structured in-depth interviews based on drivers’ daily driving experience from qualitative research. A theoretical model of pedestrian behavior at unsignalized crosswalks was constructed using the grounded theory and the theoretical saturation test. The model involved 4 three-level codes and 13 two-level codes (main category) used to obtain seven subcategories. The results show that drivers believe that pedestrian characteristics, driver characteristics, and age factors are the three factors that affect pedestrian crossing safety. Targeted improvement measures are put forward to guide the design of pedestrian crossing facilities, pedestrian management and guidance, and future research on conflicts between autonomous vehicles and pedestrians.

Automated vehicles that communicate implicitly: examining the use of lateral position within the lane

It may be necessary to introduce new modes of communication between automated vehicles (AVs) and pedestrians. This research proposes using the AV's lateral deviation within the lane to communicate if the AV will yield to the pedestrian. In an online experiment, animated video clips depicting an approaching AV were shown to participants. Each of 1104 participants viewed 28 videos twice in random order. The videos differed in deviation magnitude, deviation onset, turn indicator usage, and deviation-yielding mapping. Participants had to press and hold a key as long as they felt safe to cross, and report the perceived intuitiveness of the AV's behaviour after each trial. The results showed that the AV moving towards the pedestrian to indicate yielding and away to indicate continuing driving was more effective than the opposite combination. Furthermore, the turn indicator was regarded as intuitive for signalling that the AV will yield. Practitioner Summary: Future automated vehicles (AVs) may have to communicate with vulnerable road users. Many researchers have explored explicit communication via text messages and led strips on the outside of the AV. The present study examines the viability of implicit communication via the lateral movement of the AV.

Improving the experience in the pedestrian's interaction with an autonomous vehicle: An ergonomic comparison of external HMI

The number of studies on autonomous vehicles has increased over recent years. Many of these studies have indicated the importance of an external Human-Machine Interface of communication (eHMI) on autonomous vehicles to indicate their intentions to other road users. Using an experimental design, we compared three eHMIs coupled to three road infrastructures to observe pedestrians' crossing behavior and collect their feelings about different vehicle types. Our results showed that the eHMIs influence the pedestrians' decision to cross the street, confirming the importance of setting up eHMIs. The proportion of pedestrians who crossed in front of the autonomous vehicles was more significant for vehicles equipped with an eHMI than vehicles without an eHMI. In 10% of cases, pedestrians used circumvention strategies rather than crossing in front of a vehicle without an eHMI. This behavior was more often observed when there was no protected infrastructure. Finally, while our objective data failed to indicate whether a specific eHMI is better accepted than another, the subjective data on the participants' preferences provided some promising ideas for further studies and the eHMI final implementation.

Communication via motion - Suitability of automated vehicle movements to negotiate the right of way in road bottleneck scenarios

The introduction of automated vehicles (AVs) into urban areas initially leads to mixed traffic, consisting of AVs, human drivers, and vulnerable road users. Since the AV's passenger is no longer actively involved in the driving task, there may be changes in the interaction between AVs and surrounding human road users. Therefore, it is essential for an AV to behave in a comprehensible manner in order to maintain or even enhance traffic efficiency and traffic safety. This work investigates the interaction of an AV and a simultaneously oncoming human driver at road bottlenecks due to double-parked vehicles on both sides of the road. Based on findings derived from AV-pedestrian interaction, comfort limits in terms of driving dynamics, and traffic observations, we designed nine AV movements to either yield the right of way or to insist on it by varying the AV's speed (maintain speed, one-step deceleration, two-step deceleration) and its lateral offset (no offset, close offset, distant offset). The different vehicle movements were evaluated with 34 participants in a driving simulator study. The results show participants' shorter passing times, fewer crashes, and significantly higher ratings of the AV's communication if the AV movement contained a lateral offset. In addition to the regular encounters, we analyzed the controllability of an automation failure and its aftereffect on participants' trust in AVs. The experience of the automation failure reduced the trust rating significantly. From the results we conclude that the AV should communicate the right of way not only via speed adjustments but also via the performance of a lateral offset to enhance traffic efficiency and safety. Moreover, a change in the AV's maneuver due to an automation failure must be avoided since it is not controllable by the human driver.

How should external human-machine interfaces behave? Examining the effects of colour, position, message, activation distance, vehicle yielding, and visual distraction among 1,434 participants

External human-machine interfaces (eHMIs) may be useful for communicating the intention of an automated vehicle (AV) to a pedestrian, but it is unclear which eHMI design is most effective. In a crowdsourced experiment, we examined the effects of (1) colour (red, green, cyan), (2) position (roof, bumper, windshield), (3) message (WALK, DON'T WALK, WILL STOP, WON'T STOP, light bar), (4) activation distance (35 or 50 m from the pedestrian), and (5) the presence of visual distraction in the environment, on pedestrians' perceived safety of crossing the road in front of yielding and non-yielding AVs. Participants (N = 1434) had to press a key when they felt safe to cross while watching a random 40 out of 276 videos of an approaching AV with eHMI. Results showed that (1) green and cyan eHMIs led to higher perceived safety of crossing than red eHMIs; no significant difference was found between green and cyan, (2) eHMIs on the bumper and roof were more effective than eHMIs on the windshield, (3) for yielding AVs, perceived safety was higher for WALK compared to WILL STOP, followed by the light bar; for non-yielding AVs, a red bar yielded similar results to red text, (4) for yielding AVs, a red bar caused lower perceived safety when activated early compared to late, whereas green/cyan WALK led to higher perceived safety when activated late compared to early, and (5) distraction had no significant effect. We conclude that people adopt an egocentric perspective, that the windshield is an ineffective position, that the often-recommended colour cyan may have to be avoided, and that eHMI activation distance has intricate effects related to onset saliency.

Understanding the Headless Rider: Display-Based Awareness and Intent-Communication in Automated Vehicle-Pedestrian Interaction in Mixed Traffic

Automated vehicles do not yet have clearly defined signaling methods towards other road users, which could complement natural communication practices with human drivers, such as eye contact or hand gestures. In order to establish trust, external human–machine interfaces (eHMIs) have been proposed, but so far, these have not been widely evaluated in natural traffic contexts. This paper presents a user study where 30 participants interacted with a functional display-based visual eHMI for an automated shuttle in mixed urban traffic. Two distinct features were investigated: the communication of (1) its awareness of different obstacles on the road ahead and (2) of its intention to start or to brake. The results indicate that the majority of participants in general regarded eHMIs as necessary for automated vehicles. When reflecting their experience with the eHMIs, about half of the participants experienced an increased comprehension and safety. The combined presentation of obstacle awareness and vehicle intentions helped more participants to understand the shuttle’s behavior than the presentation of obstacle awareness only, but fewer participants regarded this combination of awareness and intent to be safe. The strength of the found effects on subjective responses varied with regard to age and gender.

A Systematic Review of Virtual Reality Applications for Automated Driving: 2009–2020

While virtual reality (VR) interfaces have been researched extensively over the last decades, studies on their application in vehicles have only recently advanced. In this paper, we systematically review 12 years of VR research in the context of automated driving (AD), from 2009 to 2020. Due to the multitude of possibilities for studies with regard to VR technology, at present, the pool of findings is heterogeneous and non-transparent. We investigated N = 176 scientific papers of relevant journals and conferences with the goal to analyze the status quo of existing VR studies in AD, and to classify the related literature into application areas. We provide insights into the utilization of VR technology which is applicable at specific level of vehicle automation and for different users (drivers, passengers, pedestrians) and tasks. Results show that most studies focused on designing automotive experiences in VR, safety aspects, and vulnerable road users. Trust, simulator and motion sickness, and external human-machine interfaces (eHMIs) also marked a significant portion of the published papers, however a wide range of different parameters was investigated by researchers. Finally, we discuss a set of open challenges, and give recommendation for future research in automated driving at the VR side of the reality-virtuality continuum.

Impact of External Human–Machine Interface Communication Strategies of Automated Vehicles on Pedestrians’ Crossing Decisions and Behaviors in an Urban Environment

The development of automated vehicles (AVs) and their integration into traffic are seen by many vehicle manufacturers and stakeholders such as cities or transportation companies as a revolution in mobility. In future urban traffic, it is more likely that AVs will operate not in separated traffic spaces but in so-called mixed traffic environments where different types of traffic participants interact. Therefore, AVs must be able to communicate with other traffic participants, e.g., pedestrians as vulnerable road users (VRUs), to solve ambiguous traffic situations. To achieve well-working communication and thereby safe interaction between AVs and other traffic participants, the latest research discusses external human–machine interfaces (eHMIs) as promising communication tools. Therefore, this study examines the potential positive and negative effects of AVs equipped with static (only displaying the current vehicle automation status (VAS)) and dynamic (communicating an AV’s perception and intention) eHMIs on the interaction with pedestrians by taking subjective and objective measurements into account. In a Virtual Reality (VR) simulator study, 62 participants were instructed to cross a street while interacting with non-automated (without eHMI) and automated vehicles (equipped with static eHMI or dynamic eHMI). The results reveal that a static eHMI had no effect on pedestrians’ crossing decisions and behaviors compared to a non-automated vehicle without any eHMI. However, participants benefit from the additional information of a dynamic eHMI by making earlier decisions to cross the street and higher certainties regarding their decisions when interacting with an AV with a dynamic eHMI compared to an AV with a static eHMI or a non-automated vehicle. Implications for a holistic evaluation of eHMIs as AV communication tools and their safe introduction into traffic are discussed based on the results.

Pedestrian assessment: Is displaying automated driving mode in self-driving vehicles as relevant as emitting an engine sound in electric vehicles?

Pedestrians rely on vehicle dynamics, engine sound, and driver cues. The lack of engine sound now constitutes an addressed pedestrian safety issue for (hybrid) electric vehicles ((H)EVs). Analogously, lacking driver cues may constitute a pedestrian safety issue for self-driving vehicles (SDVs). The purpose of this study was to systematically compare the relevance of substituting driver cues with an external human-machine interface among SDVs (no eHMI vs. eHMI) with the relevance of substituting engine sound with artificial sound among (H)EVs (no engine sound vs. engine sound). In a within-subject design, twenty-nine participants acting as pedestrians encountered a simulated SDV in a parking lot. The results revealed that both informational cues have equally large effects on subjective measures such as perceived safety. In semi-structured interviews, participants stated that it is equally crucial to equip SDVs with an eHMI as equipping (H)EVs with an artificial sound generator. We conclude that an eHMI for SDVs seems to be as relevant as an artificial sound for (H)EVs.

The Perceptual Belief Problem

The explainability of robotic systems depends on people’s ability to reliably attribute perceptual beliefs to robots, i.e., what robots know (or believe) about objects and events in the world based on their perception. However, the perceptual systems of robots are not necessarily well understood by the majority of people interacting with them. In this article, we explain why this is a significant, difficult, and unique problem in social robotics. The inability to judge what a robot knows (and does not know) about the physical environment it shares with people gives rise to a host of communicative and interactive issues, including difficulties to communicate about objects or adapt to events in the environment. The challenge faced by social robotics researchers or designers who want to facilitate appropriate attributions of perceptual beliefs to robots is to shape human–robot interactions so that people understand what robots know about objects and events in the environment. To meet this challenge, we argue, it is necessary to advance our knowledge of when and why people form incorrect or inadequate mental models of robots’ perceptual and cognitive mechanisms. We outline a general approach to studying this empirically and discuss potential solutions to the problem.

How do pedestrians distribute their visual attention when walking through a parking garage? An eye-tracking study

We examined what pedestrians look at when walking through a parking garage. Thirty-six participants walked a short route in a parking garage while their eye movements and head rotations were recorded with a Tobii Pro Glasses 2 eye-tracker. The participants' fixations were then classified into 14 areas of interest. The results showed that pedestrians often looked at the back (20.0%), side (7.5%), and front (4.2%) of parked cars, and at approaching cars (8.8%). Much attention was also paid to the ground (20.1%). The wheels of cars (6.8%) and the driver in approaching cars (3.2%) received attention as well. In conclusion, this study showed that eye movements are largely functional in the sense that they appear to assist in safe navigation through the parking garage. Pedestrians look at a variety of sides and features of the car, suggesting that displays on future automated cars should be omnidirectionally visible. Practitioner summary: This study measured where pedestrians look when walking through a parking garage. It was found that the back, side, and wheels of cars attract considerable attention. This knowledge may be important for the development of automated cars that feature so-called external human-machine interfaces (eHMIs).

eHMI: Review and Guidelines for Deployment on Autonomous Vehicles

Human-machine interaction is an active area of research due to the rapid development of autonomous systems and the need for communication. This review provides further insight into the specific issue of the information flow between pedestrians and automated vehicles by evaluating recent advances in external human-machine interfaces (eHMI), which enable the transmission of state and intent information from the vehicle to the rest of the traffic participants. Recent developments will be explored and studies analyzing their effectiveness based on pedestrian feedback data will be presented and contextualized. As a result, we aim to draw a broad perspective on the current status and recent techniques for eHMI and some guidelines that will encourage future research and development of these systems.

Evaluation of Multimodal External Human–Machine Interface for Driverless Vehicles in Virtual Reality

With the development and promotion of driverless technology, researchers are focusing on designing varied types of external interfaces to induce trust in road users towards this new technology. In this paper, we investigated the effectiveness of a multimodal external human–machine interface (eHMI) for driverless vehicles in virtual environment, focusing on a two-way road scenario. Three phases of identifying, decelerating, and parking were taken into account in the driverless vehicles to pedestrian interaction process. Twelve eHMIs are proposed, which consist of three visual features (smile, arrow and none), three audible features (human voice, warning sound and none) and two physical features (yielding and not yielding). We conducted a study to gain a more efficient and safer eHMI for driverless vehicles when they interact with pedestrians. Based on study outcomes, in the case of yielding, the interaction efficiency and pedestrian safety in multimodal eHMI design was satisfactory compared to the single-modal system. The visual modality in the eHMI of driverless vehicles has the greatest impact on pedestrian safety. In addition, the “arrow” was more intuitive to identify than the “smile” in terms of visual modality.

Communicating the intention of an automated vehicle to pedestrians: The contributions of eHMI and vehicle behavior

External Human-Machine Interfaces (eHMIs) are expected to bridge the communication gap between an automated vehicle (AV) and pedestrians to replace the missing driver-pedestrian interaction. However, the relative impact of movement-based implicit communication and explicit communication with the aid of eHMIs on pedestrians has not been studied and empirically evaluated. In this study, we pit messages from an eHMI against different driving behaviors of an AV that yields to a pedestrian to understand whether pedestrians tend to pay more attention to the motion dynamics of the car or the eHMI in making road-crossing decisions. Our contributions are twofold: we investigate (1) whether the presence of eHMIs has any objective effect on pedestrians’ understanding of the vehicle’s intent, and (2) how the movement dynamics of the vehicle affect the perception of the vehicle intent and interact with the impact of an eHMI. Results show that (1) eHMIs help in convincing pedestrians of the vehicle’s yielding intention, particularly when the speed of the vehicle is slow enough to not be an obvious threat, but still fast enough to raise a doubt about a vehicle’s stopping intention, and (2) pedestrians do not blindly trust the eHMI: when the eHMI message and the vehicle’s movement pattern contradict, pedestrians fall back to movement-based cues. Our results imply that when explicit communication (eHMI) and implicit communication (motion-dynamics and kinematics) are in alignment and work in tandem, communication of the AV’s yielding intention can be facilitated most effectively. This insight can be useful in designing the optimal interaction between AVs and pedestrians from a user-centered design perspective when driver-centric communication is not available.

Expert Drivers' Prospective Thinking-Aloud to Enhance Automated Driving Technologies - Investigating Uncertainty and Anticipation in Traffic

Current automated driving technology cannot cope in numerous conditions that are basic daily driving situations for human drivers. Previous studies show that profound understanding of human drivers' capability to interpret and anticipate traffic situations is required in order to provide similar capacities for automated driving technologies. There is currently not enough a priori understanding of these anticipatory capacities for safe driving applicable to any given driving situation. To enable the development of safer, more economical, and more comfortable automated driving experience, expert drivers' anticipations and related uncertainties were studied on public roads. First, driving instructors' expertise in anticipating traffic situations was validated with a hazard prediction test. Then, selected driving instructors drove in real traffic while thinking aloud anticipations of unfolding events. The results indicate sources of uncertainty and related adaptive and social behaviors in specific traffic situations and environments. In addition, the applicability of these anticipatory capabilities to current automated driving technology is discussed. The presented method and results can be utilized to enhance automated driving technologies by indicating their potential limitations and may enable improved situation awareness for automated vehicles. Furthermore, the produced data can be utilized for recognizing such upcoming situations, in which the human should take over the vehicle, to enable timely take-over requests.