Waymo simulated driving behavior in reconstructed fatal crashes within an autonomous vehicle operating domain

How would autonomous vehicles behave in real-world crash scenarios?

Autonomous Vehicles (AVs) have the potential to revolutionize transportation systems by enhancing traffic safety. Safety testing is undoubtedly a critical step for enabling large-scale deployment of AVs. High-risk scenarios are particularly important as they pose significant challenges and provide valuable insights into the driving capabilities of AVs. This study presents a novel approach to assess the safety of AVs using in-depth crash data, with a particular focus on real-world crash scenarios. First, based on the high-definition video recording of the whole process prior to the crash occurrences, 453 real-world crashes involving 596 passenger cars from China In-depth Mobility Safety Study-Traffic Accident (CIMSS-TA) database were reconstructed. Pertinent static and dynamic elements needed for the construction of the testing scenarios were extracted. Subsequently, 596 testing scenarios were created via each passenger car's perspective within the simulation platform. Following this, each of the crash-involved passenger cars was replaced with Baidu Apollo, a famous automated driving system (ADS), for counterfactual simulation. Lastly, the safety performance of the AV was assessed using the simulation results. A logit model was utilized to identify the fifteen crucial scenario elements that have significant impacts on the test results. The findings demonstrated that the AV could avoid 363 real-world crashes, accounting for approximately 60.91% of the total, and effectively mitigated injuries in the remaining 233 unavoidable scenarios compared to a human driver. Moreover, the AV maintain a smoother speed in most of the scenarios. The common feature of these unavoidable scenarios is that the AV is in a passive state, and the crashes are not caused by the AV violating traffic rules, but rather caused by abnormal behavior exhibited by the human drivers. Additionally, seven specific scenarios have been identified wherein AVs are unable to avoid a crash. These findings demonstrate that, compared to human drivers, AVs can avoid crashes that are difficult for humans to avoid, thereby enhancing traffic safety.

Quantification of safety improvements and human-machine tradeoffs in the transition to automated driving

The assumption of reduced human error-related crashes with increasing levels of automation in pursuing Level 5 automation lacks empirical evidence. As automation levels rise, human error-induced safety hazards are anticipated to decrease, while machine error-induced hazards will increase. However, a quantitative index capturing this tradeoff is absent. Additionally, theoretical modeling of safety improvements during the transition to automated driving remains unexplored, particularly concerning reducing human error-related hazards. These limitations impede the understanding of safety from human and machine perspectives for Automated Vehicle (AV) specialists and manufacturers. This research addresses these gaps by investigating safety performance associations between human and machine factors using the "Human-Machine conflict reduction ratio" (H/M ratio), a novel metric. The study aims to establish safety improvements related to human errors under various automation levels. Sixty participants completed driving tasks on a driving simulator at Levels 0, 4, 3, and 2. Safety performance measures, including conflict frequency and severity, were computed. As a result, Level 4 exhibits the largest decrease (93.3%) compared to manual driving, followed by Level 2 (70.7%) and Level 3 (40.5%). The H/M ratio measures the tradeoff between reducing human and machine error-induced hazards, with Level 2 demonstrating the highest ratio, followed by Levels 4 and 3. Safety performance is evaluated by considering all possible types of human errors at each automation level. Theoretical models from a human factor's perspective are employed to estimate safety improvements at each level. This research contributes to a comprehensive understanding of safety in the "human-machine cooperative driving" phase, offering insights to AV industry practitioners and stakeholders.

Runtime unknown unsafe scenarios identification for SOTIF of autonomous vehicles

Safety is a critical concern for autonomous vehicles (AVs). Current testing approaches face challenges in simultaneously meeting the requirements of being valid, safe, and fast. To address these challenges, the silent testing approach that tests functions or systems in the background without interfering with driving is motivated. Building upon our previous research, this study first extends the method to specifically address the validation of AV perception, utilizing a lane marking detection algorithm (LMDA) as a case study. Second, field experiments were conducted to investigate the method's effectiveness in validating AV systems. For both studies, an architecture for describing the working principle is presented. The efficacy of the method in evaluating the LMDA is demonstrated through the use of adversarial images generated from a dataset. Furthermore, various scenarios involving pedestrians crossing a road under different levels of criticality were constructed to gain practical insights into the method's applicability for AV system validation. The results show that corner cases of the LMDA are successfully identified by the given evaluation metrics. Furthermore, the experiments highlight the benefits of employing multiple virtual instances with different initial states, enabling the expansion of the test space and the discovery of unknown unsafe scenarios, particularly those prone to false-positive objects. The practical implementation and systematic discussion of the method offer a significant contribution to AV safety validation.

Exploratory analysis of evasion actions of powered two-wheeler conflicts at unsignalized intersection

The study investigates the braking and steering evasions of powered two-wheelers (PTWs) during severe conflicts observed at an unsignalized intersection. Traffic conflicts were detected using a surrogate safety indicator called anticipated collision time (ACT). Then the peak-over-threshold approach was used to identify the severe conflicts and the evasive actions. Conflicts between right-turning PTWs and through-moving vehicles, through-moving PTWs crossing through-moving vehicles, and merging/diverging PTWs were analyzed using the minimum ACT (ACTmin), maximum deceleration rate (DRmax), maximum yaw rate (YRmax), and time of evasive action (TEA). The evasive actions were classified into five categories: driver/rider error, no-evasion, braking-only, steering-only, and both braking and steering. Analysis reveals that right-turning PTWs experience higher crash risk (0.7 %) than the other movements. PTW riders primarily employ extreme steering maneuvers (greater than 13 degrees/s) to evade conflicts, whereas braking rates lie in the normal ranges (less than 1.5 m/s2). The time of evasive action varies between 2.04 and 2.44 s, with the right-turning PTW riders responding early. Through-moving riders commit errors while evading severe conflicts and perform fewer evasive actions than right-turning and merging/diverging riders. Right-turning riders perform more steering-only evasions than braking-only, whereas the riders involved in the other two conflicts execute more braking-only evasions. These findings suggest that conflict type influences riders' braking and steering responses. Hence, future applications in advanced driver/rider assistance systems and training programs should consider appropriate evasive action strategies for different conflict types.

Determination of functional scenarios for intersection collisions

INTRODUCTION

The National Highway Traffic Safety Administration (NHTSA) estimated that in 2019, intersection crashes accounted for $179 billion of economic damages and $639 billion in societal damages. Intersection advanced driver assist systems (I-ADASs) and automated driving systems (ADS) are designed and have been actively deployed to avoid or mitigate these intersection crash scenarios. Given the indeterminate parameter space for describing collision scenarios, evaluators, and designers are all challenged with condensing the possible intersection crash configurations into digestible, executable conditions for scenario-based simulation testing. The objective of this study is to identify functional intersection crash configurations for I-ADAS and ADS safety evaluation.

METHODS

Real-world intersection crash characteristics are important considerations for scenario testing as these features can directly correlate to or influence causality, controllability, and potential injury severity. To identify functional intersection crash types, similar crash scenarios were grouped together by identified critical features using an unsupervised decision tree model. A key advantage of this approach was that the implemented cluster crash scenarios would be understandable and interpretable by users. Unsupervised decision trees work by generating uniformly distributed synthetic data with features from real data and classifying all the data as real or synthetic. Long, non-diverging branches were manually pruned to reduce overfitting and improve model performance. Feature importance values were computed based on how effective a given variable grouped the crashes together.

DATA SOURCES

This analysis selected intersection cases that only involved two vehicles from the Crash Investigation Sampling System (CISS) spanning 2017 to 2020. Crash features such as road geometry, intersection signal, and vehicle configuration were important to consider for scenario generation. CISS contained the traffic device, device functionality, vehicle intended pre-event movement, road alignment, road profile, trafficway flow, number of lanes, and crash type for each crash case. Intersection geometry, intersecting road angle, each vehicles' legal moves, and the presence of a two-way-left-turn-lane (TWLTL), channelized roads, bike lanes, crosswalks, street parking, slip lanes, and visual obstructions were manually recorded from the scene diagram.

RESULTS

The tree identified 44 functional intersection crash configurations after pruning. These clusters have five main sections: Straight-crossing path (SCP) crashes at 4-legged intersections, Left-Turn-Across-Path/Opposite Direction (LTAP/OD) crashes at 4-legged intersections, other crash types at 4-legged intersections, roundabout and multileg intersections, and 3-legged intersection crashes. The features that best split the data were TWLTL, lane travel direction violation, and traffic control device functionality. The largest cluster was SCP crashes at 4-legged, undivided intersections where the traffic control device was working and both vehicles did not violate the direction of their lane of travel. This cluster was adjacent to 32 vehicles in similar SCP crashes except a vehicle performed an unexpected maneuver based on their lane position.

CONCLUSION

These 44 identified crash configurations could be useful in bolstering the robustness of I-ADAS and ADS intersection scenario testing as they are a compact representation of all the police reported intersection crashes where a vehicle was towed. Future studies could generate logical scenarios with distributions of initial conditions and behaviors from these clusters that could be used to evaluate an I-ADAS or ADS.

Passenger and heavy vehicle collisions with pedestrians: Assessment of injury mechanisms and risk

OBJECTIVE

Automated Driving System (ADS) fleets are currently being deployed in several dense-urban operational design domains within the United States. In these dense-urban areas, pedestrians have historically comprised a significant portion, and sometimes the majority, of injury and fatal collisions. An expanded understanding of the injury risk in collision events involving pedestrians and human-driven vehicles can inform continued ADS development and safety benefits evaluation. There is no current systematic investigation of United States pedestrian collisions, so this study used reconstruction data from the German In-Depth Accident Study (GIDAS) to develop mechanistic injury risk models for pedestrians involved in collisions with vehicles.

DATA SOURCE

The study queried the GIDAS database for cases from 1999 to 2021 involving passenger vehicle or heavy vehicle collisions with pedestrians.

METHODS

We describe the injury patterns and frequencies for passenger vehicle-to-pedestrian and heavy vehicle-to-pedestrian collisions, where heavy vehicles included heavy trucks and buses. Injury risk functions were developed at the AIS2+, 3+, 4+ and 5+ levels for pedestrians involved in frontal collisions with passenger vehicles and separately for frontal collisions with heavy vehicles. Model predictors included mechanistic factors of collision speed, pedestrian age, sex, pedestrian height relative to vehicle bumper height, and vehicle acceleration before impact. Children (≤17 y.o.) and elderly (≥65 y.o.) pedestrians were included. We further conducted weighted and imputed analyses to understand the effects of missing data elements and of weighting towards the overall population of German pedestrian crashes.

RESULTS

We identified 3,112 pedestrians involved in collisions with passenger vehicles, where 2,524 of those collisions were frontal vehicle strikes. Furthermore, we determined 154 pedestrians involved in collisions with heavy vehicles, where 87 of those identified collisions were frontal vehicle strikes. Children were found to be at higher risk of injury compared to young adults, and the highest risk of serious injuries (AIS 3+) existed for the oldest pedestrians in the dataset. Collisions with heavy vehicles were more likely to produce serious (AIS 3+) injuries at low speeds than collisions with passenger vehicles. Injury mechanisms differed between collisions with passenger vehicles and with heavy vehicles. The initial engagement caused 36% of pedestrians' most-severe injuries in passenger vehicle collisions, compared with 23% in heavy vehicles collisions. Conversely, the vehicle underside caused 6% of the most-severe injuries in passenger vehicle collisions and 20% in heavy vehicles collisions.

SIGNIFICANCE

U.S. pedestrian fatalities have risen 59% since their recent recorded low in 2009. It is imperative that we understand and describe injury risk so that we can target effective strategies for injury and fatality reduction. This study builds on previous analyses by including the most modern vehicles, including children and elderly pedestrians, incorporating additional mechanistic predictors, broadening the scope of included crashes, and using multiple imputation and weighting to better estimate these effects relative to the entire population of German pedestrian collisions. This study is the first to investigate the risk of injury to pedestrians in collisions with heavy vehicles based on field data.

Modeling driver's evasive behavior during safety-critical lane changes: Two-dimensional time-to-collision and deep reinforcement learning

Lane changes are complex driving behaviors and frequently involve safety-critical situations. This study aims to develop a lane-change-related evasive behavior model, which can facilitate the development of safety-aware traffic simulations and predictive collision avoidance systems. Large-scale connected vehicle data from the Safety Pilot Model Deployment (SPMD) program were used for this study. A new surrogate safety measure, two-dimensional time-to-collision (2D-TTC), was proposed to identify the safety-critical situations during lane changes. The validity of 2D-TTC was confirmed by showing a high correlation between the detected conflict risks and the archived crashes. A deep deterministic policy gradient (DDPG) algorithm, which could learn the sequential decision-making process over continuous action spaces, was used to model the evasive behaviors in the identified safety-critical situations. The results showed the superiority of the proposed model in replicating both the longitudinal and lateral evasive behaviors.

Learning naturalistic driving environment with statistical realism

For simulation to be an effective tool for the development and testing of autonomous vehicles, the simulator must be able to produce realistic safety-critical scenarios with distribution-level accuracy. However, due to the high dimensionality of real-world driving environments and the rarity of long-tail safety-critical events, how to achieve statistical realism in simulation is a long-standing problem. In this paper, we develop NeuralNDE, a deep learning-based framework to learn multi-agent interaction behavior from vehicle trajectory data, and propose a conflict critic model and a safety mapping network to refine the generation process of safety-critical events, following real-world occurring frequencies and patterns. The results show that NeuralNDE can achieve both accurate safety-critical driving statistics (e.g., crash rate/type/severity and near-miss statistics, etc.) and normal driving statistics (e.g., vehicle speed/distance/yielding behavior distributions, etc.), as demonstrated in the simulation of urban driving environments. To the best of our knowledge, this is the first time that a simulation model can reproduce the real-world driving environment with statistical realism, particularly for safety-critical situations.

Can non-crash naturalistic driving data be an alternative to crash data for use in virtual assessment of the safety performance of automated emergency braking systems?

INTRODUCTION

Developers of in-vehicle safety systems need to have data allowing them to identify traffic safety issues and to estimate the benefit of the systems in the region where it is to be used, before they are deployed on-road. Developers typically want in-depth crash data. However, such data are often not available. There is a need to identify and validate complementary data sources that can complement in-depth crash data, such as Naturalistic Driving Data (NDD). However, few crashes are found in such data. This paper investigates how rear-end crashes that are artificially generated from two different sources of non-crash NDD (highD and SHRP2) compare to rear-end in-depth crash data (GIDAS).

METHOD

Crash characteristics and the performance of two conceptual automated emergency braking (AEB) systems were obtained through virtual simulations - simulating the time-series crash data from each data source.

RESULTS

Results show substantial differences in the estimated impact speeds between the artificially generated crashes based on both sources of NDD, and the in-depth crash data; both with and without AEB systems. Scenario types also differed substantially, where the NDD have many fewer scenarios where the following-vehicle is not following the lead vehicle, but instead catches-up at high speed. However, crashes based on NDD near-crashes show similar pre-crash criticality (time-to-collision) to in-depth crash data.

CONCLUSIONS

If crashes based on near-crashes are to be used in the design and assessment of preventive safety systems, it has to be done with great care, and crashes created purely from small amounts of everyday driving NDD are not of much use in such assessment.

PRACTICAL APPLICATIONS

Researchers and developers of in-vehicle safety systems can use the results from this study: (a) when deciding which data to use for virtual safety assessment of such systems, and (b) to understand the limitations of NDD.

Intersection two-vehicle crash scenario specification for automated vehicle safety evaluation using sequence analysis and Bayesian networks

This paper introduces a test scenario specification procedure using crash sequence analysis and Bayesian network modeling. Intersection two-vehicle crash data was obtained from the 2016-2018 National Highway Traffic Safety Administration (NHTSA) Crash Report Sampling System (CRSS) database. Vehicles involved in the crashes are specifically renumbered based on their initial positions and trajectories. Crash sequences are encoded to include detailed pre-crash events and concise collision events. Based on sequence patterns, the crashes are characterized as 55 types. A Bayesian network model is developed to depict the interrelationships among crash sequence types, crash outcomes, human factors, and environmental conditions. Scenarios are specified by querying the Bayesian network's conditional probability table. Distributions of operational design domain (ODD) attributes (e.g., driver behavior, weather, lighting condition, intersection geometry, traffic control device) are specified based on conditions of sequence types. Also, distribution of sequence types is specified on specific crash outcomes or combinations of ODD attributes.

New needs to consider during accident analysis: Implications of autonomous vehicles with collision reconfiguration systems

Autonomous vehicles are equipped with advanced vehicle technology (AVT) that will improve road traffic safety and reduce accidents. However, due to the uncertain behavior of other road users, collisions can never be completely eliminated. Collision reconfiguration systems offer a solution by, for instance, changing where vehicles are hit and how the impact force is directed towards them. Unfortunately, the logic behind the decision-making of collision reconfiguration systems is fundamentally different from that of other AVTs. Fundamentally different feedback might thus be required from accident analyses to ensure the successful design of collision reconfiguration systems. Through simulations, this study explores decision-making strategies of collision reconfiguration systems to ascertain the implications of which feedback is required from accident analyses. Results show that different strategies can be statistically significantly different from each other in the way they affect severity; and that a new source of unobserved heterogeneity could easily be small variations in the algorithms used by collision reconfiguration systems. Based on this, three new needs to consider during accident analysis are put forth: firstly, new safety surrogate measures (SSMs) that consider severity are required; one such SSM is proposed; secondly, to identify new unobserved heterogeneity as a result of collision reconfiguration systems, the trajectories of traffic near-collisions should be recorded, and statistical tools to identify comparable scenarios developed. Thirdly, new collision patterns will make it difficult to analyze the implications of reconfigured collisions, which suggests that collision configurations must be carefully recorded to provide early feedback.

A new standard for accident simulations for self-driving vehicles: Can we use Waymo’s results from accident simulations?

Recent simulations by Scanlon et al. showed seemingly spectacular results for the Waymo self-driving vehicle in simulations of real accident situations. In this paper, it is argued that the selection criteria for accident situations must be modified in accordance with the relevant policy alternatives. While Scanlon et al. compare Waymo with old human-driven vehicles, it is argued here that the relevant policy question is whether we ought to use self-driven vehicles or human-driven vehicles in the future, which means that we need to consider whether other technological solutions, which are available but not broadly used in human-driven vehicles, could result in human-driven vehicles managing to avoid the same accidents. In this article, a proposal for a new standard of selection criteria is made.

Study on the Extraction Method of Sub-Network for Optimal Operation of Connected and Automated Vehicle-Based Mobility Service and Its Implication

There have been enormous efforts to implement automated vehicle-based mobility (AVM) by considering smart infrastructure such as cooperative intelligent transportation system. However, there is lack of consideration on economical approach for an optimal deployment strategy of the AVM service and smart infrastructure. Furthermore, the influence of travel demand in service area has been ignored. We develop a new framework for maximizing the profit of connected and automated vehicle-based mobility (CAV-M) service using cost modeling and metaheuristic optimization algorithm. The proposed framework extracts an optimal sub-network, which is selected by a set of optimal links in the service area, and identifies an optimal construction strategy for the smart infrastructure depending on given operational design domain and travel demand. Based on service network analyses with varying demand patterns and volumes, we observe that the optimal sub-network varies with the combination of trip demand patterns and volumes. It is also found that the benefit of deploying the smart infrastructure is obtainable only when there are sufficient travel demands. Furthermore, the optimal sub-network is always superior to raw network in terms of economical profit, which suggests the proposed framework has great potential to prioritize road links in the target area for the CAV-M service.