We present a new approach to automated scenario-based testing of the safety of autonomous vehicles, especially those using advanced artificial intelligence-based components, spanning both simulation-based evaluation as well as testing in the real world. Our approach is based on formal methods, combining formal specification of scenarios and safety properties, algorithmic test case generation using formal simulation, test case selection for track testing, executing test cases on the track, and analyzing the resulting data. Experiments with a real autonomous vehicle at an industrial testing facility support our hypotheses that (i) formal simulation can be effective at identifying test cases to run on the track, and (ii) the gap between simulated and real worlds can be systematically evaluated and bridged.

The Challenges and Complexities

Highly automated functions will take longer than expected to deploy. 2018

Why - Technology, cost, regulation, infrastructure

更多

But the biggest barrier for deployment of high level autonomy is SAFETY

The verification of autonomous functions is harder than development of the technology ” Automotive executive quote 2018

The automotive industry s traditional tools and methodologies can not handle the testing complexity needed to ensure safety

收起

AV verification is somewhat similar to chip verification.

Complex Systems in a Complex World。

The complexity of this technology (ADAS and Autonomous Vehicles) and scenarios in which it must operate vastly exceeds anything previously attempted in this industry.

The verification and validation of an autonomous function is harder than its implementation.

Key Messages

● AV/ADS Safety needs to be quantifiable – usage of miles and disengagement is insufficient

● AV/ADS Safety can be measured and quantified

更多

● Coverage Driven Verification is a proven method to measure and quantify maturity of complex h/w-s/w systems

● Coverage metrics can and should be used to quantify AV/ADS safety

收起

M-SDL is a small, domain-specific language designed for describing scenarios where actors (sometimes called agents), such as cars and pedestrians, move through an environment. These scenarios have parameters that let you control and constrain the actors, the movements and the environment.