(How) Can Safety of Automated Driving be Validated? | Prof. H. Winner | Virtual Vehicle Symposium Graz | May 24, 2016
(How) Can Safety of Automated Driving be Validated?
Prof. Dr. rer. nat. Hermann Winner Dipl.-Ing. Walther Wachenfeld Philipp Junietz, M.Sc.
(How) Can Safety of Automated Driving be Validated? | Prof. H. Winner | Virtual Vehicle Symposium Graz | May 24, 2016
2
Considered Levels of Automated Driving
Levels ≥3: Highly Automated Driving (ref. BASt, VDA level 3)
Limited Self-Driving Automation (ref. NHTSA level 3)
Conditional Automation (ref. SAE level 3)
High Automation (ref. SAE level 4)
Fully Automated Driving (ref. BASt, VDA level 4)
Full Self-Driving Automation (ref. NHTSA level 4)
Full Automation (ref. SAE level 5)
Driverless Vehicle (ref. VDA level 5)
For all: No responsibility of human drivers during operation of automation
Sources: bast [1], VDA [2], SAE [3], NHTSA [4]
(How) Can Safety of Automated Driving be Validated? | Prof. H. Winner | Virtual Vehicle Symposium Graz | May 24, 2016
3
Validation Challenge of Automated Driving
Challenge: Validation of promised safety level above the level of driving by humans: Evidence is needed that risk does not exceed today reference.
But what is the safety reference for validation?
(How) Can Safety of Automated Driving be Validated? | Prof. H. Winner | Virtual Vehicle Symposium Graz | May 24, 2016
4
Safety References: Mortality
Abstract reference figures of mortality:
Criterion Yearly mortality Hourly mortality
Minimum Endogenous Mortality (MEM, EN 50126) 2·10-4/a 2.3·10-8/h
MEM adapted to Germany 2012/2014 (destatis) 0.7·10-4/a 0.8·10-8/h
Mean mortality (D, EU), (destatis) 10-2/a 1.1·10-6/h
Life expectancy (destatis) 1/90a 1.3·10-6/h
(How) Can Safety of Automated Driving be Validated? | Prof. H. Winner | Virtual Vehicle Symposium Graz | May 24, 2016
5
Safety References: Today Road Traffic
Accidents per distance:
Numbers for Germany 2014 Total Autobahn
Distance travelled 726·109 km 225·109 km Total number of accidents 2.42·106 0.15·106 With personal injury 291·103 18.4·103
Distance between two accidents:
All accidents 0.34·106 km 1.67·106 km Involving personal injury 2.5·106 km 12·106 km Involving serious casualties >11·106 km >40·106 km Involving fatalities >200·106 km 660·106 km
Data from: [Statistisches Bundesamt, German Federal Statistical Office, 2014])
(How) Can Safety of Automated Driving be Validated? | Prof. H. Winner | Virtual Vehicle Symposium Graz | May 24, 2016
6
Excursion: Risk Figures for Human Drivers
1 German person drives 55 years each year 14,000 km = 770,000 km/lifetime ≈ 15,000 h (at average speed 50 km/h) The average driver is involved every 340,000 km into a reported accident,
and self caused by 60%.
The average driver is involved every 210 mio. km into an accident with fatalities.
In average 1.4 reported accidents are caused by one human in his/her lifetime.
Nearly impossible to differentiate between good and bad drivers, just between lucky and unlucky drivers.
An accident with fatalities will be caused after 450 lifetimes of driving.
(How) Can Safety of Automated Driving be Validated? | Prof. H. Winner | Virtual Vehicle Symposium Graz | May 24, 2016
7
Safety References: Time vs. Distance Normalization
Comparison: Basis: Mean velocity: 50 km/h (general)/100 km/h (Autobahn),
(total number of fatalities)/(accidents with fatalities) ≈ 1.3
Reference distance between two accidents with fatalities
Criterion Total (vav = 50 km/h)
Autobahn (vav = 100 km/h)
Minimum Endogenous Mortality (MEM, EN 50126) 2.85·109 km 5.7·109 km
MEM adapted to Germany 2012/2014 (destatis) 8.1·109 km 16.3·109 km
Mean mortality: ≈ 10-2/a (D, EU), (destatis) 57·106 km 114·106 km
Life expectancy: ≈ 90a, (destatis) 51·106 km 103·106 km Road traffic reality ≈ 260·106 km 660·106 km
(How) Can Safety of Automated Driving be Validated? | Prof. H. Winner | Virtual Vehicle Symposium Graz | May 24, 2016
8
Safety References: Alternative Mobility
Comparison: Passenger distance per fatalities (in passenger-kilometers)
Mobility type
Motorcycles 30·106 pkm
Passenger Cars 500·106 pkm
Public Transport 6·109 pkm
Aviation 300 ·109 pkm
Data from: Verkehr in Zahlen 2015
(How) Can Safety of Automated Driving be Validated? | Prof. H. Winner | Virtual Vehicle Symposium Graz | May 24, 2016
9
Safety References (Conclusion)
Reference variants: Possible safety references are within a wide bandwidth (several orders of
magnitude), much above today road safety as well as much below.
A progress in safety by automation has to be measured in comparison with today risk as reference.
At least two relevant categories have to be addressed as reference:
accidents with damage to persons and specifically
accidents with fatalities
Reference risk figures are far from today testing horizons by real driving tests, e.g. for Autobahn in Germany 2014
Distance between accidents with damage to persons 12·106 km
… with fatalities 660·106 km Data based on: Statistisches Bundesamt, German Federal Statistical Office, 2015, [5]
(How) Can Safety of Automated Driving be Validated? | Prof. H. Winner | Virtual Vehicle Symposium Graz | May 24, 2016
10
Statistical Considerations
Poisson Distribution (independent random process) for the probability, that k events occur in case of an expected value of λ:
λ = ratio between observed test kilometers and system performance
The system performance describes the expected travel distance between two events
perf
test
ss
=λ
Ref.: Wachenfeld, W., Winner, H. [6], Winner, H. [7]
(How) Can Safety of Automated Driving be Validated? | Prof. H. Winner | Virtual Vehicle Symposium Graz | May 24, 2016
11
Conclusion on statistics
Example: AD system might be twice as safe as human driver
If AD system drives about 10x the reference distance of human, probably 5 accidents would occur (instead of the mean of 10 from human).
So, with that result it is confident to about 95% that the system is not worse than human reference.
So, typically 10x the reference distance has to be expected for validation.
Validation Targets Depending on class of injury and looking for “self-caused” accidents only
more than 200·106 km (all injuries) or 10·109 km (fatalities) have to be expected for a first serious figure!
Demonstrating safety of automated driving in advance of introduction is nearly impossible => Approval Trap
(How) Can Safety of Automated Driving be Validated? | Prof. H. Winner | Virtual Vehicle Symposium Graz | May 24, 2016
12
Supplementing information
Fundamental knowledge about ADAS, components and systems and development and testing methodology
What does engineering know up to partial automation?
Technical, Legal and Social
Aspects
A glance into the future
What does research know about autonomous driving?
Handbook of ADAS Autonomous Driving
(How) Can Safety of Automated Driving be Validated? | Prof. H. Winner | Virtual Vehicle Symposium Graz | May 24, 2016
13
STOP!!!!!
For today’s vehicles (and more extreme for aviation) there is no requirement for such high testing distance, why here? What is the fundamental difference?
(How) Can Safety of Automated Driving be Validated? | Prof. H. Winner | Virtual Vehicle Symposium Graz | May 24, 2016
14
Differences between conventional and automated vehicles
Vehicle
Longitudin. and
Lateraldyn.
Driver
Navigation
Guidance/ Conducting
Stabilization
Selected route Time schedule
Desired speed and trajectory
Steering Accelerating
Vehicle motion
Environment
Road network
Traffic situation
Road surface
Alternative routes
Range of safe motion states Actual trajectory and speed
Transport mission
according to Rasmussen [8] and Donges [9]
Knowledge-based Behavior
Rule-based Behavior
Skill-based Behavior
Sensory Input
Driving robot and vehicle
Current validation of vehicle doesn‘t cover the yellow area
(How) Can Safety of Automated Driving be Validated? | Prof. H. Winner | Virtual Vehicle Symposium Graz | May 24, 2016
15
What do we know about Driving Safety Performance?
Statistics and Accident Research Reports on frequency of accidents and their causes
Figures about time gaps and exceeding speeds of some roads
Driver modeling Qualitative models for information processing and driving tasks
(Rasmussen, Donges, …) are able to explain the observed behavior.
Quantitative models for simple scenarios (car following, lane change, intersection crossing) are able to explain and predict traffic flow figures, but not accidents frequency and severity.
Human reliability models (Reichart, …) interpret the observed accidents frequency.
(How) Can Safety of Automated Driving be Validated? | Prof. H. Winner | Virtual Vehicle Symposium Graz | May 24, 2016
16
Simple Probabilistic Accident Model
Image: https://en.wikipedia.org/wiki/ Swiss_cheese_model#CITEREFReason1990
surroundingE
pavementE
trafficdriver
egodriverCheese model idea from [10]
Swiss Cheese Model (adapted to human drivers)
, , , , /
, ,
; ( , )
( , )accidents hd crit hd transition hd crit hd ego traffic road
transition hd ego hd traffic
n n n f driver Ef driver driver
ρ
ρ
= ⋅ =
= n = frequency ρ = transition probability E = exposure of circumstances for potential hazards
(How) Can Safety of Automated Driving be Validated? | Prof. H. Winner | Virtual Vehicle Symposium Graz | May 24, 2016
17
Knowledge about Driving Task and respective Safety
Lacks: Serious figure of the accident avoidance capability of human drivers
Frequency and type of non-standard situations (both self caused or innocently exposed)
Performance of human drivers in non-standard situations
Dark matter problem: We only know standard scenarios and the reported fail scenarios
(accidents), but do not know the probability for transition from accident free driving to real accident occurrence.
Avoiding the known human accident causes are not sufficient:
1. The accidents avoidance capability of humans is not recorded.
2. No quantitative figure about types of critical scenarios and their frequency where humans avoid accidents.
(How) Can Safety of Automated Driving be Validated? | Prof. H. Winner | Virtual Vehicle Symposium Graz | May 24, 2016
18
Dark Matter Problem
Uncritical scenarios (very low potential for accidents)
Critical scenarios (potential for accident)
True accident scenarios
(How) Can Safety of Automated Driving be Validated? | Prof. H. Winner | Virtual Vehicle Symposium Graz | May 24, 2016
19
Swiss Cheese Model (adapted to automated driving)
Accident Model for Automated Vehicles Automation Risks
, , , , , , , / /
, , / /
; ( , )
( , )accidents new crit ad new transition ad new crit ad old new ego traffic road
transition ad old new old new ego partner
n n n f robot Ef robot driver
ρ
ρ
= ⋅ =
=
egorobot
trafficdriver
, , , , ,
, , , , , ,
accidents ad accidents ad old accidents ad new
accidents ad old crit ad old transition ad old
n n nn n ρ
= +
= ⋅
(How) Can Safety of Automated Driving be Validated? | Prof. H. Winner | Virtual Vehicle Symposium Graz | May 24, 2016
20
Dark Matter Problem
Uncritical scenarios (very low potential for accidents)
Critical scenarios (potential for accident, old type)
True accident scenarios (old type)
Automation risk exposure (new critical scenarios)
Automation accidents (new type)
(How) Can Safety of Automated Driving be Validated? | Prof. H. Winner | Virtual Vehicle Symposium Graz | May 24, 2016
21
Knowledge Lack
For prediction of safety of automated driving we need: Valid quantitative number of critical scenarios (remaining and new critical
scenarios) and their specific characteristics
Valid models for capability of AD to control critical situations in a safe manner.
All figures have to be compared with the reference risk numbers of each relevant class.
With respect to the Swiss Cheese Model:
We have to model each slice in order to predict the risk of AD with high validity.
(How) Can Safety of Automated Driving be Validated? | Prof. H. Winner | Virtual Vehicle Symposium Graz | May 24, 2016
22
First step of Risk Minimizing: Preventing of known human errors
Reduction of ncrit,ad,old by adequate careful driving behavior Today more than 90% accidents are caused by human mistakes.
Typical causes:
Inattentiveness, drowsiness
Exceeding speed, too short distances (time gaps)
Avoidable by design of vehicle guidance controller
For standard scenarios simply testable
Possible results of validation: Factors of improvement for these categories (e.g. 1000x less frequent time gaps below 0.5 s)
Most of human caused accident types will be removed from accident statistics by automation.
But, approach addresses only the “bright” matter of safety.
(How) Can Safety of Automated Driving be Validated? | Prof. H. Winner | Virtual Vehicle Symposium Graz | May 24, 2016
23
Second step of Risk Minimizing: Limitation of Exposure in Hazardous Circumstances
Limitation of use case complexity for reduction of ncrit,ad,old Clear traffic rules
Defined or well known scenery
Motorways or similar
Defined areas
Speed limitation
Requirement for just a small set of behavioral strategies for vehicle guidance.
This approach will reduce frequency of critical scenarios, but there is no measure about the transition probability.
(How) Can Safety of Automated Driving be Validated? | Prof. H. Winner | Virtual Vehicle Symposium Graz | May 24, 2016
24
First conclusion
The obvious safety gain: The functional design of automated driving promises higher safety by
reduction of frequency of known critical situations.
Still lacking: Capability of AD to avoid accidents in the remaining critical situations
Frequency of new critical situations generated by automated driving and the capability to control them safely.
Validation of automated driving has to cover both and has to gain all necessary knowledge prerequisites.
(How) Can Safety of Automated Driving be Validated? | Prof. H. Winner | Virtual Vehicle Symposium Graz | May 24, 2016
25
Prerequisites for successful validation
We should know about … the representative worst case test cases,
the metrics for identification of critical situations,
the environmental influence on perception,
how the behavior can be tested as robust and safe,
whether the simulation models for MiL, SiL, HiL, ViL are valid and how to validate,
how representative the simulation has to be for approval purpose.
How can we gain that missing knowledge?
(How) Can Safety of Automated Driving be Validated? | Prof. H. Winner | Virtual Vehicle Symposium Graz | May 24, 2016
26
Evolutionary Approach to Gain Testing Knowledge based on Functional Evolution
Functional evolution Predecessor function gains testing knowledge for successor function:
Data bases of situations, remarkable road parts, of sensor raw signals
Test cases (for real and virtual tests)
Virtual Assessment of Automation in Field Operation (VAAFO) [11]
More realistic simulation models
Statistical data for risk assessment
How does the functional evolution look like? It depends on the Use Case of Autonomous Driving [12]
(How) Can Safety of Automated Driving be Validated? | Prof. H. Winner | Virtual Vehicle Symposium Graz | May 24, 2016
27
From ADAS to Autonomous Driving
Risk Speed
The Evolution Triangle towards Autonomous Driving
ACC: Adaptive Cruise Control LKS: Lane Keeping Support L²A: Longitudinal & Lateral Assist. FSR-ACC: Full-Speed-Range-ACC
AVP: Autonomous Valet-Parking AP: Automated Parking PSA: Park Steering Assist
Em-A: Emergency Assist CA-E: Collision Avoidance by Evading CA-B: Collision Avoidance by Braking CM-B: Collision Mitigation by Braking
(Source [Winner, H.: Quo vadis, FAS? In: Winner, H., Hakuli, S., Lotz, F., Singer, C. (eds.) Handbuch Fahrerassistenzsysteme, 3rd edn. Vieweg-Teubner-Verlag (2015)]
Ref.: Winner, H. [6]
(How) Can Safety of Automated Driving be Validated? | Prof. H. Winner | Virtual Vehicle Symposium Graz | May 24, 2016
28
Evolutionary Approach to Gain Testing Knowledge based on Use Case Extension
Use case extension Functional start with full driverless automation
Limitation of operation area
From few routes to new driving area (incremental extension of potential risk).
At first, there is no comparable benchmark for the risk.
Field experience will make the autonomous driving more and more mature.
The operation is supervised (e.g. by a provider) and can be controlled (including shut down).
(How) Can Safety of Automated Driving be Validated? | Prof. H. Winner | Virtual Vehicle Symposium Graz | May 24, 2016
29
Where and under what conditions is the automation available? Not only the level of automation and
the use case offer evolutionary paths
Also an evolution in availability is reasonable
Different approaches exist (most OEM vs. Google)
Availability vs. Degree of Automation
Availability
Something Everything Level of Automation
OEM
Every- where
Some- where
(How) Can Safety of Automated Driving be Validated? | Prof. H. Winner | Virtual Vehicle Symposium Graz | May 24, 2016
30
Virtualization of Tests for Validation
Enabling of efficient test tools Virtual testing has potential to accelerate approval, but with virtualization
always simplification takes place Validation of model?
Different ways of combining virtuality and reality exist and can be used
Sensor simulation for SiL and sensor stimulation for HiL/ViL is needed
Ref.: Wachenfeld, W., Winner, H. [7]
(How) Can Safety of Automated Driving be Validated? | Prof. H. Winner | Virtual Vehicle Symposium Graz | May 24, 2016
31
Virtual Validation
Principle: Running tests by many paralleled systems (ViL, HiL, SiL) equivalent to
billions of km (109 km ≈ 20·106 hours = 2,280 years real time)
Validation of simulation models
component models (e.g. sensors) by component testing systems
environment conditions from test drives recording
vehicle dynamics by test maneuvers
Situation creation by permutation or stochastics (Monte Carlo approach).
Big challenge for modeling:
Validation of behavior models of other traffic participants.
Valid environment and sensor models
(How) Can Safety of Automated Driving be Validated? | Prof. H. Winner | Virtual Vehicle Symposium Graz | May 24, 2016
32
Virtual Validation
Problem due to the manifold of situational combinations Many sceneries and circumstances
Many constellations of traffic partners
Various environment conditions (e.g. rain, fog, pavement condition, light brightness and direction, …)
Permutation of all influence parameters will overload any computer cluster Evaluation of safety in simulation just by counting occurred virtual
accidents needs the full combination of all variables
How can we come to a feasible virtual approval?
(How) Can Safety of Automated Driving be Validated? | Prof. H. Winner | Virtual Vehicle Symposium Graz | May 24, 2016
33
Approval by Worst Case Testing
Systematic reduction of test cases Identify critical situations only
Assumption: If all critical situations could be handled, every less critical situation can be handled, too.
Need of a validated metric for criticality in situations
Even with only checking critical situations a huge variety of these critical situation must be tested (e.g. various weather conditions, brightness,…).
Challenge of reduction and virtualization
Real world Relevant world Artificial/virtual
tests Real driving
(How) Can Safety of Automated Driving be Validated? | Prof. H. Winner | Virtual Vehicle Symposium Graz | May 24, 2016
34
Decomposition Approach for Test Case Reduction
Decomposing of the scenario into several layers Clustering of test cases (experimental as well as in simulation) related to
the layer
Situation Umgebung (dyn.) Situation Umgebung (dyn.) Situation subject vehicle
1: Information access
2: Information reception
3: Information processing
4: Decision (behavioral)
5: Action
Situation environment (dyn.) Situation statical
Object under test Test environment
1: Information access
2: Information reception
3: Information processing
4: Decision (behavioral)
5: Action
Layer after Graab et al. [13]
(How) Can Safety of Automated Driving be Validated? | Prof. H. Winner | Virtual Vehicle Symposium Graz | May 24, 2016
35
Decomposition Approach for Test Case Reduction
Decomposition may lead to reduction of test effort by Modularisation of tests and no or less re-test in case of unchanged
functional modules
Reduction of redundant tests within the same layer
Pre-requisite: Pass/fail criteria (metrics) depending on decomposing layer
Open promise: Metrics for comparison on safety performance human vs. machine
(How) Can Safety of Automated Driving be Validated? | Prof. H. Winner | Virtual Vehicle Symposium Graz | May 24, 2016
36
Virtual Approval
But: How can we be convinced that the test case set is sufficient? Check whether all situations of reconstructed accidents are part of the
test set does help just a little bit, because driving robot behavior has intentionally to differ from human driving.
And nobody knows how the behavior of human would change when they were confronted with autonomous driving.
Conclusion on virtual approval Simulation will be a very important part of the approval process, but it will
not help to overcome the approval trap due to the lack of validity for modeling and test case set in the beginning.
A virtual approval should be future objective for methodological and economical reasons.
So, the other test methods have to improve the model validity and the test catalogue.
(How) Can Safety of Automated Driving be Validated? | Prof. H. Winner | Virtual Vehicle Symposium Graz | May 24, 2016
37
The new Role of Driving Tests
Driving tests for model validation Comparison of behavior between simulated and real situations
Driving tests for environment representation validation and behavior pattern of other traffic participants
Driving tests for test catalogue Situational statistics for importance ranking of test cases
Making the test case catalogue complete (for both virtual and real tests)
Only real drives are able to assess the validity of model and test catalogue.
(How) Can Safety of Automated Driving be Validated? | Prof. H. Winner | Virtual Vehicle Symposium Graz | May 24, 2016
38
The new Role of Driving Tests
By the rate of “surprises” of new driving tests a careful extrapolation how valid the models are and how complete the current test catalogue might be possible.
Surprises per distance in 1/Mkm Number of counted new events per distance
Approx. trend line Extrapolation
1 10 100 1000
1000
100
10
1 Driven test kilometers 103 ⋅
(How) Can Safety of Automated Driving be Validated? | Prof. H. Winner | Virtual Vehicle Symposium Graz | May 24, 2016
39
Field Data Assessment
How can knowledge be extracted from driving experience: 1. Recording of data, trigger by (test) driver, evaluation of triggered time
slices in Lab.
2. Transmission of all relevant situations within real-time to Lab. Labeling and assessment by automatic data analytics.
3. Virtual Assessment of Automation in Field Operation (VAAFO), triggering and extraction of data set onboard, records could be transmitted w/o real-time need.
All methods can be used with active or emulated automation or predecessor automation.
VAAFO is suitable for implementing in series fleet to assess potential upgrade functions.
(How) Can Safety of Automated Driving be Validated? | Prof. H. Winner | Virtual Vehicle Symposium Graz | May 24, 2016
40
VAAFO Virtual Assessment of Automation in Field Operation
Combines the advantages of both methods (SiL and field test): Reality based test case generation
Test tool enabling tests without higher risk than usual real world driving
The limitations of the concept: Validity of the virtual world
Initialize (e.g. every 0,5 s) the virtual world on base of real world measurement.
Short (e.g. 2 s) simulation of the virtual world
Test distances don’t decrease
Start early in the development phase
Implement the tool in fleets field observation (data protection should be discussed)
(How) Can Safety of Automated Driving be Validated? | Prof. H. Winner | Virtual Vehicle Symposium Graz | May 24, 2016
41
VAAFO Virtual Assessment of Automation in Field Operation
The strength of the concept: Using the tool in for example 10,000 vehicles with average milage of
14.000 km a year. [Numbers from DAT for 2013]
This leads to 140 million km driving in real traffic.
Concept can be applied on different steps of development and evolution
1st testing on road to
nth software-update
An Autobahn-Chauffeur equipped with sensors and processing power is capable to record data from driving in cities or on country roads.
(How) Can Safety of Automated Driving be Validated? | Prof. H. Winner | Virtual Vehicle Symposium Graz | May 24, 2016
42
European Project:
European Initiative to Enable Validation for Highly Automated Safe and Secure Systems 05/2016 - 04/2019
Current Research Projects to overcome the Approval Trap of AD
National Project:
Project for Establishing Generally Accepted quality criteria, tools and methods as well as Scenarios And Situations for approval of highly automated driving functions 01/2016 - 06/2019
(How) Can Safety of Automated Driving be Validated? | Prof. H. Winner | Virtual Vehicle Symposium Graz | May 24, 2016
43
Remaining Approval Trap Problem
Prognosis: For the first Autonomous Driving (Level ≥ 3) application the pre-requisites
for an approval for general and unlimited introduction will not be given (like a chicken & egg-problem).
The Approval Trap might be disarmed to some extend by the methodological work before, but not sufficient.
What is the alternative?
(How) Can Safety of Automated Driving be Validated? | Prof. H. Winner | Virtual Vehicle Symposium Graz | May 24, 2016
44
Strategy: Risk limited Introduction: How statistics may help to introduce automated driving Risk limitation based on statistical figures (ref. [14]) Testing travel distance with the recorded mean value of accidents gives
an estimate for the expected accident value (depending on a given error probability one can calculate a best and a worst case factor).
Taking the worst case factor one can calculate the maximum expected risk for a given number of autonomous vehicles in the field.
Whether this worst case risk is below a detection limit in a statistical sense the vehicles can be introduced in order to record additional data helping the release for the next higher number of autonomous vehicles.
The driven travel distance increases the statistical basis and the fleet in traffic can be increased recursively.
(How) Can Safety of Automated Driving be Validated? | Prof. H. Winner | Virtual Vehicle Symposium Graz | May 24, 2016
45
Conclusion (I)
The Approval Trap is still existing. There are different promising approaches Virtual Approval,
Virtual Assessment of Automation in Field Operation (VAAFO),
Decomposition, and
Worst Case Testing
overcoming the trap, but they need prerequisites which are far from today state-of-the-art.
(How) Can Safety of Automated Driving be Validated? | Prof. H. Winner | Virtual Vehicle Symposium Graz | May 24, 2016
46
Conclusion (II)
Functional evolution or use case extension are strategies to get sufficient data. But there are still some doubts whether the quality of the methods for validation will be then sufficient. Test drives will play the key role for risk assessment and for development of all alternative test methodologies. A limiting risk introduction strategy will help to introduce autonomous driving by “tunnelling” the barrier or by skipping the trap for the first systems.
(How) Can Safety of Automated Driving be Validated? | Prof. H. Winner | Virtual Vehicle Symposium Graz | May 24, 2016
47
Back to the Title of the Presentation
Can Safety of Automated Driving be Validated? Yes,
but not in advance of introduction on the level of reference figures coming from human drivers.
How can Safety of Automated Driving be Validated?
1. Data collecting by test driving and supervised risk limited introduction
2. Development and validation of simulation models for virtual tests
3. Certification of safety by combination of mainly virtual tests and recursive validation with real driving data.
(How) Can Safety of Automated Driving be Validated? | Prof. H. Winner | Virtual Vehicle Symposium Graz | May 24, 2016
(1) Gasser, T. M.; Arzt, C.; Ayoubi, M.; Bartels, A.; Bürkle, L.; Eier, J.; Flemisch, F.; Häcker, D.; Hesse, T.; Huber, W.; Lotz, C.; Maurer, M.; Ruth-Schumacher, S.; Schwarz, J.; Vogt, W.: Rechtsfolgen zunehmender Fahrzeugautomatisierung. Gemeinsamer Schlussbericht der BASt-Projektgruppe „Rechtsfolgen zunehmender Fahrzeugautomatisierung“ Dokumentteil 1. Wirtschaftsverlag NW, Bergisch Gladbach, 2012 (Heft F 83)
(2) Verband der Automobilindustrie: From Driver Assistance Systems to Automated Driving, VDA Magazine – Automation, 2015 (3) SAE: Levels of Driving Automation, Information Report J3016, 2014 (4) US National Highway Traffic Safety Administration (NHTSA): Preliminary Statement of Policy Concerning Automated Vehicles,
2013 (5) Statistisches Bundesamt / German Federal Statistical Office, 2014, https://www.destatis.de/DE/Publikationen/
Thematisch/TransportVerkehr/ Verkehrsunfaelle/VerkehrsunfaelleJ2080700147004.pdf?__blob=publicationFile (6) Winner, H.: ADAS, Quo Vadis?, in Winner, H.; Hakuli, S.; Lotz, F.; Singer, C. (eds.): Hand of Driver Assistance Systems, Springer
2016 (7) Wachenfeld, W., Winner, H.: Die Freigabe des autonomen Fahrens. In: Maurer, M., Gerdes, J.C., Lenz, B., Winner, H. (Hrsg.)
Autonomes Fahren, pp. 439-464. Springer Berlin Heidelberg (2015) (8) Rasmussen, J.: Skills, Rules, and Knowledge; Signals, Signs, and Symbols, and Other Distinctions in Human Performance
Models. IEEE Transactions On Systems, Man, and Cybernetics SMC-13(3), 257–266 (1983)] (9) Donges, Edmund: Fahrerverhaltensmodelle. In: Winner, Hakuli, Wolf (eds.) Handbuch Fahrerassistenzsysteme, pp. 15–23 (2011) (10) Reason, James (1990-04-12). "The Contribution of Latent Human Failures to the Breakdown of Complex Systems". Philosophical
Transactions of the Royal Society of London. Series B, Biological Sciences 327 (1241): 475–484. (11) Wachenfeld, W., Winner, H.: Virtual Assessment of Automation in Field Operation – A New Runtime Validation Method, FAS
Workshop in Walting 2015 (12) Wachenfeld, W., Winner, H., Gerdes, C., Lenz, B., Maurer, M., Beiker, S.A., Fraedrich, E., Winkle, T.: Use-Cases des autonomen
Fahrens. In: Maurer, M., Gerdes, J.C., Lenz, B., Winner, H. (eds.) Autonomes Fahren, pp. 9-37. Springer Berlin Heidelberg (2015)])
(13) Graab et al.: Analyse von Verkehrsunfällen hinsichtlich unterschiedlicher Fahrerpopulationen und daraus ableitbarer Ergebnisse für die Entwicklung adaptiver Fahrerassistenzsysteme, 2008
(14) Wachenfeld, W.; Winner, H.: The new role of road testing for the safety validation of automated vehicles. In Horn, M.; Watzenig, D. (eds.): Automated Driving – Safer and more efficient future driving; Springer International Publishing AG (2016)
References