validation & verification€¦ · design, validation & verification framework for adas and...

Nico Nagl – Portfolio Development Autonomous Driving

Validation & Verification ADAS-Fahrfunktionen effizient

validieren und verifizieren

Where today meets tomorrow.Nico Nagl – Portfolio Development

Unrestricted © Siemens 2020

Siemens Digital Industries Software

Connectivity Autonomous Vehicles

Shared MobilityVehicle Electrification

Disruptive InnovationKey to sustained business

Engineering the NEXT product not just the best product for the future



Addressing challenges for autonomous driving

vehicle development

FROM ADAS TO AUTONOMOUS DRIVING

“+25% CAGR (through 2030) for Sensors”Roland Berger , on “Autonomous Driving”, 2014

…

“14.2 billion kilometers of testing is needed”Akio Toyoda, CEO of Toyota

Paris Auto Show 2016

“Design validation will be a major – if not the

largest – cost component”Roland Berger

“Autonomous Driving” 2014



Engineering implications of the AV development challenge

Increasing software and hardware complexity

Massive validation and verification cycles

Growing number and variety of sensors

Complex interactions between systems

Rethink the vehicle development processes

While balancing safety, comfort and efficiency performances

Growing number and variety of sensors

Massive validation and verification cycles

Reconciling agility with better traceability

Increased hardware and software complexity



ADAS/AV systems virtual V&V

Automotive industry needs

Virtual validation(MiL, SiL)

Semi virtual validation(HiL, DiL, VehiL)

Real validation vehicle testing

(proving ground, public road)

~106 test cases

~103 test cases

~102 test cases

~102 test cases

~102 test cases

~102 test casesSAE level 1 to 5

SAE level 1 to 5

SAE level 1 to 5

Need for efficient and automated

simulation orchestration

FAILING IN SIMULATION DOES NOT KILL PEOPLE



Do Things Right

-

Doing the Right Things

Efficiency and Effectiveness



ADAS/AV systems virtual V&VAutomotive industry needs

• Take not ideal world into account

• Need for realistic and non-ideal

environments

• Need for more vehicle physics than before

• Simulation of appropriate scenarios is

essential



Design, Validation & Verification framework for ADAS and AV

MiL / SiL / Cluster

HiL / DiL / ViL

Proving ground /

field test

V&V environments

Digital Twin

“World”

Digital Twin

“Vehicle”

Design adaptations

(HW/SW)

1M – 10M scenariosRequirements

Multiple variants

Certification -

HomologationSimulation definition

Requirements &

system architectures

Real world

Vehicle under development



Massive Verification and Validation of ADAS and AVs

Requirements

Certification -

HomologationSimulation definition

Requirements &

system architectures

Real world

Vehicle under development

Digital Twin

“World”

Digital Twin

“Vehicle”

Multiple variants

1M – 10M scenarios

MiL / SiL / Cluster

V&V environments

HiL / DiL / ViL

Proving ground /

field test



Challenge:

From thousands of scenarios (or millions of

miles) to the relevant critical representation

Closed loop automated process

for generating critical scenarios

Orchestration of virtual test scenarios

“Falsification”

Identify critical scenarios

Digital Twin of the World

1000’s of scenarios(weather, light, road types, …) (sensors, controls,

powertrain, chassis)

…

Data Mining, Analytics

Optimize vehicle only against

relevant critical scenarios

Digital Twin of the

Test Vehicle



Simcenter PrescanVirtual testing of autonomous driving functions

Complete sensor models library:

Camera, Radar, LIDAR, Ultrasone, Infrared, V2X, GPS

Scenario 1 - Adaptive Cruise Control ACCScenario 2 – Advanced Emergency Braking System

AEBS



Simcenter Prescan: camera simulationGround truth: depth camera example



Simcenter PrescanWorld modelling solutions

Scenario import

Scripted scenario generation

Ready to use scenarios

GUI



Simcenter PrescanWorld modelling: non-ideal environment

Realistic bumped asphalt Faded, dirty lane markers

Non-perfect lane markers Lane markers with snow Mud, water puddles on the road



Simcenter Prescan – Scenario generationFrom real data to simulation

16



Simcenter

Prescan

Simcenter PrescanWorld modelling: custom data source import

KITTI Dataset

Ego

GPS Data

Target

GPS Data

Ego state

Prescan API

Road

network

Target

states

Target types



Simcenter PrescanWorld modelling: DataModel API

Explore critical scenarios Prepare for certification

• Prescan DataModel API

→ programmatic creation of scenarios

→ Repeatability

• All important assets can be created via

scripting:

• Roads

• Actors

• Traffic signs

• Nature elements

• Trajectories

• Environmental conditions

• Etc.

Parameter variation



V2X

&

Ult

raso

nic

Rad

ar

&

Lid

ar

Cam

era

Simcenter PrescanReady to use sensor models

Sensor simulation

V2N

V2V

V2P V2I



Simcenter PrescanSensors models: the right fidelity level for scaled-up simulation

Balancing accuracy and

computation time of

sensor simulations

Lidar (spinning and solid-state)

Example: during night-time driving Example: Realistic lighting conditions

Simcenter Prescan Physics Based Camera (PBC) simulation

Physics-based Radar simulation



Radar simulation exampleDevelopment with model validation in mind

Two projects for radar models validation performed in close collaboration with

major Dutch Tier2 and Japanese Tier1

From a lab… To a test track… To the real world…



Radar SimulationValidating simulation results against measured data

Real World Testing

• Vehicle with radar

• Range-doppler measurements

Simulation Testing

• Simulated vehicle using the

physics-based radar model

• Range-doppler data generated

based on the simulated scenario



Simcenter AmesimWhen higher fidelity vehicle dynamics makes the difference!

For AEBS, ESC pump dynamics is

critical.

For level 4-5, redundancy will be

ensured by the ESC, the EPB and

the eBooster.

When level 4-5, we

will probably work with

steer by wire and

motor redundancy.Powertrain and braking systems

models for ACC cases

Pick the relevant fidelity level from

Simcenter Amesim

scalable modeling offer

Full vehicle dynamics models for

AEB safety cases



Simcenter AmesimWhen higher fidelity vehicle dynamics makes the difference!

45 Libraries / 4,000 Multi-physics Models

• Validated and maintained

• Supporting multiple levels of complexity

• No need for details physics expertise

• Hydraulic, hydraulic component design

• Hydraulic resistance, filling

• Pneumatic, pneumatic component design

• Gas Mixture, moist airFluids

• Signal and control

• Engine signal generator

• Real time, MIL – SIL – HIL

Control

• 1D – 2D – 3D mechanical,

• Transmission, cam and followers

• Finite-elements import

• Vehicle dynamicsMechanics

• IFP drive, IFP engine

• IFP exhaust

• CFD-1D

IC Engine

• Electrical basics, electromechanical

• Electrical motors and drives

• Electrical static conversion

• Automotive electrics, electrochemicalElectrics

• Fuel cell

• Battery

• Power generation

Energy

• Thermal, thermal hydraulics

• Thermal-hydraulic component design

• Cooling, air-conditioning

• Two-phase flowThermal



Simcenter Prescan360

Scenario authoring

Models integration environment

Sensors and environment simulationSimcenter Prescan

Process automation

Simulation plan orchestrator

Results analysis and reportingHEEDS

Third partyVehicle dynamics

Off-the-shelf validation

scenarios, metrics and

dashboards

Vehicle dynamicsSimcenter Amesim

OR




Simulation production: overall workflow and AEBS example

Numerous results analysis

Reporting

Simulation plan definition

Simulation plan execution

Scenarios Ego models

Azure

Kubernetes



Scripted scenario generation automates the process of creating scenarios at scale

Test Automation /

Design

Optimization Tool

Orchestration

• HEEDS

• Prescan APIs

• 3rd party tooling

Scenario Change

• Parametric sweeping

• Design of Experiments

• Optimization

How to run

• Single machine

• Distributed

• Cloud and cluster

Results

• Local

• Cluster

• Test automation Interface


Create wide variability with on cloud and cluster




Do not simulate any scenario.

Simulate critical scenarios related to

your application!

BUT

How do we identify critical scenarios? Should this scenario be simulated?



Simcenter Prescan360 Benefits

Plan Execute Report

• Process Automation: avoidance of manual errors

• No manual creation of scenarios saves time

• Multiple scenario testing for algorithms

• Identify critical scenarios for each individual application

• Deep insight in highly complex correlations

• Realistic simulations

• Verification traceability: ready for regulations

• SAFE TIME

• REDUCE COSTS

• ENSURE HIGHEST QUALITY

• BE INNOVATIVE

Speed Up the Development of

Autonomous Vehicles with


Where today meets tomorrow.Nico Nagl - E-Mail: [email protected]

validation & verification€¦ · design, validation & verification framework for adas and...

Documents