enabling data driven development for …...machine learning machine learning [ml] enables the...
TRANSCRIPT
ENABLING DATA DRIVEN DEVELOPMENT FOR AUTONOMOUS DRIVING.
BORIS SCHAUERTE | BMW GROUP2. DECEMBER 2019
Page 1
LESS TRAFFIC
LESS ACCIDENTS
LESS EMISSIONS
MORE TIME
HIGHER FLEXIBILITY
MORE COMFORT
HIGHER SAFETY
NEW MOBILITY CONCEPTSVEHICLES BEING PART OF OUR LIVING SPACE
Page 2
PROMISES OF AUTONOMOUS DRIVING TO CUSTOMERS AND SOCIETY.
2. December 2019 | Boris Schauerte
LESS TRAFFIC
LESS ACCIDENTS
LESS EMISSIONS
MORE TIME
HIGHER FLEXIBILITY
MORE COMFORT
HIGHER SAFETY
NEW MOBILITY CONCEPTSVEHICLES BEING PART OF OUR LIVING SPACE
Page 3
PROMISES OF AUTONOMOUS DRIVING TO CUSTOMERS AND SOCIETY.
2. December 2019 | Boris Schauerte
CHALLENGE 1: COMPLEX FUNCTION DEVELOPMENT
LESS TRAFFIC
LESS ACCIDENTS
LESS EMISSIONS
MORE TIME
HIGHER FLEXIBILITY
MORE COMFORT
HIGHER SAFETY
NEW MOBILITY CONCEPTSVEHICLES BEING PART OF OUR LIVING SPACE
Page 4
PROMISES OF AUTONOMOUS DRIVING TO CUSTOMERS AND SOCIETY.
2. December 2019 | Boris Schauerte
CHALLENGE 1: COMPLEX FUNCTION DEVELOPMENTCHALLENGE 2: PROVE SAFETY
ARTIFICIAL INTELLIGENCE VALUE LOOP.
AUTONOMOUS DRIVING.
TRAINING
DATA COLLECTION
DATA
ECO SYSTEM
ARTIFICIAL
INTELLIGENCE
ARTIFICIAL INTELLIGENCE VALUE LOOP.AUTONOMOUS DRIVING.
2. December 2019 | Boris Schauerte
ARTIFICIAL INTELLIGENCE VALUE LOOP.
AUTONOMOUS DRIVING.
ON-BOARD OFF-BOARD
PROCESSING
TRAININGDEPLOYMENT
DATA COLLECTION
CONNECTED
ENVIRONMENT
DATA
ECO SYSTEM
ARTIFICIAL
INTELLIGENCE
ARTIFICIAL INTELLIGENCE VALUE LOOP.AUTONOMOUS DRIVING.
2. December 2019 | Boris Schauerte
SEVERAL TB PER HOUR PER CAR
TARGETED DATA COLLECTION
FREQUENTLY UPDATED SW & HW
DEDICATED DATA COLLECTION AND TEST FLEET.
2. December 2019 | Boris Schauerte
CAN COVER MILLIONS OF MILES
FULL RANGE RADAR.
NIGHT VISION.SIDE VIEW CAMERA.
SIDE RANGERADAR
SURROUND VIEW CAMERA.
ULTRA-SONIC.
STEREO FRONT CAMERA.
REAR VIEW CAMERA.
SIDE RANGE RADAR.
ULTRA-SONIC.
STEERING AND LANE CONTROL ASSISTANT INCL. LANE CHANGE ASSISTANT.
SURROUND VIEW.
ACTIVE CRUISE CONTROL.
SPEED LIMIT ASSIST.
EMERGENCY STEERING ASSIST.
WRONG WAY ASSIST.
CROSSROAD ASSIST.
CUSTOMER FLEET DATA COLLECTION.
2. December 2019 | Boris Schauerte
SEVERAL MB PER HOUR
TARGETED DATA COLLECTION
STABLE SW & HW
CAN COLLECT BILLIONS OF MILES
ARTIFICIAL INTELLIGENCE VALUE LOOP.
AUTONOMOUS DRIVING.DATA LANDSCAPE.
2. December 2019 | Boris Schauerte
COLLECT STORE
ACCESSPROCESS
Data Quality
Data Retention
Indexing
Data transfer and IngestFleet control
Data recorder
Campaign mgmt.
Synthethic Data
Reprocessing
Deployment
KPIs
Function dev.
Analytics (e.g, test space coverage)(Auto-)labeling
Data(set) export
Versioning
Privacy (e.g., GDPR) Distributed Infrastructure (due to
to technical or legal constraints)
SecurityVarying sources (e.g., sensor
setups, versions, …)
Data Trustworthiness
Scalability and Orchestration
BUILDING A AUTONOMOUS DRIVING DATA FACTORY.
AUTOMATION & ARCHITECTURE:Automated processes are necessary to handle the volume.
INTRODUCTION & DATA SOURCES: Data-driven development needs large amounts of data.
Page 10
2. December 2019 | Boris Schauerte
Development Fleet
Customer Fleet
Data Collection & Preparation
Raw Data
Ground truth
generationData clearing & post
processingLabeling & Indexing
Classify sequences
Data quality
assurance,
Data access
API
DATA FACTORY.
2. December 2019 | Boris Schauerte
Development Fleet
Customer Fleet
Data Collection & Preparation
Raw Data
Ground truth
generationData clearing & post
processingLabeling & Indexing
Classify sequences
Data quality
assurance,
Data access
API
DATA FACTORY.
2. December 2019 | Boris Schauerte
Post-
Pro
cessed
Data
Evaluation FrameworkEvaluation of safety,
performance, comfort, test
coverage etc.
Development Fleet
Customer Fleet
Data Collection & Preparation
Raw Data
Ground truth
generationData clearing & post
processingLabeling & Indexing
Classify sequences
Data quality
assurance,
Data access
API
DATA FACTORY.
2. December 2019 | Boris Schauerte
Post-
Pro
cessed
Data
Evaluation FrameworkEvaluation of safety,
performance, comfort, test
coverage etc.
Simulation
Bus Simulation
Traffic Models
Sensor Models
Vehicle Model
Environment Simulation
Development Fleet
Customer Fleet
Data Collection & Preparation
Raw Data
Ground truth
generationData clearing & post
processingLabeling & Indexing
Classify sequences
Data quality
assurance,
Data access
API
DATA FACTORY.
2. December 2019 | Boris Schauerte
Post-
Pro
cessed
Data
Evaluation FrameworkEvaluation of safety,
performance, comfort, test
coverage etc.
Simulation
Bus Simulation
Traffic Models
Sensor Models
Vehicle Model
Environment Simulation
Scenario
Descriptions
Development Fleet
Customer Fleet
Data Collection & Preparation
Raw Data
Ground truth
generationData clearing & post
processingLabeling & Indexing
Classify sequences
Data quality
assurance,
Data access
API
DATA FACTORY.
2. December 2019 | Boris Schauerte
Simulation
Bus Simulation
Traffic Models
Sensor Models
Vehicle Model
Environment Simulation
Data, Simulation & Test Management
Evaluation FrameworkEvaluation of safety,
performance, comfort, test
coverage etc.
Traffic to DescriptionDerive (simulatable) scenario
from post-processed data
Scenario
Descriptions
Post-
Pro
cessed
Data
ARTIFICIAL INTELLIGENCE VALUE LOOP.
AUTONOMOUS DRIVING.EXTENDING THE SCENARIO SPACE WITH SIMULATED DATA.
2. December 2019 | Boris Schauerte
Vehicle
Speed
Number of
pedestriansObserved
Scenario
ARTIFICIAL INTELLIGENCE VALUE LOOP.
AUTONOMOUS DRIVING.EXTENDING THE SCENARIO SPACE WITH SIMULATED DATA.
2. December 2019 | Boris Schauerte
Vehicle
Speed
Number of
pedestriansObserved
ScenarioSimulated
Scenario
Seite 18
DATA VARIATION. C S
AP
2. December 2019 | Boris Schauerte
Data, Simulation & Test Management
Scenario EvaluationEvaluation Framework
Evaluation of safety,
performance, comfort, test
coverage etc.
Scenario Extraction
Traffice to DescriptionDerive (simulatable) scenario
from post-processed data
Development Fleet
Customer Fleet
Data Collection & Preparation
Raw Data
Ground truth
generationData clearing & post
processingLabeling & Indexing
Classify sequences
Data quality
assurance,
Data access
API
Scenario Variation
Auto. Scenario
Variation
DATA FACTORY.
2. December 2019 | Boris Schauerte
Simulation
Bus Simulation
Traffic Models
Sensor Models
Vehicle Model
Environment SimulationP
ost-
Pro
cessed
Data
Scenario
Descriptions
Data, Simulation & Test Management
Scenario EvaluationEvaluation Framework
Evaluation of safety,
performance, comfort, test
coverage etc.
Scenario Extraction
Traffice to DescriptionDerive (simulatable) scenario
from post-processed data
Development Fleet
Customer Fleet
Data Collection & Preparation
Raw Data
Ground truth
generationData clearing & post
processingLabeling & Indexing
Classify sequences
Data quality
assurance,
Data access
API
Scenario Variation
Auto. Scenario
Variation
DATA FACTORY.
2. December 2019 | Boris Schauerte
Simulation
Bus Simulation
Traffic Models
Sensor Models
Vehicle Model
Environment SimulationP
ost-
Pro
cessed
Data
Scenario
Descriptions
Test Space
ExplorationGenerate necessary
parameter variations
Test Space AnalyticsDerive model and statistics
for scenarios
Test Space CoverageStatistical coverage of the
test space
Data, Simulation & Test Management
Scenario EvaluationEvaluation Framework
Evaluation of safety,
performance, comfort, test
coverage etc.
Scenario Extraction
Traffice to DescriptionDerive (simulatable) scenario
from post-processed data
Development Fleet
Customer Fleet
Data Collection & Preparation
Raw Data
Ground truth
generationData clearing & post
processingLabeling & Indexing
Classify sequences
Data quality
assurance,
Data access
API
Scenario Variation
Auto. Scenario
Variation
DATA FACTORY.
2. December 2019 | Boris Schauerte
Simulation
Bus Simulation
Traffic Models
Sensor Models
Vehicle Model
Environment Simulation
Scenario
Descriptions
Test Space
ExplorationGenerate necessary
parameter variations
Test Space AnalyticsDerive model and statistics
for scenarios
Test Space CoverageStatistical coverage of the
test space
Reprocessing Platform
Post-
Pro
cessed
Data
Data, Simulation & Test Management
Scenario EvaluationEvaluation Framework
Evaluation of safety,
performance, comfort, test
coverage etc.
Scenario Extraction
Traffice to DescriptionDerive (simulatable) scenario
from post-processed data
Development Fleet
Customer Fleet
Data Collection & Preparation
Raw Data
Ground truth
generationData clearing & post
processingLabeling & Indexing
Classify sequences
Data quality
assurance,
Data access
API
Scenario Variation
Auto. Scenario
Variation
DATA FACTORY.
2. December 2019 | Boris Schauerte
Simulation
Bus Simulation
Traffic Models
Sensor Models
Vehicle Model
Environment Simulation
Scenario
Descriptions
Test Space
ExplorationGenerate necessary
parameter variations
Test Space AnalyticsDerive model and statistics
for scenarios
Test Space CoverageStatistical coverage of the
test space
Version
management
Testcase
s
Scenario
Evaluatio
n results
Reprocessing Platform
Post-
Pro
cessed
Data
Data, Simulation & Test Management
Scenario EvaluationEvaluation Framework
Evaluation of safety,
performance, comfort, test
coverage etc.
Scenario Extraction
Traffice to DescriptionDerive (simulatable) scenario
from post-processed data
Development Fleet
Customer Fleet
Data Collection & Preparation
Raw Data
Ground truth
generationData clearing & post
processingLabeling & Indexing
Classify sequences
Data quality
assurance,
Data access
API
Scenario Variation
Auto. Scenario
Variation
DATA FACTORY.
2. December 2019 | Boris Schauerte
Simulation
Bus Simulation
Traffic Models
Sensor Models
Vehicle Model
Environment Simulation
Scenario
Descriptions
Test Space
ExplorationGenerate necessary
parameter variations
Test Space AnalyticsDerive model and statistics
for scenarios
Test Space CoverageStatistical coverage of the
test space
Version
management
Testcase
s
Scenario
Evaluatio
n results
Reprocessing Platform
Post-
Pro
cessed
Data
Data
retention
Data
retention
Data, Simulation & Test Management
Scenario EvaluationEvaluation Framework
Evaluation of safety,
performance, comfort, test
coverage etc.
Scenario Extraction
Traffice to DescriptionDerive (simulatable) scenario
from post-processed data
Development Fleet
Customer Fleet
Data Collection & Preparation
Raw Data
Ground truth
generationData clearing & post
processingLabeling & Indexing
Classify sequences
Data quality
assurance,
Data access
API
Scenario Variation
Auto. Scenario
Variation
DATA FACTORY.
2. December 2019 | Boris Schauerte
Simulation
Bus Simulation
Traffic Models
Sensor Models
Vehicle Model
Environment Simulation
Scenario
Descriptions
Test Space
ExplorationGenerate necessary
parameter variations
Test Space AnalyticsDerive model and statistics
for scenarios
Test Space CoverageStatistical coverage of the
test space
Version
management
Testcase
s
Scenario
Evaluatio
n results
Reprocessing Platform
Post-
Pro
cessed
Data
Embedded test
platforms
Test
automation
Data
retention
Data
retention
Data, Simulation & Test Management
Scenario EvaluationEvaluation Framework
Evaluation of safety,
performance, comfort, test
coverage etc.
Scenario Extraction
Traffice to DescriptionDerive (simulatable) scenario
from post-processed data
Development Fleet
Customer Fleet
Data Collection & Preparation
Raw Data
Ground truth
generationData clearing & post
processingLabeling & Indexing
Classify sequences
Data quality
assurance,
Data access
API
Scenario Variation
Auto. Scenario
Variation
DATA FACTORY.
2. December 2019 | Boris Schauerte
Simulation
Bus Simulation
Traffic Models
Sensor Models
Vehicle Model
Environment Simulation
Scenario
Descriptions
Test Space
ExplorationGenerate necessary
parameter variations
Test Space AnalyticsDerive model and statistics
for scenarios
Test Space CoverageStatistical coverage of the
test space
Version
management
Testcase
s
Scenario
Evaluatio
n results
Reprocessing Platform
Embedded test
platforms
Machine Learning Platform
New Software
Increment
Post-
Pro
cessed
Data
Test
automation
Data
retention
Data
retention
Data, Simulation & Test Management
Scenario EvaluationEvaluation Framework
Evaluation of safety,
performance, comfort, test
coverage etc.
Scenario Extraction
Traffice to DescriptionDerive (simulatable) scenario
from post-processed data
Test Space AnalyticsDerive model and statistics
for scenarios
Embedded test
platforms
Orchestration of
processesSecurity ConceptData Center Integration
Platform
Access Management & IP Protection (e.g.
Partners)
Reprocessing Platform
Machine Learning Platform
Development Fleet
Customer Fleet
Data Collection & Preparation
Raw Data
Ground truth
generationData clearing & post
processingLabeling & Indexing
Classify sequences
Data quality
assurance,
Data access
API
Scenario Variation
Auto. Scenario
Variation
Test Space
ExplorationGenerate necessary
parameter variations
Test Space CoverageStatistical coverage of the
test space
DATA FACTORY.
2. December 2019 | Boris Schauerte
New Software
Increment
Simulation
Bus Simulation
Traffic Models
Sensor Models
Vehicle Model
Environment SimulationP
ost-
Pro
cessed
Data
Scenario
Descriptions
Version
management
Testcase
s
Scenario
Evaluatio
n results
Data
retention
Data
retention
Page 28
2. December 2019 | Boris Schauerte
TEST LANDSCAPE.
INFORMATION TECHNOLOGY [IT]
WHAT IS ARTIFICIAL INTELLIGENCE?THE AUTOMATION OF INTELLIGENT BEHAVIOR.
ARTIFICIAL INTELLIGENCE
Knowledge
representation
Perception and
cognition
Planning and
reasoning…
ARTIFICIAL INTELLIGENCE [AI] is a subdivision
of computer science. AI describes a set of algorithmic
methodologies to solve complex optimization problems.
Automates and supports decisions.
MACHINE LEARNING
Random forest Neural networks
Support vector
machine…
MACHINE LEARNING [ML] enables the recognition of
patterns and correlations in data automatically as apposed to
applying static rules and definitions. Allows the identification
of formerly hidden semantics.
DEEP LEARNING
DEEP LEARNING [DL] describes a class of
optimization methods loosely inspired by the
structure of biological nervous systems.
DL is able to produce results comparable to
and
in some cases superior to human experts,
especially in the field of image recognition
and natural language processing.
Deep
neural nets
Artificial
neural nets
…
19801950 2010EXPERT SYSTEMS SELF-
LEARNING SYSTEMS
rule based data driven
WHAT IS ARTIFICIAL INTELLIGENCE?THE AUTOMATION OF INTELLIGENT BEHAVIOR.
2. December 2019 | Boris Schauerte
INFORMATION TECHNOLOGY [IT]
WHAT IS ARTIFICIAL INTELLIGENCE?THE AUTOMATION OF INTELLIGENT BEHAVIOR.
ARTIFICIAL INTELLIGENCE
ARTIFICIAL INTELLIGENCE [AI] is a subdivision
of computer science. AI describes a set of algorithmic
methodologies to solve complex optimization problems.
Automates and supports decisions.
MACHINE LEARNING
MACHINE LEARNING [ML] enables the recognition of
patterns and correlations in data automatically as apposed to
applying static rules and definitions. Allows the identification
of formerly hidden semantics.
DEEP LEARNING
DEEP LEARNING [DL] describes a class of
optimization methods loosely inspired by the
structure of biological nervous systems.
DL is able to produce results comparable to
and
in some cases superior to human experts,
especially in the field of image recognition
and natural language processing.
ImageNet ’15
14m pics & 20k
cats
MS COCO ’14
2.5m inst & 91
cats
…
19801950 2010EXPERT SYSTEMS SELF-
LEARNING SYSTEMS
rule based data driven
WHAT MADE THESE BREAKTHROUGHS POSSIBLE?ONE FACTOR, THE EMERGENCE OF ACCESS TO HIGH-QUALITY DATA.
MNIST ’98
70k pics & 10 cats
PASCAL VOC ’05
20k pics & 20 cats
ImageNet ’09
3.2m pics & >5k
cats
2. December 2019 | Boris Schauerte