new machine vision sensors for autonomous driving
TRANSCRIPT
NEW MACHINE VISION SENSORS FOR
AUTONOMOUS DRIVING
RONALD MUELLER
VISION MARKETS
EMVA BUSINESS CONFERENCE 2018, DUBROVNIK
2018-06-09
Trends
Full Screen image on trendsYour global growth in Imaging and Photonics
stronger
Brandmore
Salesbetter
Offeringsbigger
Company
Trends
Full Screen image on trends
51 customers served
since 2015:
• MV OEMs
• Private Equity Firms
• MV-Users
• System Integrators
• Distributors
• EMSs
• Market researchers
• Mobile Phone OEM
Trends
Full Screen image on trends
51 customers served
since 2015:
• MV OEMs
• Private Equity Firms
• System Integrators
• Distributors
• EMSs
• Market researchers
• Mobile Phone OEM
http://www.euclidlabs.it/event/
TRENDS
full screen image on welcome and or first things first
Market Trends
1893
175 15766 1,95
72
1855
177 161 982,25
72
0
200
400
600
800
1000
1200
1400
1600
1800
2000
Mobile phones Tablets Notebooks SurveillanceCams
MV Cams Other
Million
units
Shipments of Devices with Image Sensors
2016 2017
IMAGE SENSOR MARKET
av. 2 image sensors / phone & tablet
GROWTH IN SURVEILLANCE & MV
4092
1241
41997
5360
1340
604150
0
1000
2000
3000
4000
5000
6000
Hikvision FLIR Systems Cognex Basler
Turn
ove
r [m
€]
Surveillance & Machine Vision – Selected Players
2016 2017
AUTOMOTIVE
73
77
80
68
70
72
74
76
78
80
82
Cars
Million
units
Global Car Production
2015 2016 2017
Every new car sold in
US is equipped with
a rear-view camera
Up to 12
image sensors/car
AUTOMOTIVE VISION MARKET: ~25BN$ BY 2025
Image source: Autoliv
VISION BOM IN L4/5 CARS: 385 $
Automotive
requirements are
driving sensor
technology
developments
QUIZADAS & ADS
Image source: Youtube
Image source: Youtube
Image source: Youtube
Image source: Youtube
Image source: Youtube
Image source: Youtube
Image source: Youtube
BASICS OF ADAS & ADS
• Standards
• SAE International – J3016:
Taxonomy and Definitions for Terms
Related to Driving Automation Systems
for On-Road Motor Vehicles
• Terminology
• ADAS: Advanced Driver Assistance System
• ADS: Autonomous Driving System
• DDT: Dynamic Driving Task
• Levels: 0 – 5
Image source: SAE International
AUTONOMOUS DRIVING – SAE LEVELS 0 - 5Le
vel
NameLateral &
Longitudinal ControlMonitoring Fallback
Automated
Driving Modes
0 No Automation n/a
1 Driver Assistance Some
2 Partial Automation Some
3 Cond. Automation Some
4 High Automation Many
5 Full Automation All
Level
NameLateral &
Longitudinal ControlMonitoring Fallback
Automated
Driving Modes
0 No Automation n/a
1 Driver Assistance Some
2 Partial Automation Some
3 Cond. Automation Some
4 High Automation Many
5 Full Automation All
AUTONOMOUS DRIVING – SAE LEVELS 0 - 5
Level
NameLateral &
Longitudinal ControlMonitoring Fallback
Automated
Driving Modes
0 No Automation n/a
1 Driver Assistance Some
2 Partial Automation Some
3 Cond. Automation Some
4 High Automation Many
5 Full Automation All
AUTONOMOUS DRIVING – SAE LEVELS 0 - 5
IMAGE SENSORS IN SAE LEVEL 1&2
•Machine Vision
• Driving lane recognition
(departure warning / automatic steering)
• 3D localization of other road users
(driver alerting / emergency breaking,
lane changes, adaptive front lights)
• Street sign recognition
• Driver monitoring
•Human vision (surround view)
•Night vision
https://www.sensorsmag.com/components/three-sensor-types-drive-autonomous-vehicles
IMAGE SENSORS IN SAE LEVEL 3-5
Additional functions:
•3D sensing of the road
• Lanes, road curbs, parking cars,
junctions, construction work, etc.
•3D sensing of mobile objects
• Other vehicles, cyclists, pedestrians
• Pets and wildlife
• Obstacles
•2D recognition of traffic lights
Image source: Nvidia
CHALLENGES FOR AUTOMOTIVE IMAGE SENSORS
• Sunlight, head lights, reflections
• Low light
• Compactness
• Car speed
• High spatial & depth resolution
• LED Flicker
• Temperatures
• Reliability
• Security
Image source: Sensormag
Source: Ni78.com
SENSOR ARRAY IN MERCEDES S-CLASS
Image: https://blog.caranddriver.com/sensory-overload-how-the-new-mercedes-s-class-sees-all/
SENSOR ARRAY IN MERCEDES S-CLASS
Image: https://blog.caranddriver.com/sensory-overload-how-the-new-mercedes-s-class-sees-all/
4 Wide angle
cameras for 360°
surround view
NIR camera
for night vision
display
Infra-red camera to spot
people and wildlife up to
160 m
Visible spectrum stereo camera
for object detection up to 400 m
3D sensing up to 50 m
IMAGE SENSORS REQUIRED FOR ADS LEVEL 3 - 5
https://www.sensorsmag.com/components/three-sensor-types-drive-autonomous-vehicles
TECHNICAL DEVELOPMENTS IN AUTOMOTIVE
Compact front-view stereo
camera modules
•3D sensing at up to 100 m at
35° FOV
•Traffic sign/light recognition
Image: http://www.safecar.info/early-sunsets-require-different-driving-techniques/
Image source: Denso
Image source: Subaru
TECHNICAL DEVELOPMENTS IN AUTOMOTIVE
RCCC image sensors
•Max. sensitivity
•Color recognition for traffic
signs, traffic lights, lanes
Image: http://www.safecar.info/early-sunsets-require-different-driving-techniques/
Image source: Schematic Scholar
TECHNICAL DEVELOPMENTS IN AUTOMOTIVE
Split pixels / Multi-gain readout / Super-Exposure
•Single-exposure HDR: 95 dB
•Multi-exposure HDR: 140 dB
•Full-well capacity of 100 ke-
Image source: Sony
TECHNICAL DEVELOPMENTS IN AUTOMOTIVE
LED Flicker Mitigation in daylight
Image: http://www.safecar.info/early-sunsets-require-different-driving-techniques/
Image source: ON Semiconductor
TECHNICAL DEVELOPMENTS IN AUTOMOTIVE
LiDAR
(Light Detection and
Ranging)
Image: http://www.safecar.info/early-sunsets-require-different-driving-techniques/
TECHNICAL DEVELOPMENTS IN AUTOMOTIVE
SPADs:
Single Photon Avalanche Diodes
SiPM:
Silicon Photon Multipliers
Image: http://www.safecar.info/early-sunsets-require-different-driving-techniques/
TECHNICAL DEVELOPMENTS IN AUTOMOTIVE
Solid state LiDAR at
905 or 1500 nm
•SLAM up to 300 m
•Costs down to
few hundred $
Image: http://www.safecar.info/early-sunsets-require-different-driving-techniques/
ACHIEVEMENTS AT AUTOMOTIVE SENSORS
• Resolution: 2 MP stereo, 8 MP single
• Compactness: 2.25 – 3 µm
• Sensitivity: several ke-/lux s
• Full-well capacity: 100 ke-
• Dark noise: 1 e-
• High Dynamic Range:
multi-exposure: 140 dB
single-exposure: 95 dB
• Stacked sensors with on-chip processing,
e.g. motion compensation at ME-HDR
• Frame rate:
60 fps @ 1080p
40 fps @ 7.4 MP
• Heat resistance: 130°C
• Self-monitoring: ASIL B/C & IEC26262
• On-chip security features
• Solid State Lidar based on SPAD/SiPM
& MEMS
SUMMARY
•Market potential of Automotive Vision
drives image sensor innovations
• Tough requirements and first advancements:
• Imaging performance (HDR, Sensitivity, …)
• Compactness
• Reliability
• Security
• Cost
• Solid State LiDAR about to be solved
Tuesday, June 12, 2018© VISION MARKETS
Curious for more?
NEW MARKET REPORT – AVAILABLE FROM Q3/2018
• Technology challenges deriving from application requirements for ADAS and
autonomous vehicles.
• Scoring and ranking based on technical difficulty and impact
• Detailed analysis of the current status of solutions for each top ranked technology
challenge, and forecast for the state-of-the-art until 2023
Top 20 Technology Challenges for
Image Sensors in Automotive to 2023
CONTACT
Ronald MüllerPresident, Vision Markets UG
t (DE): +49 89 21 553 665t (US): +1 302 231 1343
Your global growth in Imaging and Photonics