POWER, PARALLEL AUTONOMY, AND PEOPLEGill Pratt | CEO at Toyota Research Institute | GTC 2016
1.2 Million People
Part 1: Power How much power should it taketo drive an autonomous car?
CURRENTLY: THOUSANDS OF WATTS
Another Solution
30 WWorks even while daydreaming
How Does It Do That ?
DARPA M3-Actuation
Energy efficiency is improved by sparse utilization of complexity
Approved for Public Release, Distribution Unlimited
DARPA Neovision2: Sparse utilization of computing
complexity can improve energy efficiency as well
Approved for Public Release, Distribution Unlimited
DARPA SYNAPSE: BRAINS VS. VON NEUMANN COMPUTERS
Brains• Complexity Less Constrained
› Human Brain: 1014 synapses
• SWaP Highly Constrained
› Human Brain
› 30 W
› 3 pounds
• Sparse, Distributed Computation
› Dedicated Functionality
› Unary (Spike) Coding
Computers• Complexity Highly Constrained
› Xbox One 28 nm SoC: 5 x 109 Transistors
• SWaP Less Constrained
› IBM Sequoia Blue Gene/Q (98,304 16-core Power-PC chips) simulating 5 x 1013 synapses
› 8 Million Watts @ 1/1000 real time
› 500,000 pounds @ 1/1000 real time
• Von Neumann Architecture
› Multiplexed Functionality
› Binary Coding
20,000 : 1
1 : 500,000,000
1 : 300,000,000
Approved for Public Release, Distribution Unlimited
DARPA SyNAPSE: IBM True North
SyNAPSE Chip: 5 Billion Transistors256 Million Synapses (1 Human Brain / 400,000)
50 mW @ real time3 x 10-13 W/synapse (Brain) : 2 x 10-10 W/synapse (SyNAPSE)
1 : 1000
CONCLUSION: POWER
• Buy More Hardware
• Unroll Computations
• Avoid Multiplexers
• Turn Most Hardware Off Most of the Time
Part 2: Parallel Autonomy
Must we achieve Level 4 to deal with the handoff problem?
DARPA Robotics Challenge (Finals - June, 2015)
24 Teams Came From Around the World
DARPA Approved for Public Release, Distribution Unlimited
Program Structure
DARPA Approved for Public Release, Distribution Unlimited
SOME FELL OVER
SOME DID GREAT (KAIST)
MODEL BASED SERIES AUTONOMY
DARPA Approved for Public Release, Distribution Unlimited
Operator DRC RobotDegraded Network Emulator
𝐴𝑢𝑡𝑜𝑛𝑜𝑚𝑦 ∝1
𝐶𝑜𝑚𝑚𝑢𝑛𝑖𝑐𝑎𝑡𝑖𝑜𝑛
DARPA ARM-S – LIMITING CASE OF 0POST-COMMAND COMMUNICATIONS (JPL)
DARPA Approved for Public Release, Distribution Unlimited
Is This the Only Model of Autonomy ?
Three modes of shared control
PARALLEL AUTONOMY: BRAIN COMPUTER INTERFACE ONQUADRIPLEGIC BOX AND BLOCKS 2-MINUTE TIMED TEST
DARPA Approved for Public Release, Distribution Unlimited
ARM + RP Collaboration
PM : Gill Pratt (DSO) in collaboration with Justin Sanchez (BTO)
Performers: CMU/NREC (Bagnell et. al.) + U. Pittsburgh (Schwartz et. al.)
How Does this Apply to Autonomous Driving ?
AKIO TOYODA’S PRIORITIES
• Safety
• Environment
• Mobility for All
• Fun to Drive
NEEDED RELIABILITY NUMBERS
• About 10 Million Toyota cars are produced / yr.
• Each car lasts about 10 years
› About 100 Million Toyota cars are in service
• Each car is driven about 10 thousand miles / yr.
› Toyota cars are driven about 1 Trillion miles / yr.
• It takes only a few defect-caused accidents / yr. to cause an existential crisis
SERIES (CHAUFFER) VS. PARALLEL (GUARDIAN ANGEL)
Note: Technology Supporting Guardian Angel + Chauffer are similar
Aspect Chauffer (Series Autonomy) Guardian Angel (Parallel Autonomy)
Duty Cycle 100% < 1% : only if accident imminent
Liability Manufacturer Mostly Driver
Required Competence All of Driving Do No Harm
Development All or Nothing
lives lost until done
Incremental
lives saved sooner
Driver’s Skills Ignored Utilized as much as possible
Fun + Love of Car
Decreased
car becomes train
Increased
allows high performance experience by novice drivers
Handoff Problem ? Yes No
SERIES (CHAUFFER) VS. PARALLEL (GUARDIAN ANGEL)
Parallel(Safety)Time, Complexity
ABS CollisionAvoidanceSteering
Front Collision Warn
ASC CollisionAvoidanceBraking
SideCollisionWarn
LaneDepartureWarn
CollisionAvoidanceAcceleration
LaneDeparturePrevention
Series (Convenience)
Time, Complexity
Cruise
HybridAutonomy
Adaptive Cruise
Lane KeepingNo Hands
AutomaticHighway Passing
NAV
SpeedWarn
SpeedingPrevention
AutomaticParking
Slow SpeedAuto Driving
Medium SpeedAuto Driving
Chauffer
GuardianAngel
TRI’S AUTONOMY RELATED GOALS
• Improve Safety • Improve Access • Diversify Toyota from Mobility Outdoors to Mobility Indoors
But What About the Trillion Miles ?
SIMULATION
• Repeatable Studies of Human – Machine Interface
• Regression Testing for Software Development
• Amplification of Physical Testing
Part 3: People
TRI ANN ARBOR (TRI-ANN)
Cambridge
~50 PeopleSimulation
Ann Arbor
~50 PeopleChauffer
Palo Alto
~150 PeopleGuardian Angel
WHY ANN ARBOR ?
American Center forMobility (Planned)
M-City, MobilityTransformation Center
University ofMichigan
Toyota TechnicalCenters
TRI-ANN ARBOR AREA LEADS
Prof. Ed OlsonArea Lead, Perception
Prof. Ryan EusticeArea Lead, Mapping / Localization
DIDN’T THEY USED TO WORK FOR FORD ?
Ford Edison Assembly Plant, Metuchen, NJ, March 1961
“CO-OPETITION” ACCELERATES PROGRESS
Eiji Toyoda visiting Ford River Rouge Plant, ca. 1950
Our Hope:Constructive
Competition andCollaboration
Car ManufacturersIT CompaniesGovernmentsHardware Manufacturers
Why Co-operate?
1.2 Million People Per Year Demand Nothing Less
CONCLUSION : FIRST EVER BCI DOOR OPENING
DARPA Approved for Public Release, Distribution Unlimited
ARM + RP Collaboration
PM : Gill Pratt (DSO) in collaboration with Justin Sanchez (BTO)
Performers: CMU/NREC (Bagnell et. al.) + U. Pittsburgh (Schwartz et. al.)
Thank You NVIDIA !