POWER, PARALLEL AUTONOMY, AND PEOPLEGill Pratt | CEO at Toyota Research Institute | GTC 2016

1.2 Million People

Part 1: Power How much power should it taketo drive an autonomous car?

CURRENTLY: THOUSANDS OF WATTS

Another Solution

30 WWorks even while daydreaming

How Does It Do That ?

DARPA M3-Actuation

Energy efficiency is improved by sparse utilization of complexity

Approved for Public Release, Distribution Unlimited

DARPA Neovision2: Sparse utilization of computing

complexity can improve energy efficiency as well

Approved for Public Release, Distribution Unlimited

DARPA SYNAPSE: BRAINS VS. VON NEUMANN COMPUTERS

Brains• Complexity Less Constrained

› Human Brain: 1014 synapses

• SWaP Highly Constrained

› Human Brain

› 30 W

› 3 pounds

• Sparse, Distributed Computation

› Dedicated Functionality

› Unary (Spike) Coding

Computers• Complexity Highly Constrained

› Xbox One 28 nm SoC: 5 x 109 Transistors

• SWaP Less Constrained

› IBM Sequoia Blue Gene/Q (98,304 16-core Power-PC chips) simulating 5 x 1013 synapses

› 8 Million Watts @ 1/1000 real time

› 500,000 pounds @ 1/1000 real time

• Von Neumann Architecture

› Multiplexed Functionality

› Binary Coding

20,000 : 1

1 : 500,000,000

1 : 300,000,000

Approved for Public Release, Distribution Unlimited

DARPA SyNAPSE: IBM True North

SyNAPSE Chip: 5 Billion Transistors256 Million Synapses (1 Human Brain / 400,000)

50 mW @ real time3 x 10-13 W/synapse (Brain) : 2 x 10-10 W/synapse (SyNAPSE)

1 : 1000

CONCLUSION: POWER

• Buy More Hardware

• Unroll Computations

• Avoid Multiplexers

• Turn Most Hardware Off Most of the Time

Part 2: Parallel Autonomy

Must we achieve Level 4 to deal with the handoff problem?

DARPA Robotics Challenge (Finals - June, 2015)

24 Teams Came From Around the World

DARPA Approved for Public Release, Distribution Unlimited

Program Structure

DARPA Approved for Public Release, Distribution Unlimited

SOME FELL OVER

SOME DID GREAT (KAIST)

MODEL BASED SERIES AUTONOMY

DARPA Approved for Public Release, Distribution Unlimited

Operator DRC RobotDegraded Network Emulator

𝐴𝑢𝑡𝑜𝑛𝑜𝑚𝑦 ∝1

𝐶𝑜𝑚𝑚𝑢𝑛𝑖𝑐𝑎𝑡𝑖𝑜𝑛

DARPA ARM-S – LIMITING CASE OF 0POST-COMMAND COMMUNICATIONS (JPL)

DARPA Approved for Public Release, Distribution Unlimited

Is This the Only Model of Autonomy ?

Three modes of shared control

PARALLEL AUTONOMY: BRAIN COMPUTER INTERFACE ONQUADRIPLEGIC BOX AND BLOCKS 2-MINUTE TIMED TEST

DARPA Approved for Public Release, Distribution Unlimited

ARM + RP Collaboration

PM : Gill Pratt (DSO) in collaboration with Justin Sanchez (BTO)

Performers: CMU/NREC (Bagnell et. al.) + U. Pittsburgh (Schwartz et. al.)

How Does this Apply to Autonomous Driving ?

AKIO TOYODA’S PRIORITIES

• Safety

• Environment

• Mobility for All

• Fun to Drive

NEEDED RELIABILITY NUMBERS

• About 10 Million Toyota cars are produced / yr.

• Each car lasts about 10 years

› About 100 Million Toyota cars are in service

• Each car is driven about 10 thousand miles / yr.

› Toyota cars are driven about 1 Trillion miles / yr.

• It takes only a few defect-caused accidents / yr. to cause an existential crisis

SERIES (CHAUFFER) VS. PARALLEL (GUARDIAN ANGEL)

Note: Technology Supporting Guardian Angel + Chauffer are similar

Aspect Chauffer (Series Autonomy) Guardian Angel (Parallel Autonomy)

Duty Cycle 100% < 1% : only if accident imminent

Liability Manufacturer Mostly Driver

Required Competence All of Driving Do No Harm

Development All or Nothing

lives lost until done

Incremental

lives saved sooner

Driver’s Skills Ignored Utilized as much as possible

Fun + Love of Car

Decreased

car becomes train

Increased

allows high performance experience by novice drivers

Handoff Problem ? Yes No

SERIES (CHAUFFER) VS. PARALLEL (GUARDIAN ANGEL)

Parallel(Safety)Time, Complexity

ABS CollisionAvoidanceSteering

Front Collision Warn

ASC CollisionAvoidanceBraking

SideCollisionWarn

LaneDepartureWarn

CollisionAvoidanceAcceleration

LaneDeparturePrevention

Series (Convenience)

Time, Complexity

Cruise

HybridAutonomy

Adaptive Cruise

Lane KeepingNo Hands

AutomaticHighway Passing

NAV

SpeedWarn

SpeedingPrevention

AutomaticParking

Slow SpeedAuto Driving

Medium SpeedAuto Driving

Chauffer

GuardianAngel

TRI’S AUTONOMY RELATED GOALS

• Improve Safety • Improve Access • Diversify Toyota from Mobility Outdoors to Mobility Indoors

But What About the Trillion Miles ?

SIMULATION

• Repeatable Studies of Human – Machine Interface

• Regression Testing for Software Development

• Amplification of Physical Testing

Part 3: People

TRI ANN ARBOR (TRI-ANN)

Cambridge

~50 PeopleSimulation

Ann Arbor

~50 PeopleChauffer

Palo Alto

~150 PeopleGuardian Angel

WHY ANN ARBOR ?

American Center forMobility (Planned)

M-City, MobilityTransformation Center

University ofMichigan

Toyota TechnicalCenters

TRI-ANN ARBOR AREA LEADS

Prof. Ed OlsonArea Lead, Perception

Prof. Ryan EusticeArea Lead, Mapping / Localization

DIDN’T THEY USED TO WORK FOR FORD ?

Ford Edison Assembly Plant, Metuchen, NJ, March 1961

“CO-OPETITION” ACCELERATES PROGRESS

Eiji Toyoda visiting Ford River Rouge Plant, ca. 1950

Our Hope:Constructive

Competition andCollaboration

Car ManufacturersIT CompaniesGovernmentsHardware Manufacturers

Why Co-operate?

1.2 Million People Per Year Demand Nothing Less

CONCLUSION : FIRST EVER BCI DOOR OPENING

DARPA Approved for Public Release, Distribution Unlimited

ARM + RP Collaboration

PM : Gill Pratt (DSO) in collaboration with Justin Sanchez (BTO)

Performers: CMU/NREC (Bagnell et. al.) + U. Pittsburgh (Schwartz et. al.)

Thank You NVIDIA !