Towards Autonomous Vehicles

Chris SchwarzNational Advanced Driving Simulator

Acknowledgements

• Mid-America Transportation Center– 1 year project to survey literature and report on state of

the art in autonomous vehicles– Co-PI: Prof. Geb Thomas– Undergraduate students

• Kory Nelson• Michael McCrary• Mathew Powell• Nicholas Schlarmann

– http://matc.unl.edu/research/research_projects.php?researchID=405– https://www.zotero.org/groups/autonomous_vehicles/items

Why Autonomous Vehicles?

• Safety– 32,000 people killed each year, 93% due to driver error,

billions in property damage– Autonomous vision is ‘crashless’

• Mobility– Safely increase traffic density (x2)-(x3)– Greater access for elderly, disabled, etc.

• Sustainability– Fuel savings due to platooning (20%), eliminating traffic

jams, reducing trip times, reducing ownership, reducing parking spaces

Cycles of Innovation

Vehicle Automation Partner Matrix

Academic Government

Private Military

A G

P M

An early experiment on automatic highways was conducted by RCA and the state of Nebraska on a 400 foot strip of public highway just outside Lincoln (“Electronic Highway of the Future - Science Digest (Apr, 1958)” 2013)

A G

P M

CMU NAVLAB

• RALPH, ALVINN, YARF• In 1995, RALPH drove NAVLAB 5 over 3000 miles

from Pittsburgh to Washington, DC.– Steered autonomously 96% of the way from

Pittsburgh, PA to Washington DCPomerleau, 1995, RALPH: Rapidly Adapting Lateral Position Handler, IEEE Symposium on Intelligent Vehicles, September, 1995

A G

P M

National Automated Highway System

A demonstration of the automated highway system in San Diego (1997). University of California PATH Program

1994-1997

A G

P M

Intelligent Vehicle Initiative

• Prevent driver distraction• Facilitate accelerated

deployment of crash avoidance systems– Normal conditions

• IVIS– Degraded condition

• Visibility, drowsiness– Imminent crash

• Rear end, lane depart, intersection, ESC

1997-2005

Multiple ADAS system. Image from IVBSS materials, courtesy of UMTRI

Forward Crash Warning (FCW)

Lateral Drift Warning (LDW)

Lane-change/Merge (LCM)

Curve speed Warning (CSW)

RadarVision

A G

P M

DARPA Grand Challenge

Grand Challenge:2004 – no winner2005 – Stanley (Stanford)

Urban Grand Challenge2007 – Boss (CMU) A G

P M

Connected Vehicles

• DSRC (5.9 GHz)– Allocated in 2004

• Goals– Safety

• Forward collision, intersection movement assist, lane change, blind spot, do not pass, control loss warning, emergency brake light warning

– Mobility– Sustainability

• AERIS

2004-present

VII -> IntelliDrive -> Connected Vehicles

Regulatory decision from NHTSA recently announced. V2V will eventually be required in new cars. A G

P M

Google Self-Driving Car2010

A G

P M

NHTSA Automation Program

• Licensing• Testing• Regulations

• Cybersecurity• Currently recommends

states only allow testing

2012-present

Level ExampleTransition Time to Manual (Heuristic)

0 – No Automation Warning only --

1 – Function-specific Automation ADAS < 1 second

2 – Combined Function Automation Super cruise < 1 minute

3 – Limited Self-Driving Automation Google car < 10 minutes

4 – Full Self-Driving Automation PRT --

NHTSA Levels of Automation

A G

P M

Future Societal Impacts

Light Cars: A Virtuous Cycle

Reduce mass

Downsize engine

Drivetrain brakes tires

Smaller fuel

supply

Autonomous Car Sharing

MIT’s Stackable City Car

A Bottom-up approach

Advanced Driver Assistance Systems ACC Pre-Crash LDWS

Sensor Year Sensor Year Sensor YearAudi Radar/Video 2011 Camera 2007

BMW Camera 2007Chrysler Laser 2006

Ford Radar 2009 Radar 2009 Camera 2010GM Radar 2004 Camera 2008

Honda Radar 2003 Camera 2003Kia Camera 2010

Jaguar Radar 1999 Lexus Laser 2001

Mercedes Radar 2001 Radar 2002 Camera 2009Nissan Camera 2001

Saab Radar 2002 Toyota Laser 1998 Radar 2003 Camera 2002

Volkswagen Radar/Video 2011 Volvo Radar 2002 Radar/Video 2007

A 2011 review of commercial ADAS systems compares manufacturers, model year, and sensor type for three types of systems (Shaout, Colella, and Awad 2011)

ADAS Automation

Abb. System Abb. SystemESC Electronic Stability Control DD Drowsiness DetectionFCW Forward Collision Warning AL Adaptive LightingACC Adaptive Cruise Control PM Pedal MisapplicationLDW Lane Departure Warning TSR Traffic Sign RecognitionLKA Lane Keeping Assist TJA Traffic Jam AssistantLCA Lane Change Assist CZA Construction Zone Assist

RCTA Rear Cross Traffic Alert PA Parking AssistantBSD Blind Spot Detection PP Parking PilotEBA Emergency Brake Assist HC Highway Chauffeur

AEBS Advanced Emergency Braking System HP Highway PilotESA Emergency Steer Assist

A Top-down Approach

Personal Rapid Transit (PRT)

• Fully autonomous• No operator, no controls• Low speed• May use a guideway• Morgantown PRT

entered operation in 1975 in West Virginia

PRTs (cont.)

• Morgantown, WV• Masdar City (on hold)• London Heathrow

Airport• City Mobil 2• Suncheon, South Korea• Punjab, India

• Early criticisms of PRTs on guideways concern the scalability of the system

• But new concepts are leaving guideways behind, alleviating some of these concerns

Elements of Automation

Automation Sensors

High grade LIDAR Inconspicuous LIDAR GPS / IMU

RADAR Cameras

Digital MapsDSRC

Localization & Object Detection

Probabilistic Methods

• The world is messy with uneven edges, bad lighting, poorly marked roads, and unpredictable people

• Applications of probabilistic reasoning– Histogram filters (lane line tracking)– Particle filters, Kalman filters (object tracking)– Bayesian Networks (decision making)– Hidden Markov Models (state estimation)

Some Online Courses

• Udacity online courses

Digital Maps & Mapping

• Digital maps negate the need to dynamically map the environment

• Simultaneous Localization & Mapping (SLAM) used to create environments in unmapped areas

• Many modern path planning algorithms are based on A* algorithm

• Must find the proper correspondence between the digital map and other sensor inputs

Challenges of Automation

Weather Challenges

Bob Donaldson / Post-Gazette

Testing & Certification

LogicSensor FailuresKalman FiltersFalse Positives

Histogram FiltersParticle Filters

Data FusionMore data (images & video)

More test cases

Path PlanningDecision Making

Digital MapsAll speeds

Parking LotsMany more tests

Transfer of Control

Transfer of Control to a Platoon

Level Example Transition Time to Manual

0 – No Automation Warning only --

1 – Function-specific Automation ADAS < 1 second

2 – Combined Function Automation Super cruise < 1 minute

3 – Limited Self-Driving Automation Google car < 10 minutes

4 – Full Self-Driving Automation PRT --

Example:

Legality

• “Automated vehicles are probably legal in the United States” – Bryant Walker Smith

• 1949 Geneva Convention on Road Traffic requires that the driver of a vehicle shall be at all times able to control it

• Who is liable: the driver or the manufacturer?• California, Nevada, and Florida have paved the

way with state laws for automated vehicles

Hacking Entry PointsEntry point Weakness

TelematicsThe benefit of such systems is that the car can be remotely disabled if stolen, or unlocked if the keys are inside. The weakness is that a hacker could potentially do the same.

MP3 malwareJust like software apps, MP3 files can also carry malware, especially if downloaded from unauthorized sites. These files can introduce the malware into a vehicles network if not walled off from safety-critical systems.

Infotainment apps

Car apps are like smartphone apps…they can carry viruses and malware. If the apps are not carefully screened, or if the car’s infotainment software is not securely walled off from other systems, then an attack can start with a simple app update.

BluetoothThe system that connects your smartphone to your car can be used as another entry point into the in-vehicle network.

OBD-IIThis port provides direct access to the CAN bus, and potentially every system of the car. If the CAN bus traffic is not encrypted, it is an obvious entry point to control a vehicle.

Door LocksLocks are interlinked with other vehicle data, such as speed and acceleration. If the network allows two-way communication, then a hacker could control the vehicle through the power locks.

Tire Pressure Monitoring System

Wireless TPMS systems could be hacked from adjacent vehicles, identify and track a vehicle through its unique sensor ID, and corrupt the sensor readings.

Key FobIt’s possible to extend the range of the key fob by an additional 30’ so that it could unlock a car door before the owner is close enough to prevent an unwanted entry.

Vehicle Networks to SecureNetwork Weakness

LIN Vulnerable at a single point of attack. Can put LIN slaves to sleep or make network inoperable

CAN Can jam the network with bogus high priority messages or disconnect controllers with bogus error messages

FlexRay Can send bogus error messages and sleep commands to disconnect or deactivate controllers

MOST Vulnerable to jamming attacksBluetooth Wireless networks are generally much more vulnerable to attack than wired

networks. Messages can be intercepted and modified, even introducing worms and viruses

Privacy

• Electronic Data Recorders (Black Box)• Identified network traffic• De-identified data– The myth of anonymity

• “Google’s self-driving car gathers almost 1 Gb per second” – Bill Gross, Idealab

Privacy By Design

• Proactive not reactive• Privacy by default• Privacy embedded into the design• Full functionality (positive sum, not zero sum)• End-to-end security (full lifecycle protection)• Visibility and transparency• Respect for user privacy

Discussion

Case Study: Autonomous Intersectionsand Time to Collision Perception

• Time to Collision (TTC)– range / range rate

• Autonomous Intersection Management– U Texas at Austin– Reservation system

Van der Horst, 1991

Autonomous Intersection (Top down)

Autonomous Intersection (Driver's View)

The Trouble With Levels

The evolution of vehicle automation and associated challenges

• Levels are not a roadmap• Levels are not design guidelines• Levels discourage potentially helpful ideas like

adaptive automation strategies

5 – 30 years until autonomous vehicles hit the road