Acknowledgements
• Mid-America Transportation Center– 1 year project to survey literature and report on state of
the art in autonomous vehicles– Co-PI: Prof. Geb Thomas– Undergraduate students
• Kory Nelson• Michael McCrary• Mathew Powell• Nicholas Schlarmann
– http://matc.unl.edu/research/research_projects.php?researchID=405– https://www.zotero.org/groups/autonomous_vehicles/items
Why Autonomous Vehicles?
• Safety– 32,000 people killed each year, 93% due to driver error,
billions in property damage– Autonomous vision is ‘crashless’
• Mobility– Safely increase traffic density (x2)-(x3)– Greater access for elderly, disabled, etc.
• Sustainability– Fuel savings due to platooning (20%), eliminating traffic
jams, reducing trip times, reducing ownership, reducing parking spaces
An early experiment on automatic highways was conducted by RCA and the state of Nebraska on a 400 foot strip of public highway just outside Lincoln (“Electronic Highway of the Future - Science Digest (Apr, 1958)” 2013)
A G
P M
CMU NAVLAB
• RALPH, ALVINN, YARF• In 1995, RALPH drove NAVLAB 5 over 3000 miles
from Pittsburgh to Washington, DC.– Steered autonomously 96% of the way from
Pittsburgh, PA to Washington DCPomerleau, 1995, RALPH: Rapidly Adapting Lateral Position Handler, IEEE Symposium on Intelligent Vehicles, September, 1995
A G
P M
National Automated Highway System
A demonstration of the automated highway system in San Diego (1997). University of California PATH Program
1994-1997
A G
P M
Intelligent Vehicle Initiative
• Prevent driver distraction• Facilitate accelerated
deployment of crash avoidance systems– Normal conditions
• IVIS– Degraded condition
• Visibility, drowsiness– Imminent crash
• Rear end, lane depart, intersection, ESC
1997-2005
Multiple ADAS system. Image from IVBSS materials, courtesy of UMTRI
Forward Crash Warning (FCW)
Lateral Drift Warning (LDW)
Lane-change/Merge (LCM)
Curve speed Warning (CSW)
RadarVision
A G
P M
DARPA Grand Challenge
Grand Challenge:2004 – no winner2005 – Stanley (Stanford)
Urban Grand Challenge2007 – Boss (CMU) A G
P M
Connected Vehicles
• DSRC (5.9 GHz)– Allocated in 2004
• Goals– Safety
• Forward collision, intersection movement assist, lane change, blind spot, do not pass, control loss warning, emergency brake light warning
– Mobility– Sustainability
• AERIS
2004-present
VII -> IntelliDrive -> Connected Vehicles
Regulatory decision from NHTSA recently announced. V2V will eventually be required in new cars. A G
P M
NHTSA Automation Program
• Licensing• Testing• Regulations
• Cybersecurity• Currently recommends
states only allow testing
2012-present
Level ExampleTransition Time to Manual (Heuristic)
0 – No Automation Warning only --
1 – Function-specific Automation ADAS < 1 second
2 – Combined Function Automation Super cruise < 1 minute
3 – Limited Self-Driving Automation Google car < 10 minutes
4 – Full Self-Driving Automation PRT --
NHTSA Levels of Automation
A G
P M
Future Societal Impacts
Light Cars: A Virtuous Cycle
Reduce mass
Downsize engine
Drivetrain brakes tires
Smaller fuel
supply
Autonomous Car Sharing
MIT’s Stackable City Car
Advanced Driver Assistance Systems ACC Pre-Crash LDWS
Sensor Year Sensor Year Sensor YearAudi Radar/Video 2011 Camera 2007
BMW Camera 2007Chrysler Laser 2006
Ford Radar 2009 Radar 2009 Camera 2010GM Radar 2004 Camera 2008
Honda Radar 2003 Camera 2003Kia Camera 2010
Jaguar Radar 1999 Lexus Laser 2001
Mercedes Radar 2001 Radar 2002 Camera 2009Nissan Camera 2001
Saab Radar 2002 Toyota Laser 1998 Radar 2003 Camera 2002
Volkswagen Radar/Video 2011 Volvo Radar 2002 Radar/Video 2007
A 2011 review of commercial ADAS systems compares manufacturers, model year, and sensor type for three types of systems (Shaout, Colella, and Awad 2011)
ADAS Automation
Abb. System Abb. SystemESC Electronic Stability Control DD Drowsiness DetectionFCW Forward Collision Warning AL Adaptive LightingACC Adaptive Cruise Control PM Pedal MisapplicationLDW Lane Departure Warning TSR Traffic Sign RecognitionLKA Lane Keeping Assist TJA Traffic Jam AssistantLCA Lane Change Assist CZA Construction Zone Assist
RCTA Rear Cross Traffic Alert PA Parking AssistantBSD Blind Spot Detection PP Parking PilotEBA Emergency Brake Assist HC Highway Chauffeur
AEBS Advanced Emergency Braking System HP Highway PilotESA Emergency Steer Assist
Personal Rapid Transit (PRT)
• Fully autonomous• No operator, no controls• Low speed• May use a guideway• Morgantown PRT
entered operation in 1975 in West Virginia
PRTs (cont.)
• Morgantown, WV• Masdar City (on hold)• London Heathrow
Airport• City Mobil 2• Suncheon, South Korea• Punjab, India
• Early criticisms of PRTs on guideways concern the scalability of the system
• But new concepts are leaving guideways behind, alleviating some of these concerns
Probabilistic Methods
• The world is messy with uneven edges, bad lighting, poorly marked roads, and unpredictable people
• Applications of probabilistic reasoning– Histogram filters (lane line tracking)– Particle filters, Kalman filters (object tracking)– Bayesian Networks (decision making)– Hidden Markov Models (state estimation)
Digital Maps & Mapping
• Digital maps negate the need to dynamically map the environment
• Simultaneous Localization & Mapping (SLAM) used to create environments in unmapped areas
• Many modern path planning algorithms are based on A* algorithm
• Must find the proper correspondence between the digital map and other sensor inputs
Testing & Certification
LogicSensor FailuresKalman FiltersFalse Positives
Histogram FiltersParticle Filters
Data FusionMore data (images & video)
More test cases
Path PlanningDecision Making
Digital MapsAll speeds
Parking LotsMany more tests
Transfer of Control
Transfer of Control to a Platoon
Level Example Transition Time to Manual
0 – No Automation Warning only --
1 – Function-specific Automation ADAS < 1 second
2 – Combined Function Automation Super cruise < 1 minute
3 – Limited Self-Driving Automation Google car < 10 minutes
4 – Full Self-Driving Automation PRT --
Example:
Legality
• “Automated vehicles are probably legal in the United States” – Bryant Walker Smith
• 1949 Geneva Convention on Road Traffic requires that the driver of a vehicle shall be at all times able to control it
• Who is liable: the driver or the manufacturer?• California, Nevada, and Florida have paved the
way with state laws for automated vehicles
Hacking Entry PointsEntry point Weakness
TelematicsThe benefit of such systems is that the car can be remotely disabled if stolen, or unlocked if the keys are inside. The weakness is that a hacker could potentially do the same.
MP3 malwareJust like software apps, MP3 files can also carry malware, especially if downloaded from unauthorized sites. These files can introduce the malware into a vehicles network if not walled off from safety-critical systems.
Infotainment apps
Car apps are like smartphone apps…they can carry viruses and malware. If the apps are not carefully screened, or if the car’s infotainment software is not securely walled off from other systems, then an attack can start with a simple app update.
BluetoothThe system that connects your smartphone to your car can be used as another entry point into the in-vehicle network.
OBD-IIThis port provides direct access to the CAN bus, and potentially every system of the car. If the CAN bus traffic is not encrypted, it is an obvious entry point to control a vehicle.
Door LocksLocks are interlinked with other vehicle data, such as speed and acceleration. If the network allows two-way communication, then a hacker could control the vehicle through the power locks.
Tire Pressure Monitoring System
Wireless TPMS systems could be hacked from adjacent vehicles, identify and track a vehicle through its unique sensor ID, and corrupt the sensor readings.
Key FobIt’s possible to extend the range of the key fob by an additional 30’ so that it could unlock a car door before the owner is close enough to prevent an unwanted entry.
Vehicle Networks to SecureNetwork Weakness
LIN Vulnerable at a single point of attack. Can put LIN slaves to sleep or make network inoperable
CAN Can jam the network with bogus high priority messages or disconnect controllers with bogus error messages
FlexRay Can send bogus error messages and sleep commands to disconnect or deactivate controllers
MOST Vulnerable to jamming attacksBluetooth Wireless networks are generally much more vulnerable to attack than wired
networks. Messages can be intercepted and modified, even introducing worms and viruses
Privacy
• Electronic Data Recorders (Black Box)• Identified network traffic• De-identified data– The myth of anonymity
• “Google’s self-driving car gathers almost 1 Gb per second” – Bill Gross, Idealab
Privacy By Design
• Proactive not reactive• Privacy by default• Privacy embedded into the design• Full functionality (positive sum, not zero sum)• End-to-end security (full lifecycle protection)• Visibility and transparency• Respect for user privacy
Case Study: Autonomous Intersectionsand Time to Collision Perception
• Time to Collision (TTC)– range / range rate
• Autonomous Intersection Management– U Texas at Austin– Reservation system
Van der Horst, 1991
Autonomous Intersection (Top down)
Autonomous Intersection (Driver's View)
The Trouble With Levels
The evolution of vehicle automation and associated challenges
• Levels are not a roadmap• Levels are not design guidelines• Levels discourage potentially helpful ideas like
adaptive automation strategies