computer science and information technology (i) connected

Computer Science and Information Technology (I) Connected and Autonomous Vehicles

Chung-Wei [email protected]

CSIE Department

National Taiwan University

Introduction to Myself

❑ B.S. Student➢ 2001.09--2005.06, CSIE Department, NTU

❑ M.S. Student➢ 2005.09--2007.06, GIEE (EDA Group), NTU

❑ Ph.D. Student➢ 2009.08--2015.08, EECS Department, UC Berkeley

• Some interesting teaching experience here

❑ Researcher➢ 2015.09--2018.07, Systems and Software Division, Toyota

InfoTechnology Center (Mountain View, CA)

❑ Assistant/Associate Professor➢ 2018.08--2021.07/2021.08--, CSIE Department, NTU

2

Outline

❑ What are connected and autonomous vehicles?

❑ What are the roles of software people?

❑ A closer look at connected and autonomous vehicles

❑ Some specific problems

3

Video (20170822-2)

Intelligent Vehicles

❑ Connected and Autonomous Vehicles (CAV)➢ A good application may need both of "connectivity" and "autonomy"

➢ What if the intersection management does not have connectivity?

➢ What if the intersection management does not have autonomy?

4

Intersection Management

Levels of Autonomy

❑ Levels of driving automation [SAE J3016 / Wikipedia]➢ Level 1 (Drive Assistance): Adaptive Cruise Control (ACC), Parking

Assistance, etc.

➢ Level 2 (Partial Automation)

• The vehicle takes full control of accelerating, braking, and steering

• The driver must monitor the driving

➢ Level 3 (Conditional Automation)

• The driver can safely turn his/her attention away from the driving tasks

• The driver must be prepared to intervene, when called upon by the vehicle

➢ Level 4 (High Automation)

• No driver attention is ever required for safety

• Self driving is supported only in limited spatial areas or under special circumstances

➢ Level 5 (Full Automation)

5

Levels of Autonomy

6

https://www.birmingham.ac.uk/news/thebirminghambrief/items/2016/11/driving-the-revolution.aspx

Connected Applications

❑ Communication standards➢ Dedicated Short-Range Communications (DSRC)

➢ C-V2X (Cellular Vehicle-to-Everything)

7

!

IntersectionManagement

Emergency Vehicle Warning

Side Road MergingSharp Curve Assistant

Cooperative Adaptive Cruise Control

Video (20160325-0)

Emergency Vehicle Warning (1/2)

8

Video (20160324-1)

Video (20160324-0)

Emergency Vehicle Warning (2/2)

9

Outline





10

Software Design

❑ Various applications including Advanced Driver AssistanceSystems (ADAS) and autonomous functions

❑ Various software programs forsensing, signal processing, control,decision making, etc.➢ Values to vehicle's total value

• Embedded software: 2% → 13% from 2000 to 2010

• Electronics system: expected to be 50% in 2030

➢ Number of lines of code

• 1 → 10+ → 100 million from 2000 → 2010 → 2020

❑ Due to the safety-critical nature,correctness and quality ofsoftware are extremely important

11http://www.toyota.com/safety-sense/

Pre-Collision System (PCS)

Lane Departure Alert (LDA)

Automatic High Beams (AHB)

Pre-Collision System with Pedestrian Detection Function (PCS w/PD)

Dynamic Radar Cruise Control (DRCC)

Hardware Design

❑ Number of Electronic Control Units (ECUs)➢ 20 → 50+ in the past decade

❑ Integrated architecture➢ One function can be distributed over multiple ECUs, and multiple

functions can be supported by one ECU

• More sharing and contention among software functions

• Traditional federated architecture: each function is deployed to one ECU and provided as a black-box by its supplier

❑ New computational components➢ Field Programmable Gate Array (FPGA)

➢ Graphical Processing Unit (GPU)

❑ Next-generation communication protocols➢ Ethernet-based protocols

12

https://en.wikipedia.org/wiki/Ethernet

Design-Time vs. Runtime

❑ Runtime software is executed during vehicle operation➢ It is usually designed during design-time

➢ Examples of runtime software / functions

• Localization algorithm

• Object detection algorithm

• Vehicle control algorithm

❑ Design-time software is executed during the design stage➢ It is more and more difficult to manually design a vehicle

➢ Examples of design-time software / tools

• Modeling

• Design including optimization

• Analysis including simulation, verification, and testing

13

Outline





14

Subsystems

❑ Body

❑ Chassis

❑ Suspension

❑ Control

❑ Engine

❑ Transmission

❑ Braking

❑ Wiring

❑ Electronics

❑ And many others

15

http://www.axiscades.com/automotive.html

Layered View of Autonomous Vehicles

16

Hardware Platform, e.g., Electronic Control Unit (ECU)

(Operating System)

GPS

Camera

Lidar

Radar

Sensing

Localization

ObjectTracking

Object Detection

Segmentation

Perception

Prediction

MotionPlanning

BehaviorPlanning

RoutePlanning

Planningand Decision

EnergyManagement

EmissionControl

TorqueControl

SteeringControl

Control

Light

Wheel

Brake

Engine

Actuation

Electronic Control Unit (ECU)

❑ Is an ECU like a Central Processing Unit (CPU)?➢ Yes? It does some computation

➢ No? It is not centralized

17

http://www.denso.co.id/Electro_ecu.html

https://www.indiamart.com/proddetail/denso-engine-electronic-control-unit-18672409991.html

Layered View of Connected Vehicles

❑ From multiple vehicles to components in a single vehicle

18

ExternalNetwork

(( ))

https://slideplayer.com/slide/10709648/

CAN: Controller Area NetworkLIN: Local Interconnect NetworkMOST: Media-Oriented Systems Transport

Discussion

❑ When will autonomous driving become reality?➢ Technology

➢ Cost

➢ Law and regulation

➢ Human comfort

➢ Philosophy

19

Video (20170214)

Homework

❑ Question 1➢ List and explain (e.g., 3 sentences)

• Three benefits of connected vehicles and/or autonomous vehicles?

• Three concerns of connected vehicles and/or autonomous vehicles?

❑ Question 2➢ How to measure risk and safety?

➢ List two reasons that people may not choose the transportation method with the lowest risk (defined by themselves) to travel

❑ Visit NTU COOL for details or questions

20

Outline





21

Mapping from SW to HW (1/2)

❑ Software (functional model): task graph

❑ Hardware (architectural platform): distributed Electronic Control Units (ECUs) connected by a network

22

ECU CAN bus

fun

ctio

nal

mo

de

l

τ3

σ1

μ1,1

μ1,2

τ2

τ1

τ6

τ5

τ4

τ8

τ7

σ2

σ3

σ4

σ5

σ6

ε2ε1

μ2,1

μ2,2

ε3

arch

itec

ture

pla

tfo

rm

task allocation signal packing

broadcast

task signal

message

Mapping from SW to HW (2/2)

❑ Decide task allocation, signal packing, and priority assignments (tasks on ECUs and messages on CAN bus)

❑ Satisfy timing constraints for tasks, signals, and paths

23

ECU CAN bus

fun

ctio

nal

mo

de

l

τ3

σ1

μ1,1

μ1,2

τ2

τ1

τ6

τ5

τ4

τ8

τ7

σ2

σ3

σ4

σ5

σ6

ε2ε1

μ2,1

μ2,2

ε3

arch

itec

ture

pla

tfo

rm

task allocation signal packing

broadcast

task signal

message

Intersection Management (1/2)

❑ Every vehicle will periodically optimize its own moving path

❑ The conflict-resolution problem is reduced to a cycle-removal problem in a graph➢ If vehicles (even from different automotive companies) agree how to

remove a cycle, then "conflict-free" can be guaranteed

➢ Cycle-removal algorithm is the"agreement" between vehicles(e.g., four-way stop sign)

24

Cycle-Removal

Vehicles will go through the conflict zone (e.g., intersection) with the order in this graph

Intersection Management (2/2)

❑ The payment-based approach supports prioritized intersection management where truthfulness is guaranteed

❑ An intersection becomes "more expensive" when there are more vehicles requesting the intersection

25

Video (20170825)

Cooperative Adaptive Cruise Control (CACC)

❑ Two simplified CACC modes➢ Gap control mode

• The following vehicle (F) decidesacceleration based on the gap, speeds,and accelerations of the two vehicles

➢ Collision avoidance mode

• The following vehicle (F) decelerates with its maximum deceleration

❑ Information sources➢ Gap and speeds are obtained by sensors

➢ Accelerations are broadcasted with V2X messages

26

PFGap

CollisionAvoidance

Mode

GapControlMode

Gap is larger than gsafe

Gap is smaller than gsafe

gsafe: gap threshold

Video (20161021)

Jamming of Wireless Network (1/2)

27

Jamming of Wireless Network (2/2)

❑ An intersection manager receives requests from vehicles, schedule them, and sends confirmations to them

28

Video (20170203-1) Video (20170203-3)

Video (20170203-2) Video (20170203-4)

SomeMitigation

MoreMitigation

Without Jamming

WithJamming

Plug-and-Play Systems

❑ What if a vehicle is like a smart cellphone that applications can be downloaded, updated, and activated?➢ At dealership (customized vehicles)

➢ Before driving or even during driving?

29

Order withBSM+LKA

BSM+LKA

Configure LKA(at Dealership)

LKA

[Dealership]

[Factory] Order

[Dealership]

[User]

[Factory]

Deliver to DealershipDeliver to User Deliver to DealershipDeliver to User

[User]

BSM+LKA?

LKA? LKA?

OK

Lane Keeping Assistance (LKA)Blind Spot Monitor (BSM)

Edge Computing (1/2)

30From Automotive Edge Computing Consortium

Cloud

Data DataData

V2C2VIntelligent

Driving

High Definition

Map

too much traffic

too muchdata processing

Edge Computing (2/2)

31From Automotive Edge Computing Consortium

Cloud

Data DataData

LocalData

LocalData

LocalData

LocalData

V2C2VIntelligent

Driving

High Definition

MapV2C2V

IntelligentDriving

High Definition

Map

Edges

Software Integrity (1/3)

❑ ISO 26262 is recognized as the state-of-the-art standard for automotive systems

❑ Examples➢ Some software structures are NOT recommended for highest Safety

Integrity Level (SIL)

• Dynamic objects and variables

• Multiple uses of variable names

• Implicit type conversions

• Unconditional jumps

• Recursions

32

Management

DevelopmentSystem Level

Hardware LevelSoftware Level P

rod

uct

ion

& O

pe

rati

on

Co

nce

pt

Ph

ase

Supporting Processes


❑ A potential conflict between Original Equipment Manufacturers (OEMs) and software suppliers (developers)➢ A certification process represents a systematic way to inspect the

source codes

➢ Some source codes of software suppliers (developers) are confidential

❑ Desired properties➢ Authenticity

• Only authenticated results from compilers and analysis tools (verification, simulation, and/or testing) are considered by the certification issuers

➢ Confidentiality

• Sensitive source codes of the software suppliers and developers are not released to certification issuers

33


❑ Trusted third-party

➢ Run a certification program which consists of a compiler and an analyzer

➢ Maintain a router which controls the input and the output

❑ Certification program

➢ All of the compiler, the analyzer, and the private key are updated by the OEM

➢ The updating process must be unidirectional to guarantee confidentiality

❑ Router

➢ Only the corresponding developer can be the receiver

34

Trusted 3rd-PartyCertification Program (Installed or Updated by OEM)

Compiler

Executable

Analyzer

Log forDeveloper

Log forOEM

Signing Signature

Router

Public andPrivate Key

Developer

Censoring

Public Key

Supplier

SignatureChecking

Public Key

OEM

Public andPrivate Key

SignatureChecking

SourceCodes

Physically Located at theTrusted 3rd-Party's Site

Wire Routing

❑ The wiring weight of a system can be up to 30kg➢ The third heaviest and costliest component in an automotive system

(after the chassis and the engine)

➢ Netlist

• A set of "parts" to beconnected

➢ Splice

• Used for connectingmore than two wires

• Steiner vertex!

➢ Where to put splices?

• Steiner tree problem

35

Intrusion Detection

36

Physics RulesPrincipal Component

AnalysisHidden Markov Model

FeaturesSimple and Quick

(No Training Needed)Catch Implicit Relationships

Catch Time-Series Data

In-Car Applicable Complexity Concern Complexity Concern

Edge(Roadside Unit)

Applicable Applicable Applicable

Cloud Latency Concern Latency Concern Latency Concern

Attack A1(Lie about Acceleration)

Detected Detected Detected

Attack A2(Lie about Velocity)

Not Detected Detected Detected

Attack A3(Lie about Position)

Not Detected Not Detected Detected

Attack A4(Lie about Velocity + Position)

Not Detected Not Detected Detected

Consensus Algorithms

❑ Vehicles, road side units, edge servers, and cloud servers may have different opinions➢ Intrusion detection

➢ Dynamic map creation

➢ Event report checking

• Examples: location, speed,and acceleration of a vehicle

❑ Challenges➢ If A says that B is wrong, is

A or B actually wrong?

➢ Timing-critical information

➢ Vehicles are moving

37

Adversary Classification

❑ Adversary classification

❑ How if we add barcodes to traffic signs?

❑ A

38

[CVPR '18]

MachineLearningClassifier

There is nostop sign!

➢ "Selecting the battlefield strategically against an adversary is just likein the Battle of Thermopylae. King Leonidas of Sparta and his 300soldiers could face Persian Empire of Xerxes and his far larger armylegendarily at the strategically selected narrow coastal pass ofThermopylae, where the numerical imbalance did not make a hugedifference."

Q&A

39

Course Roadmap of UC Berkeley CS

70

61A

61B 61C

184 186188

189

169160 162 164 161

168

170

191174 176172

150

152

core

hardware

software

theory

applications

required

recommended

CS70: Discrete Mathematics and Probability Theory

Topics in CS70

❑ Propositions and Quantifiers

❑ Proofs

❑ Induction

❑ Stable Marriage

❑ Modular Arithmetic

❑ Bijections and RSA

❑ Polynomials

❑ Error Correcting Codes

❑ Counting

❑ Introduction to Discrete Probability

41

❑ Conditional Probability

❑ Random Variables

❑ Variance

❑ Two Killer Applications

❑ Chebyshev's Inequality

❑ Some Important Distributions

❑ Continuous Probability

❑ Graph Theory

❑ Infinity and Uncountability

❑ Self-Reference andUncomputability

Teaching Experience at Berkeley

❑ CS70: Discrete Mathematics and Probability Theory➢ Spring 2013, Spring 2014: teaching assistant

➢ Summer 2015: instructor (http://www-inst.eecs.berkeley.edu/~cs70/su15/)

• 300+ students on the first day

• The total tuition is about $450,000

• 11 teaching assistants --- all undergraduates

• 14 readers (to grade homework) --- all undergraduates

❑ Background➢ Some students want to meet the GPA requirement to "declare" (i.e.,

major) CS

➢ The class GPA is (was) expected to be 2.5--2.9 (3 is B and 2.7 is B-)

❑ Instructor's evaluation from students will be posted publicly

42

http://www-inst.eecs.berkeley.edu/~cs70/su15/

Weekly Schedule (8-Week Long)

43

How-To

❑ Answer students' questions➢ Homework parties

➢ Online forum

• Participation score

❑ Synchronize TAs and readers➢ Two additional forums

❑ Grade homework and exams➢ Gradescope

❑ Proctor exams (3 in 8 weeks)➢ Separated rooms

➢ Timing conflicts

➢ Special cases

44

Observations

❑ Are/do students at Berkeley…➢ Very smart?

• Pick 3 cards from 52 cards

– Sampling without replacement, order does matter

– Sampling without replacement, order does not matter

– Sampling with replacement, order does matter

– Sampling with replacement, order does not matter

➢ Study very hard?

➢ Love to ask questions?

➢ Care about grade?

• Regrading policy

➢ Have a lot of pressure?

• Special office hour

❑ How about TAs?

❑ And …45

Q&A

46

computer science and information technology (i) connected

Documents