Fei Miao Postdoc, Electrical and Systems Engineering

University of Pennsylvania

Data-Driven Dynamic Robust Resource Allocation for Efficient Transportation

1

Cyber-Physical Systems Tightintegra+onofcommunica+onandcomputa+on

systemswithcontrolofphysicalworld

BuildingAutoma+on

IndustrialAutoma+onandAdvancedManufacturing

Transporta+on&Autonomousvehicles SmartCi+es

MedicalDevicesandSystems

SmartGrids

ScienceofCPS

FeiMiao 2

Internet of Things

FeiMiao 3

A Growing Field : CPS

2010

2013

FeiMiao 3

Smart Cities

FeiMiao 5

Challenges of CPS

Automo+veAgriculture

Civil

Aeronau+cs

Materials

Energy

Manufacturing

Smart&ConnectedCommuni+es

CPSCore

Safety

Verifica-on

Privacy

HumanintheLoop

Control

IoT

NetworkingDataAnaly-cs

DesignAutonomy

Medical

Security

Informa-onManagement

Real--meSystems

5

Applica+onSectors

Data-Driven Control, Optimization of CPS

Growing complexity and dynamics: scale, uncertainty, heterogeneous…

FeiMiao 7

Data to information Machine learning Statistical method

Minimize cost or Maximize profit Optimization

Information to Action Control

Application Transportation systems Energy networks Manufacturing …....

Demand response of smart grid

Energy dispatch of smart building

Intelligent transportation systems: integration research challenges unsolved

Domain: Transportation Systems

FeiMiao 8

SmartParking

9.4 9.7 10.01 10.33 10.66 11

0 2 4 6 8

10 12

2009 2010 2011 2012 2013 2014

Revenue of US taxi services increases

Self-drivingcar

State of the Art – The Dark Side…

=20 ×

Cruising Mileage

Idle: 300 million miles/year

New York City

FeiMiao 9

Previous Work

•  Urban traffic, demand modeling –  Spatial and temporal patterns in speed prediction: [P. Jaillet et al., 2014] –  Infer traffic condition based on taxi trip dataset: [Work et al., 2015]

•  Coordination and resource allocation in smart transportation –  Smart parking: [Geng and Cassandras, 2013] –  Bike redistribution and incentives in sharing:[Morari et.al, 2014] –  Mobility-on-demand systems: autonomous vehicles/robotics Minimize re-balancing number; with future demand : [Pavone et.al, 2014, 2015] Dynamic on-demand ride-sharing [Frazzoli, et al., 2016] Evaluation metric: number of vehicles needed; simulated waiting time

10FeiMiao

Novelty of my work Brings data-driven optimization to system-wide efficient transportation

Motivation: Efficient Transportation

On demand ridesharing service •  Greedy, increase human satisfaction myopically

•  Optimal vehicle allocating with accurate, known demand distributions

FeiMiao 11

No system-level optimal, proactive resource allocation DataàPredicted demand àImproved system performance?

High Demand

Low Supply

Problem and Goal

Real-Time Control

Pickup & Delivery

Vehicle Mobility

Predicted Demand

Storage/Dispatch Center

Cellular Ratio

GPS

Occupancy status

12FeiMiao

System-level balanced supply-demand ratio (fair service) with least total idle distance

Local controller: heuristic, greedy, matching, etc. Hierarchical

Challenges: scale, uncertainty Conflict of interests Limited resource Customers: reach destination soon System optimal: minimum idle

Contributions Outline: Data-Driven Control, Optimization of CPS

Large-scale dynamic decision, receding horizon control Demand predicted from historical and real-time sensing data

Uncertain demand: computationally tractable, probabilistic cost guarantee 1.Robust optimization for the worst-case resource allocation cost 2.Distributionally robust optimization for the expected cost

13FeiMiao

Contributions Outline: Data-Driven Control, Optimization

•  Reduce idle mileage of ride-sharing service •  Considering both current and predicted future demand

14FeiMiao

Large-scale dynamic decision, receding horizon control Demand predicted from historical and real-time sensing data

Receding Horizon Control Hierarchical Vehicle Dispatch

Spatial-temporal data à demand, mobility e.g. Poisson, vector time series

Dynamically optimize: time t, consider dispatch costs of (t,…, t+τ), execute decision for t 1.Update sensing data, demand prediction 2. System-level: balanced supply towards demand with minimum total idle distance 3.Send decisions to vehicles, local dispatcher t = t+1

15FeiMiao

Spatial Urban Regions

Temporal Hourly Windows

One region, local dispatcher

n regions, optimization

Objective #1: Fair Service à Balanced Supply

16FeiMiao

rkiPredicted demand�

Decision variables : number of vacant vehicles from region i to j Xkij > 0

Supply �original number of vacant vehicles before dispatch (GPS): Lik

•  Goal: Fair service, similar average waiting time

•  Local supply/demand ratio close to global level: penalty difference

Local s/d ratio Global s/d ratio (city)

Region 1 r1 L1

X23

X13 X24

Wij=Wji

Region 2 r2 L2

Region 3 r3 L3

Region 4 r4 L4

JE =⌧X

k=1

nX

i=1

��������

nPj=1

Xkji �

nPj=1

Xkij + Lk

i

rki� Nk

nPj=1

rkj

��������

Objective #2: Minimum Cost: Total Idle Distance

•  Total idle distance to meet demand

4/6/15, 3:32 PMCable Car Museum, Mason Street, San Francisco, CA to Montgomery St. - Google Maps

Page 1 of 2https://www.google.com/maps/dir/Cable+Car+Museum,+Mason+Street,+…ff7491:0xb04336aed684aa9f!2m2!1d-122.401407!2d37.789256!3e0!5i2

Directions from Cable Car Museum, Mason Street, San Francisco, CA to Montgomery St.

Drive 1.0 mile, 5 min

Cable Car Museum, Mason Street, San Francisco, CA

1. Head south on Mason St toward Washington St

2. Turn left at the 1st cross street onto Washington St

3. Turn right onto Powell St

4. Turn left onto Clay St

5. Turn right onto Montgomery St

6. Turn left onto Market St

Montgomery St.

108 ft

482 ft

328 ft

0.4 mi

0.4 mi

223 ft

Initial position

Dispatched position

17FeiMiao

•  The distance between region i and region j at time k:

JD =⌧P

k=1

nPi=1

nPj=1

XkijW

kij

W kij

Region 1 r1 L1

X23

X13 X24

Wij=Wji

Region 2 r2 L2

Region 3 r3 L3

Region 4 r4 L4

Decision variables : number of vacant vehicles from region i to j Xkij > 0

System-Level Optimal Vehicle Dispatch

(1)

18FeiMiao

Balanced supply Idle distance to meet demand

Supply positive

State dynamics: trip

Sparsity constraint (distance limited)

Computational complexity: polynomial of variable number (τn2) Spatial-temporal granularity

min.X1:⌧ ,L2:⌧

J =⌧X

k=1

(JD(Xk) + �JE(Xk, rk))

=⌧X

k=1

nX

i=1

0

BB@nX

j=1

XkijWij + �

��������

nPj=1

Xkji �

nPj=1

Xkij + Lk

i

rki� Nk

nPj=1

rkj

��������

1

CCA

s.t. (Lk+1)T = (1TnX

k � (Xk1n)T + (Lk)T )P k,

1TnX

k � (Xk1n)T + (Lk)T > 0,

XkijW

kij 6 mkXk

ij ,

Xkij > 0

Experiment: RHC dispatch with Real-Time information

2 4 6 8 10 12 14 16 18 20 22 240

120

240

360

480

600

Time: Hour

Aver

age

tota

l idle

dist

ance

: mile

Idle distance comparison

Dispatch with RT informationDispatch without RT informationNo dispatch

19FeiMiao

•  System-level vehicle balancing and local shortest path dispatcher •  Average demand, GPS vehicle locations Average idle distance 42%

Collection Period 28 days

Number of Taxis 500

GPS Record Number

1,000,000

Experiment: Dispatch with Real-Time Information

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 160.5

0.8

1.1

1.4

1.7

2.0

Region ID

Supp

ly d

eman

d ra

tio

Supply demand ratio under different conditions

Global supply demand ratioDispatch without RT informationDispatch with RT informationNo dispatch

20FeiMiao

Collection Period 28 days

Number of Taxis 500

Record Number 1,000,000

•  System-level vehicle balancing and local shortest path dispatcher •  Average demand, GPS vehicle locations Average supply-demand ratio error 45%

Large error caused by prediction error

Contributions Outline: Data-Driven Control, Optimization of CPS

Uncertain demand: computationally tractable, probabilistic cost guarantee 1.Robust optimization for the worst-case resource allocation cost 2.Distributionally robust optimization for the expected cost

21FeiMiao

FeiMiao 22

•  Bootstrap (repeated experiments) •  Confidence region of H0:mean,

covariance, probability distribution •  Closed and convex uncertainty sets

Spatial-temporal data •  Partition city map, cluster dataset •  Trip/trajectory→ Aggregated demand

Data to Uncertain Demand or Demand Distributions

All dataWeekday

*H. David and H. Nagaraja. Order statistics. Wiley Online Library, 1970. �D. Bertsimas et al, Data-driven robust optimization. Operations Research 2015.

Box:

Second-order-cone (SOC):

Distribution set

rk+1 = fr(I[k�l,k]), rk+1 = rk+1 + �k+1.

rk+1 = fr(I[k�l,k]), rk+1 = rk+1 + �k+1.

Vector time series Poisson distribution

Uncertain Demand: Computationally Tractable Approximation

23FeiMiao

•  Objective not computationally intractable under uncertain demand

•  Predicted demand �a closed and convex set (demand at time k: rk ) rc = (r1, r2, . . . , r⌧ ) 2 � 2 R⌧n

Region 1 R1 S1

X23

X13 X24

Wij=Wji

Region 2 R2 S2

Region 3 R3 S3

Region 4 R4 S4

•  Decision variables : number of vacant taxis from region i to j Xkij > 0

supply at region i time k, linear of previous decisions Ski = f(X1:k)

JE =⌧X

k=1

nX

i=1

��������

Ski

rki�

nPj=1

Skj

nPj=1

rkj

��������

Uncertain Demand: Computationally Tractable Approximation

24FeiMiao

*Theorem: computationally tractable approximation (concave of demand) *Fei Miao et. al, CDC, 2015; Fei Miao et. al, under revision, IEEE TCST 2016

•  Predicted demand �a closed and convex set (demand at time k: rk )

rc = (r1, r2, . . . , r⌧ ) 2 � 2 R⌧n

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 11 0

5

10

15mi

smatc

h valu

e Demand and supply mismatch value change with

•  Decision variables : number of vacant vehicles from region i to j Xkij > 0

supply at region i time k, linear of previous decisions Ski = f(X1:k)

min.Xk

nPi=1

rki(Sk

i )↵

⌧Pk=1

nPi=1

������rkiSki�

nPj=1

rkj

nPj=1

Skj

������9↵ > 0

JE =⌧X

k=1

nX

i=1

��������

Ski

rki�

nPj=1

Skj

nPj=1

rkj

��������

�

Probabilistic guarantee

•  Predicted demand �a closed and convex set (demand at time k: rk )

rc = (r1, r2, . . . , r⌧ ) 2 � 2 R⌧n

min.X1:⌧

J(X1:⌧ , rc)

s.t. Prc⇠P⇤(rc)(f(X1:⌧ , rc) 6 0) > 1� ✏.

min.

X1:⌧max

rc⇠�J(X1:⌧ , rc),

s.t. f(X1:⌧ , rc) 6 0,

Robust optimization problem

Probabilistic Cost Guarantee for Worst-Case by Robust Solutions

25FeiMiao

•  Probabilistic guarantee for the dispatch cost

*Theorems: robust optimization à Equivalent convex optimization 1. Approximated objective: concave of uncertainties, convex of variables 2. Closed and convex uncertainty set (related toε,first/second order) *Fei Miao et. al, CDC, 2015; Fei Miao et. al, under revision, IEEE TCST 2016

Minimize Expected Cost: Distributionally Robust Optimization (DRO)

•  Motivation: Trade-off between the expected and the worst-case costs

•  Formulation

FeiMiao 26

X1:⌧ = {X1, X2, . . . X⌧}rc ⇠ F ⇤, F ⇤ 2 FDemand: ,

Resource allocation decision (spatial-temporal): rc = (r1, r2, . . . , r⌧ ) 2 R⌧n

Stochastic programming min.X1:⌧

Erc⇠F⇤⇥J(X1:⌧ , rc)

⇤

s.t. X1:⌧ 2 Dc.

Distributionally robust optimization min.

X1:⌧max

F2FE⇥J(X1:⌧ , rc)

⇤

s.t. X1:⌧ 2 Dc.

*Theorem: DRO à Equivalent convex optimization 1.Approximated objective: concave of uncertainties, convex of variables 2.Closed and convex set of probability distributions (first/second order) *Fei Miao et. al, accepted, CPSWeek ICCPS 2017.

Applications of Data-Driven DRO Resource Allocation

27FeiMiao

supply at region i time k, linear of previous decisions

Resource allocation under demand uncertainties •  Autonomous vehicle balancing for mobility-on-demand system •  Bicycle balancing: supply-demand ratio at each station is in a range •  Hierarchical carpooling framework: global balance, local carpool •  Real-time demand response with limited resource

min.

X1:⌧ ,S1:⌧max

F2FE"

⌧X

k=1

JD(Xk

) + �nX

i=1

rki(Sk

i )↵

!#

s.t. X1:⌧ , S1:⌧ 2 Dc

Metric of service quality (d/s ratio related) Allocation cost

Ski = f(X1:k)

(3)

Evaluations: Robust VS Non-Robust Solutions

0 60 120 180 2400

20%

40%

60%

Demand supply ratio error range

Perc

enta

ge

Demand supply ratio error comparison

Robust solutionsNon−robust solutions

FeiMiao 28

Cross-validation Second-order-cone uncertainties Probabilistic guarantee 1-ε=0.75 The average demand supply ratio error reduced by 31.7%

The average total idle distance Robust VS non-robust: ê10.13%

0.7 0.8 0.9 1 1.1 1.2 1.3x 105

0

25%

50%

75%

Total idle distance range

Perc

enta

ge

Total idle distance comparison

Robust solutionsNon−robust solutions

Collection Period 4 years Data Size 100 GB Trip number 700 million

Evaluation: Average Costs of (distributionally) Robust Solutions

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 117

8

9

10x 104 Average cost comparison of DRO and RO

ϵ

Aver

age

cost

SOCBoxNRDRO

FeiMiao 29

SOC: second-order-cone uncertainty set Box: range of demand at each region Compared with non-robust (NR) solutions (100GB NYC taxi data) The average total idle driving distance is reduced by 10.05%

Dynamic Region Partition with DRO to Reduce Cost

FeiMiao 30

•  [Quad-tree region partition, uncertainty set of demand probability distributions, distributionally robust vehicle balancing] VS [static grid, non-robust model]

à  total idle distance 60 million miles, 8 million dollars gas consumption/year •  Compared with total idle distance of original data: 55%

Region Division Grid Idle Mile Quad-Tree Idle Mile Change Rate t=1 hour 13.1%

T=30 minutes 20.0%

7.63⇥ 104 6.62⇥ 104

6.84⇥ 104 5.47⇥ 104

Summary of Contributions in Data-Driven CPS

•  Learn demand from data; real-time, hierarchical decision-making

•  Transportation efficiency Fei Miao et.al, ICCPS15 best paper finalist

31FeiMiao

Computationally tractable, probabilistic guarantee under model uncertainties

Fei Miao et.al, ICCPS17;CDC15,TCST16

Automo+ve Agriculture

Civil

Aeronau+cs

Materials

Energy

Manufacturing

Smart&ConnectedCommuni+e

s

CPSCore

Safety

Verifica-on

Privacy

HumanintheLoop

Control

IoT

NetworkingDataAnaly-c

s

DesignAutonomy

Medical

Security

Informa-onManagement

Real--meSystems

Challenges with growing complexity and dynamics: scale, uncertainty

Future work of CPS Safety and Security

Automo+veAgriculture

Civil

Aeronau+cs

Materials

Energy

Manufacturing

Smart&ConnectedCommuni+es

CPSCore

Safety

Verifica-on

Privacy

HumanintheLoop

Control

IoT

NetworkingDataAnaly-cs

DesignAutonomy

Medical

Security

Informa-onManagement

Real--meSystems

33FeiMiao

Applica+onSectors

Challenges: Safety, Security and Resilience

•  With integration of communication, computation and control, cyber attacks can cause disasters in the physical world

•  Knowledge of physical system dynamics helpful for security, resilience

33FeiMiao Connected vehicles

CPS Security: Coding for Stealthy Data Injection Attacks Detection

•  Problem: Stealthy Data Injection Attacks -Attacker is smarter with the system model knowledge: inject data to communication channel, drive system to unstable state and pass detectors.

-Communication cost for encrypted messages is too large

•  Goal: A low cost technique to detect stealthy data injection attacks

Estimator

Detector

Controlleruk

uk-1

.

.

.yk=Cxk+vk

Xk+1=Axk+Buk+wk

LTI S1

S2

Sp

a1

am

Attacker

change yk

to yk'

yk/yk'

.

.

.

yak

y'k=Cxk+yak+vk

a2

FeiMiao 35

Coding Schemes for Stealthy Data Injection Attacks Detection

Low cost: no extra bytes to communicate after coding *Research Contributions -Analyze sufficient conditions of feasible, low cost coding -Design an algorithm to calculate a feasible Σin real-time -Time-varying coding when the attacker can estimate Σ

*Fei Miao et.al, TCNS 2016. (funded by DARPA)

yk = Cxk + vk

Yk = ⌃(Cxk + vk)

Y

0k = ⌃(Cxk + vk) + y

ak

Estimator decode y0k = ⌃�1Y 0

k = yk + ⌃�1yak

FeiMiao 36

Dynamic Stochastic Game for Resilient Systems

36

Linear time invariant system

FeiMiao

•  Problem: When an attack happens? What type of attack? Not known! Higher attack detection rate, higher investment cost in security in general *Contributions: dynamic stochastic game for an optimal switching policy between subsystems to balance security overhead and control cost

*Fei Miao et.al, CDC 2013,2014; journal version submitted to Automatica (funded by DARPA)

New York�37

•  Agenda: safety, efficiency, security for CPSs with focus on smart cities, autonomous transportation systems

•  Contributions: data-driven CPSs, CPSs/Smart Cities security

•  Future work -Hierarchical decision making based on heterogeneous data information

-Design incentive mechanisms (e.g., dynamic pricing) of users and suppliers for social optimal behavior -Safety assurance of coordinated control of connected autonomous vehicles

-Security and resiliency of smart cities infrastructure with physical dynamics knowledge, distributed sensor networks