tutorial on taffic management and ai

157
Tutorial: Traffic Management and AI Biplav Srivastava, Anand Ranganathan IBM Research AAAI 2012 Tutorial 22 nd July 2012 © 2012 IBM Corporation Toronto, Canada

Upload: biplav-srivastava

Post on 06-May-2015

488 views

Category:

Technology


3 download

DESCRIPTION

This is the tutorial on AI techniques for traffic management presented at the AAAI 2012 conference, Toronto, Canada presented by Biplav Srivastava and Anand Ranganathan.

TRANSCRIPT

Page 1: Tutorial on Taffic Management and AI

Tutorial: Traffic Management and AI

Biplav Srivastava, Anand RanganathanIBM Research

AAAI 2012 Tutorial

22nd July 2012

© 2012 IBM Corporation

Toronto, Canada

Page 2: Tutorial on Taffic Management and AI

What to Expect: Tutorial Objectives

� The aim of the tutorial is to make early and experienced researchers aware of the traffic management area, provide – an insightful overview of the current efforts using AI techniques

• in Research and • in practice (real world pilots), and

– whet interest for newer efforts on important open issues.

© 2010 IBM Corporation

� From the call for tutorial: “a second type of tutorial provides a broad overview for an AI area that potentially crosses boundaries with an interesting application area”.

� Disclaimer: we are only providing a sample of the traffic management space intended to

match audience profile in the available time.

2

Page 3: Tutorial on Taffic Management and AI

Outline

1. Traffic Management Problem

2. Instrumentation1. Sensing traffic2. Traffic state estimation3. Optimizing and combining sensor data

3. Interconnection1. Middleware 2. Traffic standards

© 2010 IBM Corporation

2. Traffic standards

4. Intelligence1. Path planning

1. Simple Illustration2. Path Planning for Individual Vehicles

2. End-user analytics1. Bus arrival prediction and journey planning, with state-of-art instrumentation2. Multi-modal journey planning, without sensors

5. Supporting topics1. Traffic Simulators2. Practical considerations for real-world pilots

3

Page 4: Tutorial on Taffic Management and AI

Acknowledgements

All our collaborators, and especially those in:

� City agencies around the world– Bengaluru, India– Boston, USA– Dublin, Ireland– Ho Chi Minh City, Vietnam– New Delhi, India– New York, USA

© 2010 IBM Corporation

– New York, USA– Stockholm, Sweden

� Academia

� IBM: Many including - Raj Gupta, Ullas Nambiar, Srikanth Tamilselvam, L V Subramaniam, Chai Wah Wu, Anand Paul, Milind Naphade, Jurij Paraszczak, Wei Sun, Laura Wynter, Olivier Verscheure, Eric Bouillet, Francesco Calabrese, Tsuyoshi Ide, Xuan Liu, Arun Hampapur, Nithya Rajamani, Vivek Tyagi, Raguram Krishnapuram, Shivkumar Kalyanraman, Manish Gupta, Niterndra Rajput, Krishna Kummamuru, Raymond Rudy, Brent Miller, Jane Xu, Steven Wysmuller, Alberto Giacomel, Vinod A Bijlani, Pankaj D Lunia, Tran Viet Huan, Wei Xiong Shang, Chen WC Wang, Bob Schloss, Rosario Usceda-Sosa, Anton Riabov, Magda Mourad

4

For discussions, ideas and contributions. Apologies to anyone unintentionally missed.

Page 5: Tutorial on Taffic Management and AI

Traffic Management and AI

Section: Traffic Problem

Speaker: Biplav Srivastava

AAAI 2012 Tutorial

July 2011

© 2012 IBM Corporation

Toronto, Canada

Page 6: Tutorial on Taffic Management and AI

Outline

1. Traffic Management Problem

2. Instrumentation1. Sensing traffic2. Traffic state estimation3. Optimizing and combining sensor data

3. Interconnection1. Middleware2. Traffic standards

© 2010 IBM Corporation

2. Traffic standards

4. Intelligence1. Path planning

1. Simple Illustration2. Path Planning for Individual Vehicles

2. End-user analytics1. Bus arrival prediction and journey planning, with state-of-art instrumentation2. Multi-modal journey planning, without sensors

5. Supporting topics1. Traffic Simulators2. Practical considerations for real-world pilots

6

Page 7: Tutorial on Taffic Management and AI

We All See Traffic Daily. An Illustration from Across the Globe

Characteristics New York

City, USA

New Delhi,

India

Beijing, China Moscow,

Russia

Ho Chi Minh City,

Vietnam

Sao Paolo, Brazil

1 How is traffic pre-dominantly managed

Automated control, manual

control

Manual control

Automated control, manual

control

Automated, manual control

Manual control Automated, manual control, Rotation

system (# plate based)

2 How is data collected Inductive loops, cops,

video, GPS

Traffic surveys, cops

Video, GPS, cops GPS, some video, cops

Traffic surveys, cops Video, GPS, cops

3 How can citizens manage their resources

GPS devices, alerts on radio,

web, road signs (variable)

Alerts on radio

alerts on radio, road signs

(variable), mobile alerts

GPS, radio, road signs,

mobile alerts

Alerts on radio GPS devices, alerts on radio, web

© 2010 IBM Corporation

Source: Google map for New York City and New Delhi; Search done on Aug 20, 2010

4 Traffic heterogeneity by vehicle types(Low: <10;

Medium 10-25; High: >25)

Low High Low Low Medium Low

5 Driving habit maturity (Low: <10 yrs; Medium:

10-20; High: > 20)

High Low Low Low Low Medium

6 Traffic movement Lane driving Chaotic Lane driving Lane driving Chaotic Lane Driving

Page 8: Tutorial on Taffic Management and AI

Is it Just Supply-Demand Mismatch?

� Supply – Roads and their capacity– Personnel available– Capital and operational budget

� Demands – Travel needs of citizens

© 2010 IBM Corporation

– Travel needs of citizens– Travel needs of organizations: businesses, governments

� Solution to mis-match?– Keep building supply (roads, bridges, …)– Keep reducing demand (restrict citizens, businesses, …)– May work for some of the cities, for some of the time,

• but not for all of the cities and all of the time

8

Page 9: Tutorial on Taffic Management and AI

Levers in Cities’ Hands for Traffic Management

� Physical infrastructure– New flyovers, roads– Expanding existing capacity– New metros, …

� Policies

© 2010 IBM Corporation

� Policies– Relocating businesses– Incentivising public transport

� IT enabled technologies– Intelligent Traffic/ Transportation Systems– Social networking

Page 10: Tutorial on Taffic Management and AI

Car (car +motorcycle) takes less time regardlessof distance unless

there is congestionon road.

(a) 3 km trip (b) 6 km trip

© 2010 IBM Corporation

Which Mode of Transport is Best for a Particular Distance – Door-to-Door Trip Times

See speaker notesfor more details.

Source: Mythologies, Metros & Future Urban Transport , by Prof. Dinesh Mohan, TRIPP, 2008

Data used:

• international data on things like access time to public transportation

was taken

•Studies were done for Delhi Metro

• Minimum assumptions were made.

•E.g., walking time for metro access (5 mins) is widely acceptable

•See speaker notes too

(c) 12 km trip (d) 24 km trip

Page 11: Tutorial on Taffic Management and AI

Why Focus on Mere Congestion Leads to Anomalies

� The Pigou-Knight-Downs paradox – States that adding extra road capacity to a road does not reduce travel time. – This occurs because traffic may simply shift to the upgraded road from the other, making the upgraded road more congested.

� The Downs-Thomson paradox – States that the equilibrium speed of car traffic on the road network is determined by the average door-to-door speed of equivalent journeys by public transport.

© 2010 IBM Corporation

public transport. – This occurs when the shift from public transport causes a disinvestment in the mode such that the operator either reduces frequency of service or raises fares to cover costs.

� The Braess' paradox – States that adding extra capacity to a network, when the moving entities selfishly choose their route, can in some cases reduce overall performance and increase the total commuting time.

11

Details:

Arnott, Richard and K.A. Small, 1994, “The Economics of Traffic Congestion,” American Scientist, Vol. 82, No. 5, pp. 446-455.

Chengri Ding and Shunfeng Song , Paradoxes of Traffic Flow and Congestion Pricing,

Page 12: Tutorial on Taffic Management and AI

Pigou-Knight-Downs Paradox

Highway

Bridge

� Bridge is the direct route between A-B, while highway is the circuitous route

� The average travel time on the (congested) bridge is a linear function of the flow-to-capacity ratio and the average travel time on the uncongested highway is a constant.

� a, b, and d are positive parameters with d > a; At equilibrium, travel times are same on both routes.

� Paradox exists for any bridge capacity less than C1, because expanding the bridge will only shift travelers to the bridge but not reduce travel time.

.

© 2010 IBM Corporation12

Details:

Arnott, Richard and K.A. Small, 1994, “The Economics of Traffic Congestion,” American Scientist, Vol. 82, No. 5, pp. 446-455.

Chengri Ding and Shunfeng Song , Paradoxes of Traffic Flow and Congestion Pricing,

� Using a=10, b=10, d=15, and F=1,000, Arnott and Small (1994) showed that the average travel time remains at 15 even if the bridge capacity continues to increase until it reaches 2,000, which is twice as large as the total traffic volume

� How to address paradoxes– Car commuters ignore how much delay they cause on other travelers but only pay attention to how long it takes them to commute.

– Delay cost to others must be quantified and accounted for as social cost.

– If social cost is introduced as tolls, the paradox goes away

+=

1

11

C

FbaT ; dT =2 ; FFF =+ 21 ;

21 TT = ( )adbF

C −= 11

Page 13: Tutorial on Taffic Management and AI

Key Insights for a New Traffic Problem Look

� Two category of stake-holders putting their resources– Public resources

• Example: $1M per month for a city with 5M population• Example: $100M available for roads and bridges for the next 3 years.

– Private resources (often ignored)• Example: Average time taken to travel 1 KM • Example: Average $ needed to travel 1 KM

© 2010 IBM Corporation

• Example: Average $ needed to travel 1 KM

� Impact of time-frame– In short-term, new physical supply cannot be created due to long construction time, time to hire personnel, etc

– Public resources have a short-term and a long-term cycle, while private resources for transportation need is time-frame insensitive

� Approach: optimize both public as well as private resources

13

Page 14: Tutorial on Taffic Management and AI

Problem Statement for Traffic Management from Engineering Perspective

� Problem (Short Term)Match traffic demand to supply with optimal usage of available public resources and

concomitant optimization of citizens’ private resources for travel needs

– Example: For a given day, • minimize over-time payments to traffic personnel , while • minimizing average commute time per km.

– Alternative: Service objectives can also be stated like average commute time per km be below 10 mins/ km.

© 2010 IBM Corporation

� Implications: Conflicting goals: optimality of public resources (e.g., road, traffic cops) v/s optimality of individual’s resources (e.g., travel time)

14

Details: Biplav Srivastava. A new look at the traffic management problem and where to start. In 18th ITS Congress, Orlando, USA, Oct 16-20, 2011.

Page 15: Tutorial on Taffic Management and AI

Problem Statement for Traffic Management from Engineering Perspective

� Problem (Short Term)Match traffic demand to supply with optimal usage of available public resources and concomitant optimization of citizens’ private resources for travel needs

– Example: For a given day, minimize over-time payments to traffic personnel while minimizing average commute time per km. Serviceobjectives can also be stated like average commute time per km be below 10 mins/ km.

– Implications: Conflicting goals: optimality of public resources (e.g., road, traffic cops) v/s optimality of individual’s resources (e.g., travel time)

� Problem (Long Term)For a known future period, match traffic demand to supply with optimal usage of available

and planned public resources and concomitant optimization of citizens’ private

© 2010 IBM Corporation

resources for travel needs.

– Example: For the next 3 years, minimize city’s expenses (annual operational budget and capital investment in the period) while minimizing average travel time per km and average fare per km. Service objectives can also be stated towards citizens like average commute time be below 10 mins/ km and average fare be below $1/ km.

� Implications: Information is used to plan city’s regions (including businesses, communities) and roads, thereby influencing traffic patterns

15

Details : Biplav Srivastava. A new look at the traffic management problem and where to start. In 18th ITS Congress, Orlando, USA, Oct 16-20, 2011.

Page 16: Tutorial on Taffic Management and AI

Starting Point from Problem Statement for Traffic Management

� Problem (Short Term)Match traffic demand to supply with optimal usage of available public resources and concomitant optimization of citizens’private resources for travel needs

– Example: For a given day, minimize over-time payments to traffic personnel while minimizing average commute time per km. Service objectives can also be stated like average commute time per km be below 10 mins/ km.

– Implications: Conflicting goals: optimality of public resources (e.g., road, traffic cops) v/s optimality of individual’s resources (e.g., travel time)

� Problem (Long Term)For a known future period, match traffic demand to supply with optimal usage of available and planned public resources

and concomitant optimization of citizens’ private resources for travel needs.

© 2010 IBM Corporation

– Example: For the next 3 years, minimize city’s expenses (annual operational budget and capital investment in the period) while minimizing average travel time per km and average fare per km. Service objectives can also be stated towards citizens like average commute time be below 10 mins/ km and average fare be below $1/ km.

– Implications: Information is used to plan city’s regions (including businesses, communities)and roads, thereby influencing traffic patterns

� Starting point for any solution requires traffic to be measured accurately at sustained, continuous basis, at a rate that helps effective traffic control

16

Details: Biplav Srivastava. A new look at the traffic management problem and where to start. In 18th ITS Congress, Orlando, USA, Oct 16-20, 2011.

Page 17: Tutorial on Taffic Management and AI

strategic

planning

Level 1Silo

Level 2Centralized

Level 3Partially Integrated

Level 4Multimodal Integrated

Level 5Multimodal Optimized

PlanningFunctional Area Planning (single mode)

Project-based Planning (single mode)

Integrated agency wide planning (single mode)

Integrated corridor-based multimodal planning

Integrated regional multimodal planning

Data Collection Limited or Manual Input

Near real-time for major routes

Real-time for major routes using multiple inputs

Real-time coverage for major corridors, all significant modes

System-wide real -time data collection across all modes

Performance

MeasurementMinimal

Defined metrics by mode

Limited integration across organizational silos

Shared multimodal system-wide metrics-

Continuous system-wide performance measurement

Customer

Management

Minimal capability, no customer accounts

Multi-channel account interaction per mode

Unified customer account across multiple modes

Integrated multimodal incentives to optimize multimodal use

Customer accounts managed separately for each system/mode

“Average” City

Top 3 City: range

Leading Practice

A Summary of where Global Cities are within the Transportation Maturity Model

Consulting Solution Approach

© 2010 IBM Corporation

real-time

information

creation

capability

real-time

intervention

capability

Data Integration Limited NetworkedCommon user interface

2-way system integration

Extended integration

Network Ops.

ResponseAd-Hoc, Single Mode

Centralized, Single Mode

Automated, Single Mode

Automated, Multimodal

Multimodal Real-time Optimized

Payment

MethodsManual Cash Collection

Automatic Cash Machines

Electronic PaymentsMultimodal integrated fare card

Multimodal, multi-media (fare cards, cell phones, etc)

Incident

Management

Manual detection, response and recovery

Manual detection, coordinated response, manual recovery

Automatic detection, coordinated response and manual recovery

Automated pre-planned multimodal recovery plans

Dynamic multimodal recovery plans based on real-time data

Demand

ManagementIndividual static measures

Individual measures, with long term variability

Coordinated measures, with short term variability

Dynamic pricingMultimodal dynamic pricing

Traveler

Information Static Information

Static trip planning with limited real-time alerts

Multi-channel trip planning and account-based alert subscription

Location-based, on-journey multimodal information

Location-based, multimodal proactive re-routing

Analytics Ad-hoc analysisPeriodic, Systematic analysis

High-level analysis in near real-time

Detailed analysis in real-time

Multi-modal analysis in real-time

Multimodal Network Management Maturity Model version 1.1 © Copyright IBM Corporation 2007

More details at: http://www-935.ibm.com/services/us/igs/pdf/transport-systems-white-paper.pdf

Page 18: Tutorial on Taffic Management and AI

An Intelligent Transportation Integration & Analytics Framework

Integration & AnalysisCollection

In-Vehicle Detectors

Mapping Services

(Creation and Maintenance of maps etc)

DisseminationCommand & Control

Roadside Infrastructure

(Variable Message Signs, Traffic Signals

etc)

Traveller Information Portal

In- vehicle

Devices

Mobile

Devices

Data Fusion(Creation of Single View)

Data Fusion(Creation of Single View)

Route Guidance &

Traffic Control

Center Dashboard

ITS Asset

Management,

Maintenance &

Optimization

Incident

Integration

Transport Management Subsystems

(Traffic Signals, Bridges, Parking, Mgt, etc)

Technology Approach

© 2010 IBM Corporation18

Mass Transit

Operational

Management (timetables, etc)

Infrastructure Detectors

(Loops, CCTV, Tag & Beacon etc)

In-Vehicle Detectors

(Probes etc)

Third Party

Information Feeds (e.g. Floating Cellular

Data)

Portal

(Info Display, Route Planning)

Traveler Information Gateway

(B2B info feeds etc)

Traveller Information Gateway (B2B info Feeds,

etc).

Self- service

Traveler Portal (view bill, pay charge,

etc)

External Systems (Vehicle Licensing,

Electronic Payments,

Bill Printing, etc)

Devices

Web

Browsers

Third Party

Systems

Mass Transit

Operational

Management (timetables, etc)

Data Warehouse(Reporting, Archiving)

Data Analytics(Forecasting , Journey Time

Determination)

Security

Systems Management

Route Guidance &

Trip Planning

Incident Management

(Process Execution)

Geographic

Information

System

Geographic

Information

System

Vehicle

Identification

Pricing ModelsPublic Access

Devices

(e.g. kiosks)

Integration

Integration

CRM (Call Centre, Interactive Voice)

Finance en& Paymts

Pricing Models

Page 19: Tutorial on Taffic Management and AI

References

� Mythologies, Metros & Future Urban Transport , by Prof. Dinesh Mohan, TRIPP, 2008

� A new look at the traffic management problem and where to start, by Biplav Srivastava, In 18th ITS Congress, Orlando, USA, Oct 16-20, 2011.

� Arnott, Richard and K.A. Small, 1994, “The Economics of Traffic Congestion,” American Scientist, Vol. 82, No. 5, pp. 446-455.

© 2010 IBM Corporation

� Chengri Ding and Shunfeng Song , Paradoxes of Traffic Flow and Congestion Pricing,

19

Page 20: Tutorial on Taffic Management and AI

Traffic Management and AI

Section: Traffic Sensing

Speaker: Biplav Srivastava

AAAI 2012 Tutorial

July 2011

© 2012 IBM Corporation

Toronto, Canada

Page 21: Tutorial on Taffic Management and AI

Outline

1. Traffic Management Problem

2. Instrumentation

1. Sensing traffic

2. Traffic state estimation

3. Optimizing and combining sensor data

3. Interconnection1. Middleware2. Traffic standards

© 2010 IBM Corporation

2. Traffic standards

4. Intelligence1. Path planning

1. For individual vehicles2. For public transportation

2. End-user analytics1. Bus arrival prediction and journey planning, with state-of-art instrumentation2. Multi-modal journey planning, without sensors

5. Supporting topics1. Traffic Simulators2. Practical considerations for real-world pilots

21

Page 22: Tutorial on Taffic Management and AI

A Feel For Traffic Sensing

No. Data type Format Ability for using in sensing

I1 Collected by manual

method

Document Can be used right after they

are integrated, cleaning and

conversion

I2 Mobile data CDR Data will be converted to

transport data in standard

format during project

implementation

I3 Data from mobile

which have GPS

Follow format

of data traffic

Data will be collected directly

during project deployment and

I3: 28km/hr

I2: 20km/hr

I1: 25km/hr

I2: 20km/hr

I2: 4 km/hr

© 2010 IBM Corporation

device switch to the format needed

I1: 25km/hr

Situation: Sensor readings from different types, various locations; for some links, no sensor present

Problem: What is the overall view of traffic?

Page 23: Tutorial on Taffic Management and AI

Sensed Traffic Metrics

� Traffic flow – number of vehicles* that pass a certain point during a specified time interval (vehicles/hour)

� Speed – rate of motion in distance/time (mph)

� Density – number of vehicles occupying a given length of highway or lane (vehicles per mile per lane)

© 2010 IBM Corporation

per lane)

� Spacing – the distance between successive vehicles in a traffic stream as they pass some common reference point on the vehicles

� Time headway – the time between successive vehicles in a traffic stream as they pass some common reference point on the vehicles

23

* What is a vehicle? There are 48 types in India! So measurement can be non-trivial!

Page 24: Tutorial on Taffic Management and AI

Traffic Flow and Time Headway

Traffic Flow given by:

t

nq =

q= traffic flow in vehicles per unit time

t= duration of time interval

n= number of vehicles passing some

designated roadway point during time interval t

© 2010 IBM Corporation

Page 25: Tutorial on Taffic Management and AI

Time Mean Speed

� Measure of speed at a certain point in space

Arithmetic mean of vehicles speeds is given by:

n

© 2010 IBM Corporation

n

u

u

n

i

i

t

∑== 1

ut=time-mean speed in unit distance per unit time

ui=spot speed of the ith vehicle

n=number of measured vehicle spot speeds

Page 26: Tutorial on Taffic Management and AI

Space Mean Speed

� Time necessary for a vehicle to travel some known length of roadway

t

lus =

© 2010 IBM Corporation

∑=

=n

i

itn

t1

1

us= space-mean speed in unit distance per unit time

l = length of roadway used for travel time measurements of vehicles

t = vehicle travel time

ti= time necessary for vehicle i to travel a roadway section of length l

Page 27: Tutorial on Taffic Management and AI

Evolution of Traffic Flow Sensor Technology

Goal: Get traffic information (speed, volume) for city region on sustained, continuous basis, at a rate that helps effective traffic control at low cost

Latest Buzz: • Social Media• Mobile phones

© 2010 IBM Corporation27

1960s

Inductive-loop detector

1970s

Video image processor

1990s

GPS-based floating car

After 2000

Mobile phones as traffic probes

1920s

Auto traffic signal

• High spatial coverage• Cost effectiveness• Network-wide info.• 24x7 availability

• Weather insensitive• Rich traffic data• High flexibility

• High spatial coverage• Cost effectiveness• Network-wide info.• 24x7 availability

• Weather insensitive• Rich traffic data• High flexibility

• Network-wide info• Enlarged coverage• Increased flexibility• Rich traffic data through XFCD

• Network-wide info• Enlarged coverage• Increased flexibility• Rich traffic data through XFCD

Accuracy & reliabilityAccuracy & reliability

• Multiple lanes and zones monitoring• Rich traffic data

• Flexibility

• Multiple lanes and zones monitoring• Rich traffic data

• Flexibility

Source: IBM China Research Lab

Page 28: Tutorial on Taffic Management and AI

Conventional Sensing Approach

� Take whatever sensors are in place and in immediate control of the concerned department, ignoring other sensing data

� Sample References– Handbook reference for : inductive loop detectors, magnetic sensors and detectors, video image processors, microwave radar sensors, laser radars, passive infrared and passive acoustic array sensors, and ultrasonic sensors, plus combinations of sensor technologies. http://www.fhwa.dot.gov/publications/research/operations/its/06108/

– Inductive loop detectors: Ingram, J.W. The Inductive Loop Vehicle Detector: Installation Acceptance Criteria and Maintenance Techniques, Report No. TL 631387, prepared by California Department of Transportation, Sacramento, CA, for Federal Highway Administration, Washington, DC. U.S. Department of Commerce, National Technical Information Service, PB-263 948, Washington, DC, March 1976.

© 2010 IBM Corporation

National Technical Information Service, PB-263 948, Washington, DC, March 1976.– Manual: N. Bansal, and B. Srivastava. On Using Crowd for Measuring Traffic at Aggregate Level for Emerging Countries. IIWeb workshop, WWW 2011, Hyderabad, India, March 28, 2011.

– Audio: H. Bischof, M. Godec, C. Leistner, B. Rinner, and A. Starzacher. Autonomous Audio-Supported Learning of Visual Classifiers for Traffic Monitoring. IEEE Intell. Sys., 25(3) pg 15–23, May/June 2010.

– Video: M. Bramberger, R. Pflugfelder, A. Maier, B. Rinner, B. Strobl, and H. Schwabach. A Smart Camera for Traffic Surveillance. WISES03 pages 12, Vienna, Austria, June 2003.

– Mobile: Y. Zhao. Mobile Phone Location Determination and Its Impact on Intelligent Transportation Systems. IEEE Trans. ITS, col. 1, NO. 1, Mar. 2000.

– GPS: M. A. Quddus, W. Y. Ochieng, and R. B. Noland. Current mapmatching algorithms for transport applications: State-of-the art and future research directions. Transportation Research, Part C, vol. 15, no. 5, pp. 312–328, Oct. 2007.

– Hybrid: M. Prashanth, V. N. Padmanabhan, and R. Ramjee. Nericell: rich monitoring of road and traffic conditions using mobile smartphones. Proc. ACM Sensys, Pg. 323-336, 2008

Page 29: Tutorial on Taffic Management and AI

Technology Complementarity

Inductive-loop detector

Timeliness

Accuracy

Reliability

Security & Privacy

Temporal coverage

Spatial coverageLow operational cost

Resource

consolidation

Network-wide

information

Rich traffic data

Analysis easiness

Video image processor

Timeliness

Accuracy

Reliability

Security & Privacy

Temporal coverage

Spatial coverageLow operational cost

Resource

consolidation

Network-wide

information

Rich traffic data

Analysis easiness

No Single Panacea

© 2010 IBM Corporation29

Trustiness (legal)Low capital cost Trustiness (legal)Low capital cost

Floating car data

Timeliness

Accuracy

Reliability

Security & Privacy

Temporal coverage

Spatial coverage

Trustiness (legal)Low capital cost

Low operational cost

Resource

consolidation

Network-wide

information

Rich traffic data

Analysis easiness

Mobile traffic probe

Timeliness

Accuracy

Reliability

Security & Privacy

Temporal coverage

Spatial coverage

Trustiness (legal)Low capital cost

Low operational cost

Resource

consolidation

Network-wide

information

Rich traffic data

Analysis easiness

Source: IBM China Research Lab

Page 30: Tutorial on Taffic Management and AI

Illustration of Some Sensor Types

© 2010 IBM Corporation30

Page 31: Tutorial on Taffic Management and AI

Induction Loop System

© 2010 IBM Corporation

31

Page 32: Tutorial on Taffic Management and AI

Average Speed Calculation from Inductive Loop Detector

� Method to calculate the average speed of the links:

© 2010 IBM Corporation

� Reliability of method depends on the estimated value for g

32

Page 33: Tutorial on Taffic Management and AI

Video-based Vehicle Counting

© 2010 IBM Corporation33

http://www.youtube.com/watch?v=KWobYJzQl_k

Page 34: Tutorial on Taffic Management and AI

34

Available Mobility Data

1. Passive collection:

on-call phonesstandby phones

3. GPS Enabled Phones:

Network Based Terminal Based

Need to upload the DataPower Consumption Issue

© 2010 IBM Corporation

LU (Location Update)

Location Area Dimension: 3~10km

Continuous comm. behaviors (SMS/MMS/Call) and HO (Handover)

Location Area Dimension: 100m~2km

2. Active collection:

Positioning TechniquesAccuracy: 50~200m

Accuracy: 5~30m

4. Other Software Installed:

Accuracy: 50m~2km

Network Overhead

Source: IBM China Research Lab

Y. Zhao. Mobile Phone Location Determination and Its Impact on Intelligent Transportation Systems. IEEE Trans. ITS, col. 1, NO. 1, Mar. 2000.

Page 35: Tutorial on Taffic Management and AI

Road-side Acoustics based Sensing

� Honking

� Road sounds–Engine–Road–Air

© 2010 IBM Corporation

–Air–…

35

Page 36: Tutorial on Taffic Management and AI

Honk-Ok-Please [IIT Bombay]

© 2010 IBM Corporation36``Horn-Ok-Please'', Rijurekha Sen, Bhaskaran Raman, Prashima Sharma, The Eighth International Conference on Mobile Systems, Applications, and Services, ACM MobiSys 2010, Jun 2010, San Francisco, CA

Summary

• Can detect speed with high accuracy (relativeerror < 20% in most cases)

• Can detect congestion v/s free-flow

• Results heavily dependent on positioning of sensors (recorders),

Page 37: Tutorial on Taffic Management and AI

Traffic State: Audio spectrograms of various traffic density states, free flow, medium and jammed, and their corresponding cues

Road-side cumulative acoustics composed of various noise signals for eg: tire noise, engine noise, engine idling noise, occasional honks, air turbulence noise;

Traffic density state determines combination of noise signals. Therefore use

© 2010 IBM Corporation

combination of noise signals. Therefore use the spectral modulation acoustic features along with the Gaussian mixture Hidden Markov models to statistically characterize these traffic density states.

Initial experimentation have established the feasibility and accuracy of using this approach.

Vehicular Traffic Density State Estimation Based on Cumulative Road Acoustics, IEEE Transactions on Intelligent Transportation Systems, 2011

Classification accuracy using MFCC featuresframes covering T sec of audio, SVM classifier

Page 38: Tutorial on Taffic Management and AI

Sensor Subset Selection and Optimization

© 2010 IBM Corporation38

Page 39: Tutorial on Taffic Management and AI

Problem Considered

� City authorities need to decide what sensors to use to get traffic data for traffic of their region– Slew of techniques available varying in accuracy, coverage and cost to install and maintain– Diversity in how they can be set up – How sensing methods may complement each other

� City can make an initial decision but it will need to be re-visited over time

© 2010 IBM Corporation

� City can make an initial decision but it will need to be re-visited over time– Traffic patterns changes– Sensing technologies changes

Problem : find the subset of sensors from available types that give the best cost-benefit for a given traffic pattern

Page 40: Tutorial on Taffic Management and AI

Effect of Increasing Number of Sensors on RMSE

© 2010 IBM Corporation

[Traffic Pattern 1 on a Grid] Effect of increasing number of sensors of different type. a)Shows that RMSE decreases with increase in percentage of sensors from 10 to 100%. b)Shows increase in cost is highest for manual sensors whereas it is lowest for CDR.

(a) (b)

Page 41: Tutorial on Taffic Management and AI

Subset Selection Approach

� Model sensor types based on cost, accuracy and coverage

� Create a sample space of sensor combination choices

� Use a traffic simulator (MATSIM)– To measure the sensing error distribution entailed in each sensor

combination– To ensure physical characteristics of the city are taken into account

� Choose Pareto sensor combinations (non-dominated); Call this “Optimal Candidate Set (OCS)”

© 2010 IBM Corporation

� Optional filtering steps– Remove combinations above a give cost threshold– Remove combinations above an error threshold

� For a given set of ‘k’ optimal combinations to be returned– Select a preference function– Use OCS selection using ICP approximation (Carlyle et al)

� Return ‘k’ optimal sensor combinations

Manual Video CDR GPS RMSE Cost Norm RMSE Norm Cost

100 0 0 0 1.2 4200 0 1

0 0 0 100 2.25 2520 0.028074866 0.592034968

0 0 100 0 6 840 0.128342246 0.184069937

0 0 40 0 20.55 335 0.517379679 0.061437591

0 0 10 0 38.6 82 1 0

0 0 10 0 38.6 82 1.00 0.00

0 0 20 0 29.4 167 0.75 0.02

0 0 30 0 25.05 247 0.64 0.04

0 0 40 0 20.55 335 0.52 0.06

0 0 50 0 17.85 414 0.45 0.08

0 0 60 0 15.6 500 0.39 0.10

0 0 70 0 11.95 586 0.29 0.12

0 0 80 0 9.65 668 0.23 0.14

0 0 90 0 8.45 753 0.19 0.16

0 0 100 0 6 840 0.13 0.18

0 0 100 10 5.8 1088 0.12 0.24

10 0 100 0 5.4 1175 0.11 0.27

0 0 100 30 5.1 1600 0.10 0.37

0 0 100 40 4.65 1835 0.09 0.43

0 0 100 50 4.45 2093 0.09 0.49

0 0 100 60 3.95 2356 0.07 0.55

0 0 0 100 2.25 2520 0.03 0.59

10 0 0 100 2.2 2855 0.03 0.67

10 0 10 100 2.15 2937 0.03 0.69

20 0 0 100 2.05 3348 0.02 0.79

30 0 20 100 2 4005 0.02 0.95

0

5

10

15

20

25

30

35

40

45

0 2000 4000 6000

CostRMSE

Non SelectedPareto

Selected Pareto

Sensor Subset Selection for Traffic Management, R. Gupta and B. Srivastava, in 14th International IEEE Annual Conference on Intelligent Transportation Systems (ITSC 2011), Washington DC, USA, Oct 5-7, 2011.

Page 42: Tutorial on Taffic Management and AI

Examples of City Preferences in Sensor Selection

� (Case A): a city which already has some sensors in place would want to add only those new sensor types which complement its investments– Maximize incremental accuracy improvement

� (Case B): a city building the ITS infrastructure for the first time would want the highest accuracy possible within its budget– Highest accuracy

© 2010 IBM Corporation

– Highest accuracy

� (Case C): a city may have low budgets and would want to have high coverage even at the loss of some accuracy (which it may compensate by putting more people on the field)– Minimize cost

� (Case D): a city may want to minimize operational costs, even trading it off with higher capital expenditure, possibly because they come from a different budget– Minimize operational cost

42

Page 43: Tutorial on Taffic Management and AI

Key Results from Sensing

� Low-cost, noisy, CDR-based sensing complements existing sensors in a city due to easy coverage it provides– Highly beneficial when other, more accurate, sensor types are few– Benefit diminishes as the number of more accurate sensors increases– Important result for traffic management of emerging countries

© 2010 IBM Corporation

� Sensor type combinations can be suggested for a city’s traffic patterns using a simulator balancing accuracy and cost of sensing– Returns non-dominated results– Return few results approximating the search space

43

Page 44: Tutorial on Taffic Management and AI

Extracting Real City Characteristics for Simulation to Choose Sensors

© 2010 IBM Corporation44

Page 45: Tutorial on Taffic Management and AI

Combining Multiple Sensor Data

An example from Boston, USA completed in June 2012

© 2010 IBM Corporation45

Page 46: Tutorial on Taffic Management and AI

Traffic Context in Boston: Background

Consumers

� Urban planning users– Needs counts done annually (max) or more frequently

– Currently uses 10 year old data

� Environment users– Needs counts every 15 mins (min) or longer time

� Transportation users

Data Sources for Traffic Count

� Data in hand– Inductive loop detectors (Transportation asset)

– Manual count by consultants during urban development (Urban planning asset)

– Tube count (by state authorities)

� Future

© 2010 IBM Corporation

� Transportation users– Collects and uses counts every 15 mins

� Others– Residents: Consume for their personal choices

– Businesses: identify new opportunities

� Future – Video camera (multiple organizations)– GPS (city vehicles)– Directly by citizens (e.g, Citizen Connect)– Social media

All consumers demand quality over quantity and frequency, since no sensor is perfect.

Page 47: Tutorial on Taffic Management and AI

Model Consolidation

• Objectives• Make it simple to consume (traffic count) data*; easily provide content that is needed

commonly by applications• Enable entitled users who want to see details to access them• Ease the process to import, process and manage data for IT staff

• Approach

© 2010 IBM Corporation

• Approach1. Identify requirements from relevant consumers, negotiate2. (Domain knowledge): know what information typical consumers (traffic and other domains) want3. Look at sample data4. Select attributes commonly of interest5. Have additional attributes for data quality, provenance and cost, as applicable6. Validate selected attributes with consumers and implementers7. Reconcile with applicable standards (NIEM, CAP, TMDD, Datex2) and enhance description8. Sign-off: Freeze for development and create documentation

* Set expectation for data quality and error checking

Page 48: Tutorial on Taffic Management and AI

Common Model

Standards Aligned,Uniform format, Uniform Error Semantics

A Snapshot of Common Model and Mapping to Data Sources

Source Models

© 2010 IBM Corporation

Mapping to Source

DataTransformation

Data Source Metadata

Page 49: Tutorial on Taffic Management and AI

References

� Handbook reference for : inductive loop detectors, magnetic sensors and detectors, video image processors, microwave radar sensors, laser radars, passive infrared and passive acoustic array sensors, and ultrasonic sensors, plus combinations of sensor technologies. http://www.fhwa.dot.gov/publications/research/operations/its/06108/

� Ingram, J.W. The Inductive Loop Vehicle Detector: Installation Acceptance Criteria and Maintenance Techniques, Report No. TL 631387, prepared by California Department of Transportation, Sacramento, CA, for Federal Highway Administration, Washington, DC. U.S. Department of Commerce, National Technical Information Service, PB-263 948, Washington, DC, March 1976.

� N. Bansal, and B. Srivastava. On Using Crowd for Measuring Traffic at Aggregate Level for Emerging Countries. IIWeb workshop, WWW 2011, Hyderabad, India, March 28, 2011.

� H. Bischof, M. Godec, C. Leistner, B. Rinner, and A. Starzacher. Autonomous Audio-Supported Learning of Visual Classifiers for Traffic Monitoring. IEEE Intell. Sys., 25(3) pg 15–23, May/June 2010.

� M. Bramberger, R. Pflugfelder, A. Maier, B. Rinner, B. Strobl, and H. Schwabach. A Smart Camera for Traffic Surveillance. WISES03 pages 12, Vienna, Austria, June 2003.

© 2010 IBM Corporation

WISES03 pages 12, Vienna, Austria, June 2003.� Y. Zhao. Mobile Phone Location Determination and Its Impact on Intelligent Transportation Systems. IEEE Trans. ITS, col.

1, NO. 1, Mar. 2000.� M. A. Quddus, W. Y. Ochieng, and R. B. Noland. Current mapmatching algorithms for transport applications: State-of-the

art and future research directions. Transportation Research, Part C, vol. 15, no. 5, pp. 312–328, Oct. 2007.� M. Prashanth, V. N. Padmanabhan, and R. Ramjee. Nericell: rich monitoring of road and traffic conditions using mobile

smartphones. Proc. ACM Sensys, Pg. 323-336, 2008� Sensor Subset Selection for Traffic Management, R. Gupta and B. Srivastava, in 14th

International IEEE Annual Conference on Intelligent Transportation Systems (ITSC 2011), Washington DC, USA, Oct 5-7, 2011.

� V. Tyagi, S. Kalyanaraman, R. Krishnapuram. Vehicular Traffic Density State Estimation Based on Cumulative Road Acoustics, IEEE Transactions on Intelligent Transportation Systems, 2011

� ``Horn-Ok-Please'', Rijurekha Sen, Bhaskaran Raman, Prashima Sharma, The Eighth International Conference on Mobile Systems, Applications, and Services, ACM MobiSys 2010, Jun 2010, San Francisco, CA

49

Page 50: Tutorial on Taffic Management and AI

Traffic Management and AI

Section: Interconnection

Speaker: Anand Ranganathan

AAAI 2012 Tutorial

July 2012

© 2012 IBM Corporation

Toronto, Canada

Page 51: Tutorial on Taffic Management and AI

Outline

1. Traffic Management Problem

2. Instrumentation1. Sensing traffic2. Traffic state estimation3. Optimizing and combining sensor data

3. Interconnection

1. Middleware

2. Traffic standards

© 2010 IBM Corporation

2. Traffic standards

4. Intelligence1. Path planning

1. Simple Illustration2. Path Planning for Individual Vehicles

2. End-user analytics1. Bus arrival prediction and journey planning, with state-of-art instrumentation2. Multi-modal journey planning, without sensors

5. Supporting topics1. Traffic Simulators2. Practical considerations for real-world pilots

51

Page 52: Tutorial on Taffic Management and AI

Challenges in Middleware

� Scalability

–processing large volumes of real-time and static data

� Ability to incorporate various kinds of specialized as well as reusable analytics–Classifiers, forecast/prediction algorithms, simulators, spatio-temporal processing, image/video processing, etc.

© 2010 IBM Corporation

processing, image/video processing, etc.

� Extensibility

–being able to add sources and data and new kinds of analyses on the data rapidly

– support different kinds of one-time and continuous queries from diverse end-users• commuters, highway patrols, public service vehicles like re-engines and ambulances, departments of transportation, urban planners, commercial vehicle operators, etc.

Page 53: Tutorial on Taffic Management and AI

Addressing these challenges on the IBM InfoSphere Streams platform

� Stream Processing–New data processing paradigm where large volume of real-time (sensor) data is processed on the fly, without storing in a database

–Continuous queries (and processing)• Queries are long running, while data keeps moving

© 2010 IBM Corporation

� Key Features of InfoSphere Streams–Parallel and high performance stream processing software platform –Analysis of structured and unstructured in information

Page 54: Tutorial on Taffic Management and AI

Data is being produced continuously in very large volumes in different domains

Stock market• Impact of weather on

securities prices

• Analyze market data at ultra-low latencies

Radio Astronomy

• Detection of transient events

Natural Systems

• Seismic monitoring

• Wildfire management

• Water management

© 2010 IBM Corporation

Transportation

• Intelligent traffic management

Law Enforcement

• Real-time multimodal surveillance

Fraud prevention

• Detecting multi-party fraud

• Real time fraud preventionManufacturing

• Process control for microchip fabrication

Health & Life Sciences

• Neonatal ICU monitoring

• Epidemic early warning system

• Remote healthcare monitoring

Page 55: Tutorial on Taffic Management and AI

� continuous ingestion � continuous analysis

How Streams Works

© 2010 IBM Corporation

achieve scale by

partitioning applications into components

Page 56: Tutorial on Taffic Management and AI

� continuous ingestion

� continuous analysis

How Streams Worksinfrastructure provides services for

scheduling analytics across h/w nodes

establishing streaming connectivity

…Transform

Filter

Annotate

© 2010 IBM Corporation

achieve scale

by partitioning applications into components

by distributing across stream-connected hardware nodes

Classify

Correlate

• where appropriate,

• elements can be ““““fused”””” together

• for lower communication latencies

Page 57: Tutorial on Taffic Management and AI

Application Programming

Source Adapters Sink AdaptersOperator Repository

MARIO: Automated Application Composition

© 2010 IBM Corporation

� Consumable

� Reusable set of operators

� Connectors to external static or streaming data sources and sinks

SPL: Stream processing dataflow scripting language

MARIO: Automated Application Composition

Platform Optimized Compilation

Page 58: Tutorial on Taffic Management and AI

SPL Language Highlights

� Intermediate language describing how a set of individual stream processing operators are connected to one another in a graph

� Each individual operator has zero or more input and output streams:–A Built-In Operator provided by the SPL language

• Filter, Aggregator, Join, Split, Barrier, –A User Defined Operator

• Written in C++ or Java

© 2010 IBM Corporation

• Written in C++ or Java• Can be written in a schema independent manner to enhance reusability

–A source or sink operator to read/write from/to external data sources

� Procedural constructs– To create distributed and parallel flow graphs

� Settings to influence placement of operators, fusion into processes, execution model, etc.

Page 59: Tutorial on Taffic Management and AI

A simple example

[Application]

SourceSink trace

[Typedefs]

typedef id_t Integer

typedef timestamp_t Long

[Program]

# a source stream is generated by a Source operator – in this case tuples come from an input file

stream SenSource ( id : id_t, location : Double, light : Float, temperature : Float, timestamp : timestamp_t )

:= Source( ) [ “file:///SenSource.dat” ] {}

SinkSource Aggregate Functor

SenSourceSenAggregator

SenFunctor

© 2010 IBM Corporation

:= Source( ) [ “file:///SenSource.dat” ] {}

# this intermediate stream is produced by an Aggregate operator, using the SenSource stream as input

stream SenAggregator ( id : id_t, location : Double, light : Float, temperature : Float, timestamp : timestamp_t )

:= Aggregate( SenSource <count(100),count(1)> ) [ id . location ]

{ Any(id), Any(location), Max(light), Min(temperature), Avg(timestamp) }

# this intermediate stream is produced by a functor operator

stream SenFunctor ( id: Integer, location: Double, message: String )

:= Functor( SenAggregator ) [ log(temperature,2.0)>6.0 ]

{ id, location, “Node ”+toString(id)+ “ at location ”+toString(location) }

# result management is done by a sink operator – in this case produced tuples are sent to a socket

Nil := Sink( SenFunctor ) [ “cudp://192.168.0.144:5500/” ] {}

Page 60: Tutorial on Taffic Management and AI

Infosphere Streams Runtime Services

Optimizing scheduler assigns operators to

processing nodes, and continually

manages resource allocation

Runs on commodity hardware – from single

node to blade centers to high performance

multi-rack clusters

© 2010 IBM Corporation

X86

Box

X86 Blade

Cell

Blade

X86 Blade

FPGA

Blade

X86

Blade

X86 Blade

X86

Blade

X86 Blade

X86

Blade

Transport

Infosphere Streams Data Fabric

Processing Element Container

Processing Element Container

Processing Element Container

Processing Element Container

Processing Element Container

Page 61: Tutorial on Taffic Management and AI

Infosphere Streams Runtime Services

Optimizing scheduler assigns operators to

processing nodes, and continually

manages resource allocationAdapts to changes in resources,

workload, data ratesRuns on commodity hardware – from single

node to blade centers to high performance

multi-rack clusters

© 2010 IBM Corporation

X86

Box

X86 Blade

Cell

Blade

X86 Blade

FPGA

Blade

X86

Blade

X86 Blade

X86

Blade

X86 Blade

X86

Blade

Transport

Infosphere Streams Data Fabric

Processing Element Container

Processing Element Container

Processing Element Container

Processing Element Container

Processing Element Container

Capable of exploiting specialized hardware

Page 62: Tutorial on Taffic Management and AI

Real-Time GPS Data Processing – Map Matching

� Inputs:• {GPS-id, Latitude, Longitude, Heading} tuples • Collection of poly-lines (the map)

� Output:• {GPS-id, Shape-id, distance} tuple, representing the line-segment that is the best match for the GPS reading

© 2010 IBM Corporation

the best match for the GPS reading• Filter based on heading and distance

� Our technique : Grid-Based Decomposition

Page 63: Tutorial on Taffic Management and AI

Scaling Map-Matching in InfoSphere Streams - how can it be done?

Source Map

Map

Map

Map

Map

Reduce

Map

Map

Map

(…)

Source

© 2010 IBM Corporation

Source Map

Map

Map

Map

Map (…)Reduce

(…)

(…)

(…)

Can implement a streaming version of map-reduce on the InfoSphere Streams platform

Given those operators, building and extending the application is just a matter of wiring those operators.

Source

Page 64: Tutorial on Taffic Management and AI

No. of Sources Map Size

(# links)

CPU % Throughput

(#GPS points

mapped)

1 200M 10 310K

2 200M 16 589K

3 200M 20 777K

4 200M 27 1003K

Results

© 2010 IBM Corporation

4 200M 27 1003K

1 500M 14 301K

2 500M 24 582K

4 500M 37 964K

1 1000M 18 294K

2 1000M 33 596K

4 1000M 53 941K

Page 65: Tutorial on Taffic Management and AI

Illustration: Streams for State Estimation

© 2010 IBM Corporation65

Page 66: Tutorial on Taffic Management and AI

Recall: Elements of Traffic State

� Traffic flow – number of vehicles that pass a certain point during a specified time interval (vehicles/hour)

� Speed – rate of motion in distance/time (mph)

� Density – number of vehicles occupying a given length of highway or lane (vehicles per mile per lane, vpmpl)

© 2010 IBM Corporation

� Spacing – the distance between successive vehicles in a traffic stream as they pass some common reference point on the vehicles

� Time headway – the time between successive vehicles in a traffic stream as they pass some common reference point on the vehicles

66

Page 67: Tutorial on Taffic Management and AI

Using GPS Data … Prototype for Stockholm

� Historic GPS data traces from Stockholm for the year 2008

–1500 taxis, 40 trucks– 170 million GPS points for the year–Each taxi produces a reading every 60 seconds–Trucks – every 30 seconds–Peak rate of about 1000 readings/min

© 2010 IBM Corporation

–Peak rate of about 1000 readings/min–Often replayed at a higher rate – over 100,000 readings per sec

� Road Network

–628,095 links, spread over a 80km x 80km area

Page 68: Tutorial on Taffic Management and AI

Overall Application Description

InteractiveStorage

GPS

Data

Streams

Real Time Transformation

Logic

Real Time Geo

Mapping

Real Time Speed & Heading Estimation

Real-time GPS data processing

Cleaning

Map-Matching

Per-Vehicle Speed Estimation

Real Time Aggregates & Statistics

Aggregated Statistics

Per-link / per-region

Stochastic Link Travel Time

© 2010 IBM Corporation

Interactive

visualization

Data

Warehouse

Offline

statistical

analysis

adapters

Web

Server

Google

Earth

Calculation

Stochastic Path Travel Time

Calculation

Splitter

Shortest

Path 1

Shortest

Path n

User-driven computations

Travel times, Shortest Paths

Page 69: Tutorial on Taffic Management and AI

Matching map artifact

GPS probe

Real Time Geo Mapping & Speed Estimation

© 2010 IBM Corporation

Estimated path

Estimated speed & heading

Page 70: Tutorial on Taffic Management and AI

Traffic Statistics Generation

� Aggregation and Inter-Quartiles Generation

Output Stream Schema

Aggregation operation, specifying window boundaries

© 2010 IBM Corporation

specifying window boundaries and attributes to group by

Logic for computing fields in output schema

execution process

Specifies which execution container to run operation in. SPADE allows fusing multiple

operators into a single execution process

Page 71: Tutorial on Taffic Management and AI

User Specific Computations

� Time Dependent Shortest Paths

� Stochastic Travel Times

� Stochastic Shortest Paths

© 2010 IBM Corporation

Page 72: Tutorial on Taffic Management and AI

ITS Application Flow-graph (125k GPS/second)

5min Aggr.

KML Color map

Inter-quartiles (IQ) week-

© 2010 IBM Corporation

GPS source

ISO 8601 to timestamp

Time of day, day of week

month of year

Filter taxis and invalid

values

GeoMatch

GpsTrack

Clean Sort Adapt

Travel times

JoinQuery

DSPends

IQ

IQ

Sink

Current Traffic

Statistics

Sink

Page 73: Tutorial on Taffic Management and AI

Showing Placement of Operators on 4 nodes

© 2010 IBM Corporation

Page 74: Tutorial on Taffic Management and AI

Scaling of throughput as we add more nodes

© 2010 IBM Corporation

Page 75: Tutorial on Taffic Management and AI

Features of InfoSphere Streams / SPADE that helps development

� Component Based Programming Model

–Applications developed as a graph of reusable operators–Operators can be built-in or user defined

� Modular Application Development

–Applications can import and export streams–Allows composing entire applications

© 2010 IBM Corporation

–Allows composing entire applications

� Declarative Application Configuration within SPADE

–Degree of parallelization–Fusion of operators–Placement of operators onto nodes

� Rich toolkits of available Operators

–Relational, Geo-Spatial, Stream Mining,…

Page 76: Tutorial on Taffic Management and AI

References

� Alain Biem, Eric Bouillet, Hanhua Feng, Haris Koutsopoulos, Carlos Moran, Anand Ranganathan, Anton Riabov, Olivier Verscheure, IBM Infosphere Streams for Scalable, Real-Time, Intelligent Transportation Services, In ACM International Conference on Management of Data (SIGMOD 2010), June 6-11, 2010, Indianapolis, IN

� Eric Bouillet, Anand Ranganathan, Scalable, Real-time Map-Matching using IBM’s System S, In 11th International Conference on Mobile Data Management (MDM 2010),

© 2010 IBM Corporation

System S, In 11th International Conference on Mobile Data Management (MDM 2010), Kansas City, MO, May 23- 26, 2010

76

Page 77: Tutorial on Taffic Management and AI

Traffic Management and AI

Section: Useful Standards (or Commonly Used)

Speaker: Biplav Srivastava

AAAI 2012 Tutorial

July 2011

© 2012 IBM Corporation

Toronto, Canada

Page 78: Tutorial on Taffic Management and AI

Outline

1. Traffic Management Problem

2. Instrumentation1. Sensing traffic2. Traffic state estimation3. Optimizing and combining sensor data

3. Interconnection

1. Middleware 2. Traffic standards

© 2010 IBM Corporation

2. Traffic standards

4. Intelligence1. Path planning

1. Simple Illustration2. Path Planning for Individual Vehicles

2. End-user analytics1. Bus arrival prediction and journey planning, with state-of-art instrumentation2. Multi-modal journey planning, without sensors

5. Supporting topics1. Traffic Simulators2. Practical considerations for real-world pilots

78

Page 79: Tutorial on Taffic Management and AI

Outline

� Motivation: Why we are talking standards

� Overview of immediately relevant standards

� Recommendations on standards

© 2010 IBM Corporation

Page 80: Tutorial on Taffic Management and AI

Motivation: Assembling the Smarter City … today’s cities

The data/ information, services, structures and activities in cities are described in different ways by the agencies which manage them, by the denizens which use them and by companies which

provide them

The data/ information, services, structures and activities in cities are described in different ways by the agencies which manage them, by the denizens which use them and by companies which

provide them

This cacophony of views and understanding drive many of the inefficiencies, create uneconomic costs in cities and limit the This cacophony of views and understanding drive many of the inefficiencies, create uneconomic costs in cities and limit the

© 2010 IBM Corporation

inefficiencies, create uneconomic costs in cities and limit the growth of city quality of life

inefficiencies, create uneconomic costs in cities and limit the growth of city quality of life

Research efforts ongoing to provide structures for the data, and the city infrastructure and services which reduce this confusion, significantly reduce costs and

improve life in the city. Example: SCRIBE (IBM).

Research efforts ongoing to provide structures for the data, and the city infrastructure and services which reduce this confusion, significantly reduce costs and

improve life in the city. Example: SCRIBE (IBM).

80

Page 81: Tutorial on Taffic Management and AI

Motivation: Traffic

� All parts of a city affect each other – none can work in isolation–Sample: Traffic incidents may affect public safety, water management, environment monitoring.

–Sample: External events may affect traffic, like a broken water pipe.

� Issue: how do participants share common information about the domain

© 2010 IBM Corporation

� Issue: how do participants share common information about the domain unambiguously? –E.g., congestion by traffic personnel , fire dept, electrical utility, mayor office, citizens, civil contractors, IT companies implementing Intelligent Transportation Systems

–How are concepts related to the data actually being collected. E.g., congestionto (motorized) traffic count.

Page 82: Tutorial on Taffic Management and AI

Motivation: Reducing every agency’s software to understandingat most 2 representations, not n representations

Common Data Model

(supporting RDF/XML

orSQL)

Police

Applications

Police

Formats

Traffic

Applications

Traffic

Formats

Fire DeptFire WaterWater

© 2010 IBM Corporation82

orSQL)Fire Dept

Applications

Fire

Formats

Water

ApplicationsFormats

Traffic

Applications

Traffic

Formats

Building

Applications

Bldg

Formats

Recreation

Applications

Recr’tn

Formats Align with select standards

(put annotations, support data formats)

Page 83: Tutorial on Taffic Management and AI

Sample Transportation StandardsStandard Examples of supported

concepts

Current deployments

Advanced Traveler Information Systems (ATIS), SAE 2354

Messages defined for traveler information, trip guidance, parking, Mayday (emergency information)

International standard (SAE) with traction in North AmericaGainingmomentum in the US; Nebraska, Washington and Minnesota planning support. See Washington State Department of Transportation Advanced Traveler Information Systems Business Plan for details.

DATEX II Traffic elements, operator actions, impacts, non-road event information, elaborated data, and measured data

European standardVersion I deployed in several European countries. DATEX II deployment appears to be gaining momentum in several European countries. See DATEX Deployments for details.

IEEE 1512 Common incident management message sets for use by emergency management centers, traffic management, public safety, hazardous materials, and entities external to centers

International standardEarly deployments include Washington D.C., NYC, and Milwaukee. See 1512 Public Safety Early Deployment Projects for details.

Source: Smarter city data model standards landscape, Part 2: Transportation

© 2010 IBM Corporation

external to centers

National Transportation Communication for ITS Protocol (NTCIP) 1200 Series

Currently thirteen data dictionaries defined, including object definitions for: dynamic message signs, CCTV camera control, ramp meter control, transportation sensor systems

U.S. specific standardAccording to NTCIP web site, several cities and states in the U.S. have NTCIP projects underway. See NTCIP Deployment: Projects for details.

Service Interface for Real Time Information (SIRI)

Information exchange of real-time information about public transportation services and vehicles

European specific standardBased on best practises of various national and proprietary standards from across Europe.

Traffic Management Data Dictionary (TMDD)

OwnerCenter, ExternalCenter, Device, DateTime, Dynamic Message Sign, Event, Generic, Organization, and RoadNetwork

International standard (ITE and AASHTO) with traction in North AmericaIn early stages of deployment in the US. TMDD is backed by US DoT and multiple vendors (Transcom, Siemens). Multiple state DoTs are planning to move to TMDD in their next rev (including Florida and Utah). See "A Report to the ITS Standards Community ITS Standards Testing Program, For TMDD and Related Standards as Deployed by the Utah Department of Transportation" for TMDD testing details from the Utah DoT.

Page 84: Tutorial on Taffic Management and AI

Relevant Standards

� Information exchanged – Transportation and external agencies: Datex 2– Between agencies: NIEM

� Format of information given: CAP

� Public Transportation: GTFS

© 2010 IBM Corporation

84

� Public Transportation: GTFS

� In addendum: Road Conditions– Road Surface Conditions– Road types– Types of restriction

Page 85: Tutorial on Taffic Management and AI

Information Exchanged

Information exchanged with DATEX II systems is composed of different basic elements:

• Road and traffic related events (called “Traffic elements”)• Operator actions• Advice

– Different types of advice: speed, lane usage, winter driving …

• Impacts• Non road event information

© 2010 IBM Corporation

• Non road event information– It concerns information about events which are not directly on the road: transit information, service disruption, car parks.

• Elaborated data (derived/computed data, e.g. travel times, traffic status)• Measured data (direct measurement data from equipments or outstations, e.g. traffic and weather measurements)

85

Datex 2Version 1.0

Page 86: Tutorial on Taffic Management and AI

Road Events (Elements)

Road and traffic related events (called “Traffic elements”)These are all events which are not initiated by the traffic operator and force him to undertake(re)actions. They are classified in 6 main categories:• Abnormal traffic (long queues, stop and go, …)• Accidents - are events in which one or more vehicles lose control and do not recover. They include collisions between vehicle(s) or other road user(s), between vehicle(s) and any obstacle(s), or they result from a vehicle running off the road. Details can be given:

– cause type,– accident type,– overview of people involved (number, injury status, involvement role, type),– overview of vehicles involved per type (number, status, type, usage),

© 2010 IBM Corporation

– overview of vehicles involved per type (number, status, type, usage),– details of involved vehicles (type + individual vehicle characteristics)

• Obstructions :– animal presence,– vehicles presence,– obstructions due to environment (avalanches, flooding, fallen trees, rock falls, …),– obstructions due to equipments (fallen power cables, …)

• Activities• Incidents on infrastructure equipments (Variable message sign out of order, tunnel ventilation not working, emergency telephone not working, …)

• Specific conditions : road conditions related to weather (ice, snow, … ) or not (oil, …), environment conditions (precipitation, wind, …), …

86

Datex 2Version 1.0

Page 87: Tutorial on Taffic Management and AI

Operator actions

Operator actions are classified in 4 main categories:

� Network management :road closure, alternate traffic, contra flow …– traffic control : rerouting, temporary limits

� Roadworks: resurfacing, salting, grass cutting …

© 2010 IBM Corporation

� Roadside assistance: vehicle repair, helicopter rescue, food delivery …

� Sign settings: variable message signs, matrix signs.

87

Datex 2Version 1.0

Page 88: Tutorial on Taffic Management and AI

Impact

� Delays and traffic status (Impact) have 5 possible values:

– free flow,– heavy,– congested,– impossible,

© 2010 IBM Corporation

– impossible,– Unknown

88

Datex 2Version 1.0

Page 89: Tutorial on Taffic Management and AI

Elaborated Data - These sets of data are normally derived on a periodic basis by the Traffic Centre from input data for specified locations

� Traffic values (normally published as measured data, but can be derived on a periodic basis and published as elaborated data):

– flow, – speed, – headway, – concentration and – individual vehicle measurements.

© 2010 IBM Corporation

– individual vehicle measurements.

� Weather values (normally published as measured data, but can be derived on a periodic basis and published as elaborated data):

– precipitation, – wind, – temperature, – pollution, – Road surface condition and – visibility.

89

Datex 2 Version 1.0

Page 90: Tutorial on Taffic Management and AI

Measured Data - These data sets are normally derived from direct inputs from outstations or equipments at specific measurement sites (e.g. loop detection sites or weather stations) which are received on a regular (normally frequent) basis

� traffic values: – flow, speed, headway, concentration and individual vehicle measurements.

� weather values: – precipitation, wind, temperature, pollution, road surface condition and visibility.

� travel times: – elaborated time, free flow time, normally expected time

© 2010 IBM Corporation

� traffic status: – free flow, heavy, congested, impossible, Unknown

90

Datex 2Version 1.0

Page 91: Tutorial on Taffic Management and AI

National Information Exchange Model: Summary

� “NIEM provides a common vocabulary for consistent, repeatable exchanges of information between agencies and domains. The model is represented in a number of forms, including a data dictionary and a reference schema, and includes the body of concepts and rules that underlie its structure, maintain its consistency, and govern its use.

� NIEM:– Is a foundation for information exchange.– Offers a common vocabulary so that when two or more people talk to each

other they can exchange information based on common words that they both understand.

– Is community-driven.– Provides a data model, governance, methodologies, training, technical

© 2010 IBM Corporation

– Provides a data model, governance, methodologies, training, technical assistance, and an active community to assist users in adopting a standards-based approach to exchanging information.

– Provides technical tools to support development, discovery, dissemination, and reuse of information exchanges.

– Provides a forum for accelerating collaboration as well as identifying common approaches and challenges to exchanging information.

� NIEM is Not:– The actual exchange of information. NIEM describes the data that is in

motion in the exchange.– A database or system. NIEM does not store information.– Intrusive to existing systems. NIEM allows organizations to move information

quickly and effectively without rebuilding systems.– Software.– A technology stack. NIEM is technology agnostic and addresses the data

layer, which means you can use NIEM irrespective of the particular technologies used within an organization.

– Just for law enforcement and justice. Fourteen domains participate in NIEM with additional domains forming.

– Strictly for federal government. NIEM is used in all 50 states, as well as local and tribal governments and private industry.

– Limited by national borders. NIEM is used internationally.”

Source: NIEM Website

Page 92: Tutorial on Taffic Management and AI

Common Alerting Protocol (CAP)

� “The Common Alerting Protocol (CAP) is an XML-based data format for exchanging public warnings and emergencies between alerting technologies.– Alerts from the United States Geological Survey, the Department of Homeland Security, NOAA and

<?xml version = "1.0" encoding = "UTF-8"?><alert xmlns = "urn:oasis:names:tc:emergency:cap:1.2"><identifier>43b080713727</identifier><sender>[email protected]</sender><sent>2003-04-02T14:39:01-05:00</sent><status>Actual</status><msgType>Alert</msgType><scope>Public</scope> <info><category>Security</category><event>Homeland Security Advisory System Update</event><urgency>Immediate</urgency><severity>Severe</severity><certainty>Likely</certainty><senderName>U.S. Government, Department of Homeland Security</senderName>

© 2010 IBM Corporation

of Homeland Security, NOAA and the California Office of Emergency Services can all be received in the same format, by the same application.

– The CAP data structure is backward-compatible with existing alert formats including the Specific Area Message Encoding(SAME) used in Weatheradio and the broadcast Emergency Alert System as well as new technology such as theCommercial Mobile Alert System (CMAS)”.

Acknowledgement: Wiki - http://en.wikipedia.org/wiki/Common_Alerting_Protocol

<senderName>U.S. Government, Department of Homeland Security</senderName><headline>Homeland Security Sets Code ORANGE</headline><description>The Department of Homeland Security has elevated the Homeland Security

Advisory System threat level to ORANGE / High in response to intelligence which may indicate a heightened threat of terrorism.</description><instruction> A High Condition is declared when there is a high risk of terrorist attacks. In

addition to the Protective Measures taken in the previous Threat Conditions, Federal departments and agencies should consider agency-specific Protective Measures in accordance with their existing plans.</instruction><web>http://www.dhs.gov/dhspublic/display?theme=29</web><parameter><valueName>HSAS</valueName><value>ORANGE</value></parameter><resource><resourceDesc>Image file (GIF)</resourceDesc><mimeType>image/gif</mimeType><uri>http://www.dhs.gov/dhspublic/getAdvisoryImage</uri></resource><area><areaDesc>U.S. nationwide and interests worldwide</areaDesc></area></info></alert>

Page 93: Tutorial on Taffic Management and AI

Public Transportation: General Transit Feed Specification (GTFS)

� Proposed by Google– Started in 2006– 2009: moves from Google-specific to general– 2011: supports real-time update

� Purpose: public transportation schedules and associated geographic information

� Format is useful even if the data is not shared with Google

© 2010 IBM Corporation93

Source: https://developers.google.com/transit/gtfs/examples/gtfs-feed Source: Making Public Transportation Schedule Information Consumable for Improved Decision

Making, Raj Gupta, Biplav Srivastava, Srikanth Tamilselvam, In 15th International IEEE Annual Conference on Intelligent Transportation Systems (ITSC 2012), Anchorage, USA

Page 94: Tutorial on Taffic Management and AI

General Transit Feed Specification (GTFS)

Filename Required Defines

agency.txt Required One or more transit agencies that provide the data in this feed.

stops.txt Required Individual locations where vehicles pick up or drop off passengers.

routes.txt Required Transit routes. A route is a group of trips that are displayed to riders as a single service.

trips.txt Required Trips for each route. A trip is a sequence of two or more stops that occurs at specific time.

stop_times.txt Required Times that a vehicle arrives at and departs from individual stops for each trip.

calendar.txt Required Dates for service IDs using a weekly schedule. Specify when service starts

© 2010 IBM Corporation94

and ends, as well as days of the week where service is available.

calendar_dates.txt

Optional Exceptions for the service IDs defined in the calendar.txt file. If calendar_dates.txt includes ALL dates of service, this file may be specified instead of calendar.txt.

fare_attributes.txt Optional Fare information for a transit organization's routes.

fare_rules.txt Optional Rules for applying fare information for a transit organization's routes.

shapes.txt Optional Rules for drawing lines on a map to represent a transit organization's routes.

frequencies.txt Optional Headway (time between trips) for routes with variable frequency of service.

transfers.txt Optional Rules for making connections at transfer points between routes.

feed_info.txt Optional Additional information about the feed itself, including publisher, version, and expiration information.

Source: https://developers.google.com/transit/gtfs/

Page 95: Tutorial on Taffic Management and AI

Recommendation on Standards

� DATEX2–Use terms from Datex 2

• Impact, Measured Data, Elaborated Data are directly applicable• Evaluate others

–Use traffic state from Impact • Can be rule based using flow and/or speed.

� NIEM

© 2010 IBM Corporation

� NIEM–Align time, data, organizations, address, …, based on NIEM, if in North America

–Consider adopting NIEM in other regions, depending on relevance• Address needs localization

� CAP–Support CAP format as actual data exchange

� GTFS–Adopt as much as relevant

Page 96: Tutorial on Taffic Management and AI

References

� Smarter city data model standards landscape, Part 2: Transportation– http://www.ibm.com/developerworks/industry/library/ind-smartercitydatamodel2/

� CAP: http://docs.oasis-open.org/emergency/cap/v1.2/CAP-v1.2-os.html

� Datex 2: www.datex2.eu/

© 2010 IBM Corporation

� NIEM: www.niem.gov

�GTFS: https://developers.google.com/transit/gtfs/

� SCRIBE: http://researcher.watson.ibm.com/researcher/view_project.php?id=2505

Page 97: Tutorial on Taffic Management and AI

Addendum: Road Conditions

© 2010 IBM Corporation

97

Page 98: Tutorial on Taffic Management and AI

Road Surface Conditions

� Surface Condition� Boggy Conditions

An unsealed road surface that is wet and soft, which may lead to a vehicle becoming bogged. Can also apply to loose dry conditions.� Bulldust

A very fine material with a flour like texture often covering surface irregularities or deep holes.� Changing Surface Conditions

The condition of the road surface may change along the length of the road, and following road maintenance activities. Wet weather may also cause a localised change in road condition.

� CorrugationsAn unsealed road surface having ripples or undulations along the road.

� FloodwayA localised section of road designed to accommodate temporary overtopping allowing water to flow over the surface of the road. Refer also “Stream Crossing”.

� Loose SurfaceA layer of unbound or coarse material on the pavement surface. Can be very fine material (bull dust or sand), or large material in

© 2010 IBM Corporation

A layer of unbound or coarse material on the pavement surface. Can be very fine material (bull dust or sand), or large material in coarse gravel pavements.

� PotholesBowl-shaped depressions in the road surface, resulting from the loss of pavement material.

� RoughThe consequence of irregularities, uneven in nature, in the longitudinal profile of a road with respect to the intended profile.Interpretation of conditions will not allow vehicles to use the road at customary speeds.

� RuttingA longitudinal deformation of a pavement surface formed by the wheels of vehicles. May be on a sealed or unsealed road.

� Slippery SurfaceSurface becomes slippery in wet conditions or from fuel or material spills.

� Stream CrossingNatural watercourses that cross the road alignment. On sealed roads the crossing would generally be a bridge, floodway or culvert. On unsealed roads there may be no structure.Water levels at floodways and stream crossings are likely to change rapidly and may present an extreme hazard. The ability to cross these floodways and streams will depend on depth of water, velocity of flow, driver experience and vehicle type. ALWAYS assess conditions prior to crossing or travelling along the affected section. See also “Water over Road”.

� WashoutsSteep, irregularly sided grooves in the road surface caused by erosion of the road surface by water.

98http://www.ntlis.nt.gov.au/roadreport/include.jsp?pageName=term_files/index

Page 99: Tutorial on Taffic Management and AI

Road Types

� Road Types

� FormedAn unsealed road that has been constructed to above the natural surface using local materials. Generally of a higher standard than unformed.

� GravelledAn unsealed road that has been formed and strengthened with a good quality gravel material. Generally of a higher standard than formed.

� SealedA road that is constructed with a bitumen surface.

� Unformed

© 2010 IBM Corporation

� UnformedAn unsealed road that is generally a flat track following the natural terrain. Often occurs as a rough track with two wheel paths, and close vegetation.

� UnsealedA road without a bituminous surface that could be gravelled, formed or unformed.

99

http://www.ntlis.nt.gov.au/roadreport/include.jsp?pageName=term_files/index

Page 100: Tutorial on Taffic Management and AI

Types of Restrictions

� DetourAn alternative route available for traffic during a temporary closure of a road, or part of that road. Detours may be sealed or unsealed.

� High ClearanceA light vehicle with clearance sufficient to travel over rough and uneven road surfaces. The clearance on these vehicles generally exceeds common passenger vehicles.

� High Clearance 4WDA light vehicle with two axles that is built for on and off road use and clearance sufficient to travel over rough and uneven road surfaces. The clearance on these vehicles generally exceeds common passenger vehicles.

� Impassable

© 2010 IBM Corporation

� ImpassableAccess along the section of road is affected by flooding or other obstructions. Road conditions are likely to change rapidly and may present an extreme hazard. Persons should not attempt to use/access the road.

� Lane ClosureTemporary closure of one or more traffic lanes. Traffic control devices are in place -to enable traffic to use unaffected lanes.

� Road ClosedTemporary closure of the road where passage of motor vehicles is not permitted. Barriers are in place and penalties shall be applied to drivers ignoring the road closure.

� Weight and Vehicle Type RestrictionA restriction placed on heavy vehicles to control axle group loads and/or vehicle size to protect the road during adverse conditions.

� With CautionThe condition of part of the road may have deteriorated so that due care must be exercised by the travelling public.

� 4WDA light vehicle with two axles that is built for on and off road use. All four wheels can be engaged to improve traction in adverse conditions.

100http://www.ntlis.nt.gov.au/roadreport/include.jsp?pageName=term_files/index

Page 101: Tutorial on Taffic Management and AI

Traffic Management and AI

Section: Intelligence

Speakers: Anand Ranganathan, Biplav Srivastava

AAAI 2012 Tutorial

July 2012

© 2012 IBM Corporation

Toronto, Canada

Page 102: Tutorial on Taffic Management and AI

Outline

1. Traffic Management Problem

2. Instrumentation1. Sensing traffic2. Traffic state estimation3. Optimizing and combining sensor data

3. Interconnection1. Middleware 2. Traffic standards

© 2010 IBM Corporation

2. Traffic standards

4. Intelligence

1. Path planning

1. Simple Illustration

2. Path Planning for Individual Vehicles

2. End-user analytics

1. Bus arrival prediction and journey planning, with state-of-art instrumentation

2. Multi-modal journey planning, without sensors

5. Supporting topics1. Traffic Simulators2. Practical considerations for real-world pilots

102

Page 103: Tutorial on Taffic Management and AI

Simple Illustration

© 2010 IBM Corporation103

Page 104: Tutorial on Taffic Management and AI

Description� City’s road network is a 5x5 grid

� Travelers in the city:– A, B, C, and D live as shown on the map– Their workplace is W [2,2]

� Objective:

1.We want to help all the travelers plan their trips to work in the morning, and back home daily using shortest time.

2. If the office can be moved, where should it be moved to reduce the commute time for all employees?

Commute constraints– Speed of all travelers, by default, is 1 hop per 10 minutes.

– Speed is reduced by half if sun on the face while commuting. Hence, in the morning, travelers going east are slowed by half, and in the evening, travelers going west are slowed by half. Some examples:

• Time taken from [0,0] to [0,1] is 20 minutes in the morning, and 10 minutes for the rest of the day

© 2010 IBM Corporation

for the rest of the day• Time taken from [0,1] to [0,0] is 20

minutes in the evening, and 10 minutes for the rest of the day

• Time taken from [0,0] to [ 1,0] remains 10 minutes throughout the day

– If two persons travel on the same road, their speeds reduce by half.

– The two slowing factors work independently. An example is that two vehicles going from [0,0] to [0,1] in the morning will take 40 minutes

Loosely Synchronized Multi-Modal Plans for Traffic Improvement and Commuter Convenience, Biplav Srivastava, Raj Gupta and Nirmit Desai, IBM Research Report 2012

Page 105: Tutorial on Taffic Management and AI

[4,4][4,0]

[2,2]

C (Chumki)D (Deb)

W (Workplace)

D (Car2) time- 60

C (Car2) time- 40

[Coordinated Planning] Total time for all: 200 minutes

© 2010 IBM Corporation

[0,0] [0,4]

A (Alice) B (Bob)

W (Workplace)

A (Car0) time- 60B (Car1) time- 40

Page 106: Tutorial on Taffic Management and AI

[4,4][4,0]

[2,2]

C (Chumki)D (Deb)

W (Workplace)

D (Car2) time- 80

C (Car2) time- 50

[Non Coordinated Planning] Total time for all: 260 minutes

© 2010 IBM Corporation

[0,0] [0,4]

A (Alice) B (Bob)

W (Workplace)

A (Car0) time- 80B (Car1) time- 50

East

North

Page 107: Tutorial on Taffic Management and AI

Path Planning for Individual Vehicles

© 2010 IBM Corporation107

Page 108: Tutorial on Taffic Management and AI

Traffic - Dependent Route Planning

� Definition: Find a path from a source to a destination on a road network with minimum travel time delay and that considers current and future traffic conditions on all road links.

� Needs dynamic time-dependent shortest path computations–Cost of each link is different at different intervals of time, and

© 2010 IBM Corporation

–These time-dependent costs get updated as new traffic information is obtained.

� Challenges related to performance–Need high throughput and low latency

• 100s of users, need response in less than a second

–Scale well as we increase size of road network• 10,000s or 100,000s nodes and links

–Handle real-time updates to road network conditions

108 7/22/2012

Page 109: Tutorial on Taffic Management and AI

Dynamic Time-Dependent Shortest Path

� Previous works have focused either on dynamism or on time-dependency, but not both

� A* adapted for time-dependent shortest path networks (Chabini and Lan)–Assumes First In – First Out rule on each link

• A traveler who travels on a link will reach the end of the link before anyone who departs later from the beginning of the link.

© 2010 IBM Corporation

� Our approach : Modify A* as follows :–As paths are extended with links during the execution of A*, time is advanced and therefore future delay values of links are used as link costs.

– To ensure the FIFO property in a time-dependent network,• If the time interval changes while traveling on a link, new delay value used for the rest of the link.

• A link can be travelled during many time intervals–Update link costs with new traffic information –Use heuristic function as (Euclidean Distance / max speed limit)

109 7/22/2012

Page 110: Tutorial on Taffic Management and AI

Implementing Shortest Path as a Stream Processing Application

� Shortest Path Algorithm implemented as a C++ user defined Operator

� One time input to Operator :–Road Network

� Continuous Updates to Operator:

© 2010 IBM Corporation

� Continuous Updates to Operator:–Stream with Delays on different links in road network–Stream with Queries (containing Origin, Destination, Departure Time)

� Result : Stream containing Links in Shortest Path and Expected Travel Time

� Can Parallelize effectively–By Destination Point–Allows scaling and caching of heuristic function

110 7/22/2012

Page 111: Tutorial on Taffic Management and AI

© 2010 IBM Corporation111 7/22/2012

Page 112: Tutorial on Taffic Management and AI

Experiments on Shortest Path

� Updates to Traffic Conditions– Generated continuously as GPS data is streamed in

– Divided day into 5 minute intervals

– 61422 updates on 11177 links in 1 day

© 2010 IBM Corporation112 7/22/2012

Page 113: Tutorial on Taffic Management and AI

Performance Experiments

� Measured delay and throughput on 10, 20 and 50 machines when there is a burst of 10,000 shortest path queries

� Scales almost linearly as number of machines increases

© 2010 IBM Corporation113 7/22/2012

Page 114: Tutorial on Taffic Management and AI

Evaluating the benefits of doing dynamic time-dependent shortest paths – quantifying the time savings

� How much do we gain from using traffic data in the shortest path calculation?– Compared to path obtained using static speed limits

� Ran shortest path algorithm for 50 different departure times for 500 origin-destination pairs

� Upto 62% decrease in travel time

© 2010 IBM Corporation114 7/22/2012

Page 115: Tutorial on Taffic Management and AI

Number of changes in Shortest Path

� Shortest Path for a certain origin-destination pair changed upto 37 times (out of the the 50 consecutive runs)–30 changes for more than half of the queries

© 2010 IBM Corporation115 7/22/2012

Page 116: Tutorial on Taffic Management and AI

Degree of change of shortest path

� Degree of change = (1 - (number of common links / total number of links in two paths) )*100

� Paths can change by upto 98% between consecutive runs

© 2010 IBM Corporation116 7/22/2012

Page 117: Tutorial on Taffic Management and AI

Visualizing the paths for origin-destination pair with max number of changes

© 2010 IBM Corporation117 7/22/2012

Page 118: Tutorial on Taffic Management and AI

End User Analytics

© 2010 IBM Corporation118

Page 119: Tutorial on Taffic Management and AI

Bus Arrival Prediction and Travel Planner

Business Challenge

�Meanwhile, public transportation represents a major cost for governments and is one of the most visible services available to citizens.

� However, cities mostly lack the tools, information and visibility to understand and optimize how public transportation is performing

© 2010 IBM Corporation

understand and optimize how public transportation is performing and meeting citizen’s needs.

Page 120: Tutorial on Taffic Management and AI

IBM’s Public Transportation Awareness tool

� Continuously analyses data from Automatic Vehicle Location (AVL) systems

� Collects, processes, and visualizes location data of all public transportation vehicles

� Analytic algorithms are optimized for high performance, on-demand access

� Automatically generated transportation routes and stop locations

© 2010 IBM Corporation120

Page 121: Tutorial on Taffic Management and AI

Map & Dashboard

© 2010 IBM Corporation

Page 122: Tutorial on Taffic Management and AI

Bus Delay Overview

© 2010 IBM Corporation

Page 123: Tutorial on Taffic Management and AI

Bus Delay Query http://www.youtube.com/watch?v=VBv5XRGQ7nA

© 2010 IBM Corporation

Page 124: Tutorial on Taffic Management and AI

Bus Stop, Bus Route Queries

© 2010 IBM Corporation

Page 125: Tutorial on Taffic Management and AI

Bus Arrival Prediction

© 2010 IBM Corporation

Page 126: Tutorial on Taffic Management and AI

Dublin Experience

© 2010 IBM Corporation

Page 127: Tutorial on Taffic Management and AI

Dublin Experience – Deployed Dublin Bus RTPI

150 bus routes / 5000 bus stops

600 buses contemporary on routes during the day

20 stops per bus

20*600 = 12,000 predictors per min

© 2010 IBM Corporation

Page 128: Tutorial on Taffic Management and AI

Use Case and Performance

� Deployed in a production environment at the Dublin City Council since Jan 2011

� Run on VMWare Linux RHEL 5.4, 4 cores Intel Xeon© 2.27GHz, 6GB RAM

� Has been running on the average 2-3 months between

© 2010 IBM Corporation

1

� Has been running on the average 2-3 months between updates

� Typically 1 – 3 user sessions running simultaneously between DCC and IBM

Page 129: Tutorial on Taffic Management and AI

Technical Challenges

1. Data diversity, heterogeneity

2. Data accuracy, sparsity

3. Data volume

Routes & maps

Parkingcapacity

SCATS

Induction loop

TimetablesCar

700 intersections

4,000 loop detectors

20,000 tuples / min

© 2010 IBM Corporation

Bus AVL (GPS)

Accessibility

CCTV

Bike

1,000 buses

3,000 GPS / min

200 CCTV cameras

Page 130: Tutorial on Taffic Management and AI

SIRI

VehicleMonitoring

Adapter

GIS

Map bus to route

Bus to route

tracking

Bus KPI: Average bus speed, traffic status, bus outside routes …

Bus to link spatiotemporal aggregations

Link KPI (speed, …)

LinkUpdateLinkUpdate

TMDD

Adapter

Real time Bus Information adapter

PTA Design – Streams Flow Graph

© 2010 IBM Corporation

to route tracking bus outside routes … aggregationsAdapterAdapter

Bus Estimated Arrival Times at Stops

SIRI StopMonitoring

Adapter(*)

(*) Returns StopMonitoring info for all stops

Page 131: Tutorial on Taffic Management and AI

Data

� GPS readings once every 20 seconds, with:

� Time of capture, GPS location, Bus line ID, Bus vehicle ID– Delay information => Calculated by Dublin bus with respect to timetables– Delay information embedded on the GPS readings and available for processing

� Approx 19 million GPS readings per month

© 2010 IBM Corporation

� Also, GPS coordinates of Bus Stops

131

Page 132: Tutorial on Taffic Management and AI

Pre-Processing

� Detect arrival/departure buses– Radius of 20m around the bus stop and track a bus from a bus line.

� Modeling of delays at every bus stop

� Recovering timetables

� Modeling of delay correlation between delay at departure bus stop and arrival bus stop

© 2010 IBM Corporation

� Modeling of delay correlation between delay at departure bus stop and arrival bus stop

� Calculate Travel Times

132

Page 133: Tutorial on Taffic Management and AI

Modeling delays at bus stops

� Delay: bus arrival/departure earlier/later than the expected on the timetables

� Detect when the bus is at a bus stop (within a 20m radius)

� Calculate Histograms– Per hour of the day, – Weekdays, Weekends

� Use of Kernel Density Estimators

© 2010 IBM Corporation133

Page 134: Tutorial on Taffic Management and AI

Modeling delays at bus stops

© 2010 IBM Corporation134

Page 135: Tutorial on Taffic Management and AI

Recovering timetables

© 2010 IBM Corporation135

Page 136: Tutorial on Taffic Management and AI

End to end trip times

� Estimate distributions of travel times from A to B on the network

� As a function of:

© 2010 IBM Corporation

� As a function of:

� Buses early/late/missing connections => how the trip changes and probabilities

136

Page 137: Tutorial on Taffic Management and AI

Scenario and Simulation

© 2010 IBM Corporation137

Page 138: Tutorial on Taffic Management and AI

Results

© 2010 IBM Corporation

� Expected value differs by 10 minutes of the deterministic trip time

� Variance is also large

� Travel time distribution mean and variance is function of the time of the day138

Page 139: Tutorial on Taffic Management and AI

Multi-modal journey planning, without sensors

Based on joint work by Raj Gupta, Srikanth

© 2010 IBM Corporation139

Based on joint work by Raj Gupta, Srikanth Tamilselvam, Biplav Srivastava

Making Public Transportation Schedule Information Consumable for Improved Decision Making, Raj Gupta, Biplav Srivastava, Srikanth Tamilselvam, In 15th International IEEE Annual Conference on Intelligent Transportation Systems (ITSC 2012), Anchorage, USA, Sep 16-19, 2012.

Page 140: Tutorial on Taffic Management and AI

Problem

� Input: – A person wants to travel from place A to B

� Invariant Inputs: – Mode and mode categories for commuting

• The city has taxis, buses, metros, autos, rickshaws• Buses and metros have published routes, frequency and stops. No other instrumentation.• Autos and rickshaws can be available at stands, or opportunistically, on the road• Taxis can be ordered over the phone

© 2010 IBM Corporation

– Preferences over commuting modes and categories• The person can have a vehicle (e.g., car), • Can walk short distances• Has preferences over commuting modes based on cost, time, safety, physical effort.

� Output– Suggest to the person which mode or combination of modes to select

�Observation: there is no coordination assumed from the transport operators

Page 141: Tutorial on Taffic Management and AI

Multi-Mode Commuting Recommender in Delhi And Bangalore, India

© 2010 IBM Corporation141

Highlights

• Published data of multiple authorities used; repeatable process

•Multiple modes searched

• Preference over modes, time, hops and number of choices supported; more extensions, like fare possible

Page 142: Tutorial on Taffic Management and AI

Technical Challenges in Data Processing

� Factors which affect/ can improve accuracy– Quality of schedule published by public transportation operators (bus and metro for us)

• Names, spelling and conventions in stops by different agencies• We correct and can do more - If we correct too much, we remove the traceability to original published schedules

– Lack of co-relationship across stop names and location• Affects what the user sees when they select• We can include geo-spatial analysis when we offer choice of

locations

– Increase inter-operability across agencies

© 2010 IBM Corporation

– Increase inter-operability across agencies• Make traffic data into linked open data format

– Integrate with geo-spatial analysis of tools

142

Page 143: Tutorial on Taffic Management and AI

References

� Ugur Demiryurek, Farnoush Banaei-Kashani, Cyrus Shahabi and Anand Ranganathan. Online Computation of Fastest Path in Time-Dependent Spatial Networks. In 12th International Symposium on Spatial and Temporal Databases (SSTD 2011), Aug 24-26, 2011, Minneapolis, MN

� Baris Gu�c, Anand Ranganathan. Real-Time, Scalable Route Planning Using a Stream-Processing Infrastructure. In IEEE Intelligent Transportation Systems Conference, Sept 20-22, 2010, Madeira, Portugal

� A. Botea, M. Mueller, and J. Schaeffer. Near optimal hierarchical path-finding. Journal of Game Development, 1:7–28, 2004.

� I. Chabini. Discrete dynamic shortest path problems in transportation applications: Complexity and algorithms with optimal run time.Transportation Research Records, 1645:170–175, 1998.

� I. Chabini and S. Lan. Adaptations of the A* algorithm for the computation of fastest paths in deterministic discrete-time dynamic networks. IEEE Transactions on Intelligent Transportation Systems,3(1):60–74, 2002.

� R. Dechter and J. Pearl. Generalized best-first search strategies and the optimality of A*. JOURNAL OF THE ACM, 32(3):505–536, 1985.

� B. Ding, J. X. Yu, and L. Qin. Finding time-dependent shortest paths over large graphs. In EDBT ’08: Proceedings of the 11th international conference on Extending database technology, pages 205–216, New York, NY, USA, 2008. ACM.

© 2010 IBM Corporation

conference on Extending database technology, pages 205–216, New York, NY, USA, 2008. ACM.

� S. Ganugapati. Parallel algorithms for dynamic shortest path problems. International Transactions in Operational Research, 9:279–302(24), May 2002

� E. Kanoulas, Y. Du, T. Xia, and D. Zhang. Finding fastest paths on a road network with speed patterns. In ICDE ’06: Proceedings of the 22nd International Conference on Data Engineering, page 10, Washington, DC, USA, 2006. IEEE Computer Society.

� D. Kotzinos. A massively parallel time-dependent least-time-path algorithm for intelligent transportation systems applications. Computer Aided Civil and Infrastructure Engineering, 16:337–346(10), September 2001.

� C.-C. Lee, Y.-H. Wu, and A. L. P. Chen. Continuous evaluation of fastest path queries on road networks. In SSTD, pages 20–37, 2007.

� A. Orda and R. Rom. Shortest-path and minimum-delay algorithms in networks with time-dependent edge-length. J. ACM, 37(3):607–625,1990.

� Y. Tian, K. C. K. Lee, and W.-C. Lee. Monitoring minimum cost paths on road networks. In GIS ’09: Proceedings of the 17th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, pages 217–226, New York, NY, USA, 2009.ACM.

143

Page 144: Tutorial on Taffic Management and AI

References

� L. Gasparini, E. Bouillet, F. Calabrese, O. Verscheure, Brendan O’Brien, Maggie O’Donnell, "System and Analytics for Continuously Assessing Transport Systems from Sparse and Noisy Observations: Case Study in Dublin", ITSC 2011

� A. Baptista, E. Bouillet, F. Calabrese, O. Verscheure, "Towards Building an Uncertainty-aware Multi-Modal Journey Planner", ITSC 2011

� A. Ozal, A. Ranganathan, N. Tatbul, "Real-time Route Planning with Stream Processing Systems: A Case Study for the City of Lucerne", ACM SIGSPATIAL International Workshop on GeoStreaming (IWGS'11), Chicago, IL, USA, November 2011

� Making Public Transportation Schedule Information Consumable for Improved Decision Making, Raj Gupta, BiplavSrivastava, Srikanth Tamilselvam, In 15th International IEEE Annual Conference on Intelligent Transportation Systems (ITSC 2012), Anchorage, USA, Sep 16-19, 2012.

© 2010 IBM Corporation

� Loosely Synchronized Multi-Modal Plans for Traffic Improvement and Commuter Convenience, Biplav Srivastava, Raj Gupta and Nirmit Desai, IBM Research Report, 2012.

144

Page 145: Tutorial on Taffic Management and AI

Tutorial: Traffic Management and AI

Section: Supporting Topics

Biplav Srivastava, Anand RanganathanIBM Research

AAAI 2012 Tutorial

22nd July 2011

© 2012 IBM Corporation

Toronto, Canada

Page 146: Tutorial on Taffic Management and AI

Outline

1. Traffic Management Problem

2. Instrumentation1. Sensing traffic2. Traffic state estimation3. Optimizing and combining sensor data

3. Interconnection1. Middleware 2. Traffic standards

© 2010 IBM Corporation

2. Traffic standards

4. Intelligence1. Path planning

1. Simple Illustration2. Path Planning for Individual Vehicles

2. End-user analytics1. Bus arrival prediction and journey planning, with state-of-art instrumentation2. Multi-modal journey planning, without sensors

5. Supporting topics

1. Traffic Simulators

2. Practical considerations for real-world pilots

146

Page 147: Tutorial on Taffic Management and AI

Why Use Traffic Simulators

� Use them to prepare for real-world experience

� Overcome– Data issues– Try what-ifs– Cost reasons

� No simulator is a replacement for actual deployment

© 2010 IBM Corporation

� No simulator is a replacement for actual deployment

147

Page 148: Tutorial on Taffic Management and AI

Traffic Simulators

Availability LicensingProgramming

language

Code

Extension

Development

Started

Current

status

Macro /

micro traffic

flow

Location

Dynamit Academiamit.edu/its/dyn

amit.html

Freesim Open Source GNU java Possible 2004last release 25-7-2011

bothhttp://www.freewaysimulator.com/index.html

Sumo Open Source SourceForge C++ Possible 2002 upto date microhttp://sumo.sourceforge.net/

Microsimulation

of Road Traffic

Flow

Academia java applet PossibleLast release June 2011

microhttp://www.traffic-simulation.de/

Credit: Table compiled by Raj Gupta, IBM Research

© 2010 IBM Corporation

TraNS Academia 2007 2008http://lca.epfl.ch/projects/trans

Megaffic IBM

http://www.research.ibm.com/trl/projects/socsim/project.htm

MITSIMlab Open Source Free to use Possible 1999No support since 2005

micromit.edu/its/mits

imlab.html

TransmodelerSingle User licence

10 K micro

http://www.caliper.com/TransModeler/default.htm

Matsim Open source GNU Java Possible 2003last release 28-3-2011

macrohttp://www.matsim.org/

148

Page 149: Tutorial on Taffic Management and AI

Matsim

� Agent-Based Transport Simulations

� Key Features of MATSim– Fast Dynamic and Agent-Based Traffic SimulationSimulate whole days within minutes It’s not time based.

– Supports Large ScenariosMATSim can simulate millions of agents or huge, detailed networks

© 2010 IBM Corporation

MATSim can simulate millions of agents or huge, detailed networks – Versatile Analyses and Simulation OutputE.g. compare simulated data to real-world counting stations

– Modular ApproachEasily extended with your own algorithms

– Sophisticated Interactive VisualizerSee what each agent is doing during the simulation

– Open SourceYou get the Java Source Code, which runs on all major operating systems

� Easy integration with Open Street Maps

Page 150: Tutorial on Taffic Management and AI

Matsim: Input and Output

Matsim

Network<node id="23" x="6900" y="0"/><link id="0" from="0" to="1" length="300" capacity="60" freespeed="30" permlanes="60" />

Plan<person id="1"> <plan> <act type="h" x="0" y="0" end_time="00:18" /><leg mode="car"> </leg><act type="h" x="6900" y="0" />

© 2010 IBM Corporation

Matsim<act type="h" x="6900" y="0" /></plan>

</person>

Network_Change<networkChangeEvent startTime="02:07:00" ><link refId="0"/><freespeed type="scaleFactor" value="0.99"/>

</networkChangeEvent>

Events<event time="1080.0" type="actend" person="1" link="0" actType="h" /><event time="1080.0" type="departure" person="1" link="0" legMode="car" /><event time="1080.0" type="wait2link" person="1" link="0" /><event time="1081.0" type="left link" person="1" link="0" /><event time="1081.0" type="entered link" person="1" link="2" /><event time="1092.0" type="left link" person="1" link="2" />

Page 151: Tutorial on Taffic Management and AI

Practical Considerations

© 2010 IBM Corporation151

Page 152: Tutorial on Taffic Management and AI

High-level Suggestions

� Follow and understand the data– Problem characteristic depends on it– Relevance of algorithms depends on it– Result quality depends on it

� Prototype early– Optionally, in a simulator, and

© 2010 IBM Corporation

– Optionally, in a simulator, and– Definitely, in real-world– No solution works unless millions are using it

� Think long term– Traffic systems last for decades, not 2-3 years (typical life of a computer)– Support inter-operability in IT as well as in users– Quantify benefits and costs using standard metrics– Always think of people issues

• Good to have analogies with computer networks, but remember that packets can be dropped while travelling, but not people

• If people adopt, an imperfect solution will still be successful

152

Page 153: Tutorial on Taffic Management and AI

Experience in Dublin, Ireland

� Close collaboration with the city– Jointly funded research center

� Access to a number of live data feeds– GPS from buses, loop, traffic signal cycles, CCTV

© 2010 IBM Corporation

� City was willing to try out research prototypes

� Goal was to improve different aspects of transportation

153

Page 154: Tutorial on Taffic Management and AI

Experience in Stockholm, Sweden

� Close collaboration with KTH and with Trafik Stockholm– Research funding

� Access to taxi GPS data and to “MCS” data (overhead microwave sensors)– Through a 3rd party collector– Privacy restrictions limited completeness of data

© 2010 IBM Corporation

– Privacy restrictions limited completeness of data

� Ongoing research collaboration

� Key goal is to improve travel time estimation to various points in the city

154

Page 155: Tutorial on Taffic Management and AI

Experience in New York, USA

� Research Collaboration with CUNY and NYU

� Access to traffic data (only start and stop information)

� Privacy restrictions limited identification information

� Very limited in terms of ability to extract traffic information

© 2010 IBM Corporation

� Very limited in terms of ability to extract traffic information

� However, can still extract human mobility patterns

155

Page 156: Tutorial on Taffic Management and AI

Experience in Boston, USA

� Smarter Cities Challenge (SCC), an IBM philanthropic initiative to bring IBM's experts to cities around a topic and get a recommendation on what the city needs to do. – SCC in Boston was on traffic in June 2012 for 3 weeks. – The project has been unique for two reasons: (a) academia has been involved and (b) the team does a running prototype along with a recommendation.

� Boston collects a significant amount of data from many sources that could be quite useful to researchers, developers, transportation engineers, urban planners, and above all, citizens.– Siloed, in multiple formats and not fully exploited.

© 2010 IBM Corporation

– Siloed, in multiple formats and not fully exploited. – Developed a prototype that created a standards-based common model, loaded > 1M data from 3 sensor types already in city, and showed the potential of insights from them; also recommendations for more sensor types and analytics

156

Press on the IBM SCC Boston team work:

1. Boston Globe, June 29, 2012http://www.boston.com/business/technology/articles/2012/06/29/ibm_gives_advice_on_how_to_fix_boston_traffic__first_get_an_app/

2. Popular Science, 2 July 2012http://www.popsci.com/technology/article/2012-07/bostons-ibm-built-traffic-app-merges-multiple-data-streams-predict-ease-congestion

3. Others: National Public Radio (USA), and a range of local TV stations on the work.

Team at work – Source: Boston Globe article

Page 157: Tutorial on Taffic Management and AI

More Attempts

� Around the World– Level of instrumentation can vary drastically, or even be absent – Ideas needed to incentivize people to reduce demand– Funding for large-scale supply (infrastructure) increase is reducing

� Beyond city bodies– Collaboration with mobile companies

© 2010 IBM Corporation

– Collaboration with mobile companies– Collaboration with bus companies– …

� And by individual organizations. Any can take a lead to reduce traffic demand• Car-pooling. [Making Car Pooling Work – Myths and Where to Start, Biplav Srivastava, in 19th World Congress on Intelligent Transportation Systems, Vienna, Austria, 2012]

157