big data “triage” for long range planning transportation engineering and safety conference...
TRANSCRIPT
![Page 1: Big Data “Triage” for Long Range Planning Transportation Engineering and Safety Conference Reuben S MacMartin December 12, 2014](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649ea85503460f94bac56e/html5/thumbnails/1.jpg)
Big Data “Triage” for Long Range Planning
Transportation Engineering and Safety Conference
Reuben S MacMartinDecember 12, 2014
![Page 2: Big Data “Triage” for Long Range Planning Transportation Engineering and Safety Conference Reuben S MacMartin December 12, 2014](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649ea85503460f94bac56e/html5/thumbnails/2.jpg)
Delaware Valley Regional Planning Commission
Metropolitan Planning Organization (MPO) 2 States 9 Counties 351 Municipalities 5.6 Million Population 3,800 sq. miles ~115 employees
Activities – Long Range Plan (LRP) Transportation Improvement Program (TIP) Wide range of planning and technical support for
regional partners
![Page 3: Big Data “Triage” for Long Range Planning Transportation Engineering and Safety Conference Reuben S MacMartin December 12, 2014](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649ea85503460f94bac56e/html5/thumbnails/3.jpg)
Outline
What we use data for? Traditional data sources – traffic counts,
surveys, demographic data The old-new – OSM, GTF, VPP Suite,
Bluetooth The new-new – CycleTracks, real-time
transit data,…, data-mined GPS data, etc.
![Page 4: Big Data “Triage” for Long Range Planning Transportation Engineering and Safety Conference Reuben S MacMartin December 12, 2014](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649ea85503460f94bac56e/html5/thumbnails/4.jpg)
What do we use data for?
Current conditions on transportation studies
![Page 5: Big Data “Triage” for Long Range Planning Transportation Engineering and Safety Conference Reuben S MacMartin December 12, 2014](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649ea85503460f94bac56e/html5/thumbnails/5.jpg)
Current Conditions
![Page 6: Big Data “Triage” for Long Range Planning Transportation Engineering and Safety Conference Reuben S MacMartin December 12, 2014](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649ea85503460f94bac56e/html5/thumbnails/6.jpg)
![Page 7: Big Data “Triage” for Long Range Planning Transportation Engineering and Safety Conference Reuben S MacMartin December 12, 2014](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649ea85503460f94bac56e/html5/thumbnails/7.jpg)
What do we use data for?
Current conditions on transportation studies
Definition and analysis of congestion for the Congestion Management Process (CMP)
![Page 8: Big Data “Triage” for Long Range Planning Transportation Engineering and Safety Conference Reuben S MacMartin December 12, 2014](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649ea85503460f94bac56e/html5/thumbnails/8.jpg)
A bad day compared to average
![Page 9: Big Data “Triage” for Long Range Planning Transportation Engineering and Safety Conference Reuben S MacMartin December 12, 2014](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649ea85503460f94bac56e/html5/thumbnails/9.jpg)
A bad day compared to average
![Page 10: Big Data “Triage” for Long Range Planning Transportation Engineering and Safety Conference Reuben S MacMartin December 12, 2014](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649ea85503460f94bac56e/html5/thumbnails/10.jpg)
What do we use data for?
Current conditions on transportation studies
Definition and analysis of congestion for the Congestion Management Process (CMP)
Long Range Planning
![Page 11: Big Data “Triage” for Long Range Planning Transportation Engineering and Safety Conference Reuben S MacMartin December 12, 2014](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649ea85503460f94bac56e/html5/thumbnails/11.jpg)
Long Range Planning
![Page 12: Big Data “Triage” for Long Range Planning Transportation Engineering and Safety Conference Reuben S MacMartin December 12, 2014](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649ea85503460f94bac56e/html5/thumbnails/12.jpg)
Long Range Planning
![Page 13: Big Data “Triage” for Long Range Planning Transportation Engineering and Safety Conference Reuben S MacMartin December 12, 2014](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649ea85503460f94bac56e/html5/thumbnails/13.jpg)
What do we use data for?
Current conditions on transportation studies
Definition and analysis of congestion for the Congestion Management Process (CMP)
Long Range Planning Calibration and validation of travel
forecasting models250 Riders in 2040
![Page 14: Big Data “Triage” for Long Range Planning Transportation Engineering and Safety Conference Reuben S MacMartin December 12, 2014](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649ea85503460f94bac56e/html5/thumbnails/14.jpg)
Also a data provider – eg. RIMIS
![Page 15: Big Data “Triage” for Long Range Planning Transportation Engineering and Safety Conference Reuben S MacMartin December 12, 2014](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649ea85503460f94bac56e/html5/thumbnails/15.jpg)
“Traditional” Planning Data Sources Inventories
Traffic counts (78,300+) Bike and Ped counts (1000+) Travel time surveys
Behavioral Surveys Household travel survey (2012-2013) Transit on-board (2010-2012)
Demographic Data Census, American Community Survey National Employment Time Series (NETS)
![Page 16: Big Data “Triage” for Long Range Planning Transportation Engineering and Safety Conference Reuben S MacMartin December 12, 2014](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649ea85503460f94bac56e/html5/thumbnails/16.jpg)
The old “new” data
These were innovative 5 years ago – Open source data for our travel demand
model networks
![Page 17: Big Data “Triage” for Long Range Planning Transportation Engineering and Safety Conference Reuben S MacMartin December 12, 2014](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649ea85503460f94bac56e/html5/thumbnails/17.jpg)
Travel Demand Model Networks The need:
Accurate representations of regional highway and transit networks
The past: “hand” code from paper maps, schedules,
etc. or, combine a multitude of different data
sources The innovation:
Fuse OpenStreetMap (OSM) and GTF (i.e. “Google-transit”) and add extra data for modeling
![Page 18: Big Data “Triage” for Long Range Planning Transportation Engineering and Safety Conference Reuben S MacMartin December 12, 2014](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649ea85503460f94bac56e/html5/thumbnails/18.jpg)
Open Data Mash-up for Transportation Modeling
Data integration Data objects
of different origin are merged
New relationships are created
from OSM
Stop Point
Number
Line
Name
Service Pattern
Line NameRoute NameDirection
Scheduled Run
Line NameRoute NameDirectionIndex
Travel DemandData
Stop Area
Number
from GTFS
Node
Number
Link
From NodeTo Node
2
1 or more
0 or more
Exactly 1
Legend
Connector
Zone NumberNode NumberDirection
Zone
Zone Number
![Page 19: Big Data “Triage” for Long Range Planning Transportation Engineering and Safety Conference Reuben S MacMartin December 12, 2014](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649ea85503460f94bac56e/html5/thumbnails/19.jpg)
Integrated Street & Transit Network
© in part by OSM and CC-by-SA
![Page 20: Big Data “Triage” for Long Range Planning Transportation Engineering and Safety Conference Reuben S MacMartin December 12, 2014](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649ea85503460f94bac56e/html5/thumbnails/20.jpg)
TIM 2 Highway Network
© in part by OSM and CC-by-SA
New, accurate topology (& routable) Legacy DVRPC network model
![Page 21: Big Data “Triage” for Long Range Planning Transportation Engineering and Safety Conference Reuben S MacMartin December 12, 2014](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649ea85503460f94bac56e/html5/thumbnails/21.jpg)
Original SEPTA GTFS (2010)
![Page 22: Big Data “Triage” for Long Range Planning Transportation Engineering and Safety Conference Reuben S MacMartin December 12, 2014](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649ea85503460f94bac56e/html5/thumbnails/22.jpg)
VISUM Imported Network
![Page 23: Big Data “Triage” for Long Range Planning Transportation Engineering and Safety Conference Reuben S MacMartin December 12, 2014](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649ea85503460f94bac56e/html5/thumbnails/23.jpg)
VISUM Exported Network(WKTPoly shape)
![Page 24: Big Data “Triage” for Long Range Planning Transportation Engineering and Safety Conference Reuben S MacMartin December 12, 2014](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649ea85503460f94bac56e/html5/thumbnails/24.jpg)
The old “new” data
These were innovative 5 years ago – Open source data for our travel demand
model networks Bluetooth detectors for speed and O-D
data
![Page 25: Big Data “Triage” for Long Range Planning Transportation Engineering and Safety Conference Reuben S MacMartin December 12, 2014](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649ea85503460f94bac56e/html5/thumbnails/25.jpg)
![Page 26: Big Data “Triage” for Long Range Planning Transportation Engineering and Safety Conference Reuben S MacMartin December 12, 2014](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649ea85503460f94bac56e/html5/thumbnails/26.jpg)
The old “new” data
These were innovative 5 years ago – Open source data for our travel demand
model networks Bluetooth detectors for speed and O-D
data Automated Passenger Counter (APC)
data - SEPTA
![Page 27: Big Data “Triage” for Long Range Planning Transportation Engineering and Safety Conference Reuben S MacMartin December 12, 2014](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649ea85503460f94bac56e/html5/thumbnails/27.jpg)
Why APC data?
Time stamped boarding and alighting data by line by stop
Time period level targets for modeling Stop and line level expansion values for
On Board Survey work Used in calibration/validation of path
builder Transit studies: O-D matrices by line
![Page 28: Big Data “Triage” for Long Range Planning Transportation Engineering and Safety Conference Reuben S MacMartin December 12, 2014](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649ea85503460f94bac56e/html5/thumbnails/28.jpg)
The new “new” data
User-sourced bike data - CyclePhilly
![Page 29: Big Data “Triage” for Long Range Planning Transportation Engineering and Safety Conference Reuben S MacMartin December 12, 2014](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649ea85503460f94bac56e/html5/thumbnails/29.jpg)
CyclePhilly – User Generated GPS Data
![Page 30: Big Data “Triage” for Long Range Planning Transportation Engineering and Safety Conference Reuben S MacMartin December 12, 2014](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649ea85503460f94bac56e/html5/thumbnails/30.jpg)
www.cyclephilly.org
![Page 31: Big Data “Triage” for Long Range Planning Transportation Engineering and Safety Conference Reuben S MacMartin December 12, 2014](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649ea85503460f94bac56e/html5/thumbnails/31.jpg)
Raw GPS Trace
Snapped GPS Model Path Model vs. Data
![Page 32: Big Data “Triage” for Long Range Planning Transportation Engineering and Safety Conference Reuben S MacMartin December 12, 2014](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649ea85503460f94bac56e/html5/thumbnails/32.jpg)
The new “new” data
User-sourced bike data – CyclePhilly Vehicle probe data – INRIX
![Page 33: Big Data “Triage” for Long Range Planning Transportation Engineering and Safety Conference Reuben S MacMartin December 12, 2014](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649ea85503460f94bac56e/html5/thumbnails/33.jpg)
PM Peak TTI – INRIX
![Page 34: Big Data “Triage” for Long Range Planning Transportation Engineering and Safety Conference Reuben S MacMartin December 12, 2014](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649ea85503460f94bac56e/html5/thumbnails/34.jpg)
Archived Operational Data – INRIX
![Page 35: Big Data “Triage” for Long Range Planning Transportation Engineering and Safety Conference Reuben S MacMartin December 12, 2014](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649ea85503460f94bac56e/html5/thumbnails/35.jpg)
The new “new” data
User-sourced bike data – CyclePhilly Vehicle probe data – INRIX SEPTA Key (new fare payment
technology) data – SEPTA (availability TBD)
![Page 36: Big Data “Triage” for Long Range Planning Transportation Engineering and Safety Conference Reuben S MacMartin December 12, 2014](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649ea85503460f94bac56e/html5/thumbnails/36.jpg)
Fare Card Data – Possibilities Anonymized full day transit-based tour
data for all riders O-D data Route choice data Transfer behavior Frequency of transit use Much higher resolution data than current
survey methods
![Page 37: Big Data “Triage” for Long Range Planning Transportation Engineering and Safety Conference Reuben S MacMartin December 12, 2014](https://reader036.vdocuments.us/reader036/viewer/2022062802/56649ea85503460f94bac56e/html5/thumbnails/37.jpg)
Triage – Making Data Usable
Aggregation – Resolution and limits of existing analytical tools/methods
Cleaning – You can’t check every data point Initial spot check and clean as you go if you
find discrepancies Sampling Biases – Not all big data is
truly random Compare non-random to random sources
whenever possible Declare biases of data when using it