public-transit data extraction from openstreetmap data · openstreetmap data zhiwei zhang february...
TRANSCRIPT
IntroductionBackground
ProcedureEvaluation
Further Research
Public-Transit Data Extraction FromOpenStreetMap Data
Zhiwei Zhang
February 13, 2017
Reviewer: Prof. Dr. Hannah Bast
Supervisor: Prof. Dr. Hannah Bast
Tutor: Patrick BrosiZhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Table of Contents
1 Introduction2 Background3 Procedure
Data ExtractionRepair System
Oder RepairGap RepairClassificationConnection RepairTopological Sort
Generating GTFS Feed4 Evaluation5 Further Research
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Table of Contents
1 Introduction2 Background3 Procedure
Data ExtractionRepair System
Oder RepairGap RepairClassificationConnection RepairTopological Sort
Generating GTFS Feed4 Evaluation5 Further Research
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Introduction
1 OpenStreetMap?
2 Meaning of topic?
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Introduction
1 OpenStreetMap?an open-source project providing free geographic data
2 Meaning of topic?
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Introduction
1 OpenStreetMap?an open-source project providing free geographic data
2 Meaning of topic?
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Introduction
1 OpenStreetMap?an open-source project providing free geographic data
2 Meaning of topic?Provide data-support for route planing softwares
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Table of Contents
1 Introduction2 Background3 Procedure
Data ExtractionRepair System
Oder RepairGap RepairClassificationConnection RepairTopological Sort
Generating GTFS Feed4 Evaluation5 Further Research
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Mapping Relation
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Data ExtractionRepair SystemGenerating GTFS Feed
Table of Contents
1 Introduction2 Background3 Procedure
Data ExtractionRepair System
Oder RepairGap RepairClassificationConnection RepairTopological Sort
Generating GTFS Feed4 Evaluation5 Further Research
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Data ExtractionRepair SystemGenerating GTFS Feed
Brief
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Data ExtractionRepair SystemGenerating GTFS Feed
Data Extraction
OSM Data
Node
Way
Relation
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Data ExtractionRepair SystemGenerating GTFS Feed
Data Extraction
OSM Data
Node
Way
Relation
route=railway/route=rail/
route=train/route=tracks
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Data ExtractionRepair SystemGenerating GTFS Feed
Data Extraction
OSM Data
Node
Way
Relation
Network
Vertex
Edge
Graph
route=railway/route=rail/
route=train/route=tracks
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Data ExtractionRepair SystemGenerating GTFS Feed
Data Extraction
OSM Data
Node
Way
Relation
Network
Vertex
Edge
Graph
route=railway/route=rail/
route=train/route=tracks
in Networkor
railway=rail/railway=
narrow gauge
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Data ExtractionRepair SystemGenerating GTFS Feed
Data Extraction
OSM Data
Node
Way
Relation
Network
Vertex
Edge
Graph
route=railway/route=rail/
route=train/route=tracks
in Networkor
railway=rail/railway=
narrow gauge
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Data ExtractionRepair SystemGenerating GTFS Feed
Data Extraction
OSM Data
Node
Way
Relation
Network
Vertex
Edge
Graph
route=railway/route=rail/
route=train/route=tracks
in Networkor
railway=rail/railway=
narrow gauge
in Network
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Data ExtractionRepair SystemGenerating GTFS Feed
Data Extraction
OSM Data
Node
Way
Relation
Network
Vertex
Edge
Graph
route=railway/route=rail/
route=train/route=tracks
in Networkor
railway=rail/railway=
narrow gauge
in Network
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Data ExtractionRepair SystemGenerating GTFS Feed
Repair System
before: class: unknown
E1
E 2
E5E4E3
after: class: local
E1
E 3
E5 E6
E7E8
E 4
E2
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Data ExtractionRepair SystemGenerating GTFS Feed
Oder Repair - Introduction
: not contained in this graph
: contained in this graph
E4E 1
E2
E3
E5
Original Edge Order (containing fake gaps):
E1 → E2 → E3 ⇢ E4 ⇢ E5
⇢ : means a gap.
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Data ExtractionRepair SystemGenerating GTFS Feed
Order Repair - fixing gap between E4 and E5
E4 E5
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Data ExtractionRepair SystemGenerating GTFS Feed
Order Repair - fixing gap between E4 and E5
bidirectional A*
search for path
E4 E5
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Data ExtractionRepair SystemGenerating GTFS Feed
Order Repair - fixing gap between E4 and E5
bidirectional A*
R-Tree(local edges)
search for neighbors
search for path
E4 E5
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Data ExtractionRepair SystemGenerating GTFS Feed
Order Repair - fixing gap between E4 and E5
bidirectional A*
R-Tree(local edges)
return neighborssearch for neighbors
search for path
E4 E5
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Data ExtractionRepair SystemGenerating GTFS Feed
Order Repair - fixing gap between E4 and E5
bidirectional A*
R-Tree(local edges)
return neighborssearch for neighbors
insert found pathsearch for path
E4
E1
E2 E3 E5
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Data ExtractionRepair SystemGenerating GTFS Feed
Order Repair - fixing gap between E4 and E5
fake gap
real gap
bidirectional A*
R-Tree(local edges)
return neighborssearch for neighbors
insert found pathsearch for path
E4
E1
E2 E3 E5
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Data ExtractionRepair SystemGenerating GTFS Feed
Order Repair - fixing gap between E4 and E5
fake gap
real gap
bidirectional A*
R-Tree(local edges)
return neighborssearch for neighbors
insert found pathsearch for path
E4
E1
E2 E3 E5
Edge Order (after using bidirectional A*):E1 → E2 → E3 → E2 → E1 → E4 → E1 → E2 → E3 ⇢ E5
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Data ExtractionRepair SystemGenerating GTFS Feed
Order Repair - fixing gap between E4 and E5
fake gap
real gap
bidirectional A*
R-Tree(local edges)
return neighborssearch for neighbors
insert found pathsearch for path
E4
E1
E2 E3 E5
Edge Order (after Order Repair):E4 → E1 → E2 → E3 ⇢ E5
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Data ExtractionRepair SystemGenerating GTFS Feed
Oder Repair - Procedure
Algorithm 1: Order RepairInput: a network contains graphs, edges and vertexs
1 foreach graph ∈ network do2 gap list ← GapCheck(graph );3 r-tree ← BuildRTreeUsingLocalEdges(graph );4 foreach gap ∈ gap list do5 GapRepairUsingBiAStar(r-tree, graph, gap );
/* find the first non-repetitive edge and delete edges front
it
make sure the front end-edge is in position */
6 LocateFirstEdge(graph );
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Data ExtractionRepair SystemGenerating GTFS Feed
Oder Repair - Explanation
Questions:1 repetitive edges?
key to keep the local order rightremovable through DFS
2 theoretical basis of LocateFirstEdge?the front end-edge intersects with its neighbors at oneidentical conjoint point
3 why bidirectional A* algorithm?fast and efficientif a complete path doesn’t exist, highly possible to find apartial path to transform a fake gap into a real one
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Data ExtractionRepair SystemGenerating GTFS Feed
Gap Repair - Introduction
: not contained in this graph
: contained in this graph
E4E 1
E2
E3
E5
Original Edge Order (containing real gaps):
E4 → E1 → E2 → E3 ⇢ E5
⇢ : means a gap.
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Data ExtractionRepair SystemGenerating GTFS Feed
Gap Repair - Introduction
E4E 1
E2
E3
E5
path finding
search for path
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Data ExtractionRepair SystemGenerating GTFS Feed
Gap Repair - Introduction
E4
E 1
E2
E3
E5
path finding
R-Tree(global edges)
search for neighbors
search for path
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Data ExtractionRepair SystemGenerating GTFS Feed
Gap Repair - Introduction
E4
E 1
E2
E3
E5
path finding
R-Tree(global edges)
return neighborssearch for neighbors
search for path
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Data ExtractionRepair SystemGenerating GTFS Feed
Gap Repair - Introduction
E4
E 1
E2
E3
E6E5
path finding
R-Tree(global edges)
return neighborssearch for neighbors
insert found pathsearch for path
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Data ExtractionRepair SystemGenerating GTFS Feed
Gap Repair - Introduction
E4E 1
E2
E3
E6E5
Edge Order (after Gap Repair):
E4 → E1 → E2 → E3 → E6 → E5
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Data ExtractionRepair SystemGenerating GTFS Feed
Gap Repair - Procedure
Algorithm 2: Gap RepairInput: a network contains graphs, edges and vertexs
1 gap list dic ← empty dictionary; // key is graph , value is a gap list2 foreach graph ∈ network do3 gap list ← GapCheck(graph );4 gap list dic .Add(graph, gap list );5 r-tree ← BuildRTreeUsingGlobalEdges(network ); // all the edges in network6 foreach (graph, gap list) ∈ gap list dic do7 foreach gap ∈ gap list do
/* GapRepair returns true/false to indicate if a gap is successfully fixed
pathFindingAlgo could be A* Algorithm or Dijkstra’s Algorithm */
8 IsFixed ← GapRepair(r-tree, graph, gap, pathFindingAlgo );9 if ¬IsFixed then
10 break;
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Data ExtractionRepair SystemGenerating GTFS Feed
Classification - Introduction
Graph
service : expressref : ICE255
Graph
service : regionalref : RB1101
Graph
service : nullref : ICE255
Graph
service : nullref : null
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Data ExtractionRepair SystemGenerating GTFS Feed
Classification - Introduction
Graph
service : expressref : ICE255
Graph
service : regionalref : RB1101
Graph
service : nullref : ICE255
Graph
service : nullref : null
long-distancelocal unknown
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Data ExtractionRepair SystemGenerating GTFS Feed
Classification - Introduction
Graph
service : expressref : ICE255
Graph
service : regionalref : RB1101
Graph
service : nullref : ICE255
Graph
service : nullref : null
Dic{ice:long-distance,rb:local }
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Data ExtractionRepair SystemGenerating GTFS Feed
Classification - Introduction
Graph
service : expressref : ICE255
Graph
service : regionalref : RB1101
Graph
service : expressref : ICE255
Graph
service : nullref : null
Dic{ice:long-distance,rb:local }
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Data ExtractionRepair SystemGenerating GTFS Feed
Classification - Introduction
Graph
service : expressref : ICE255
Graph
service : regionalref : RB1101
Graph
service : expressref : ICE255
Graph
service : nullref : null
SVMs-training model
length=500 km length=100 km
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Data ExtractionRepair SystemGenerating GTFS Feed
Classification - Introduction
Graph
service : expressref : ICE255
Graph
service : regionalref : RB1101
Graph
service : expressref : ICE255
Graph
service : regionalref : null
SVMs-predict model
length=90 km
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Data ExtractionRepair SystemGenerating GTFS Feed
Classification - ProcedureAlgorithm 3: ClassificationInput: a network contains graphs, edges and vertexs
1 locals , long distances , unknows ← empty list;2 foreach graph ∈ network do3 if IsLocal(graph ) then4 locals .Add(graph );5 else if IsLongDistance(graph ) then6 long distances .Add(graph );7 else8 unknows .Add(graph );
/* get a dictionary, in which one prefix points to only one class label local or
long-distance */
9 ref predix dic ← SummarizeUniqueRefPredix(locals, long distances );10 PredictUsingDic(ref predix dic, locals, long distances, graph );11 svms ← SVMs();12 svms .Training(locals, long distances ); // train SVMs with the lengths
13 PredictUsingSVMs(svms, graph );
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Data ExtractionRepair SystemGenerating GTFS Feed
Connection Repair - Introduction
E1
E 2 E3 E4
E5
Edge Order:E1 → E2 → E3 → E4 ⇢ E5
Vertex Order:E1: v1 → v2 → v3
E2: v3 → v4 → v5 → v6 → v7
E3: v5 → v8 → v9
E4: v11 → v10 → v9
E5: v13 → v14 → v15
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Data ExtractionRepair SystemGenerating GTFS Feed
Connection Repair - Introduction
E1
E 7
E3 E4
E 6
Edge Order:E1 → E7 → E6 → E3 ⇢ E4
Vertex Order:E1: v1 → v2 → v3
E7: v3 → v4 → v5
E3: v5 → v8 → v9
E4: v9 → v10 → v11
E6: v5 → v6 → v7
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Data ExtractionRepair SystemGenerating GTFS Feed
Connection Repair - Procedure
Algorithm 4: Connection RepairInput: a network contains graphs, edges and vertexs
1 foreach graph ∈ network do2 foreach edge front, edge back ∈ graph do
/* edge front and edge back are index-adjacent in graph */
3 if IsConjoint(edge front, edge back ) then4 if ¬IsHeadTailType(edge front, edge back ) then
/* make sure they are conjoint in head-tail type. If
necessary, separate one of them to achieve it */
5 SepatateEdge(edge front, graph ) orSepatateEdge(edge back, graph );
6 else7 foreach edge ∈ graph ⋀ edge behind edge back do8 graph .DeleteEdge(edge );
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Data ExtractionRepair SystemGenerating GTFS Feed
Topological Sort - Introduction
Problems:repetitive edges exist in graph
edges in graph are poor organized
Aim: each graph has a topologically sorted edge-list.
Algorithm 5: Topological SortInput: a network contains graphs, edges and vertexs
1 foreach graph ∈ network do2 adjacency list ← CreateAdjacencyLists(graph );3 DFS(adjacency list, graph );
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Data ExtractionRepair SystemGenerating GTFS Feed
Generating GTFS Feed - Introduction
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Data ExtractionRepair SystemGenerating GTFS Feed
Generating GTFS Feed - Procedure
Algorithm 6: Generating GTFS FeedInput: a network contains graphs, edges and vertexsOutput: a GTFS feed
1 GTFSSystem← GTFSSystem();2 r-tree ← BuildRTreeUsingGlobalStations(network );3 foreach graph ∈ network do4 route ←Route(graph ); // store the attributes of graph5 GTFSSystem .Add(route );6 railway list← SeparateIntoRailways(graph ); // create several
railways, each of them could store the information of a branch of graph7 foreach railway ∈ railway list do
/* separate the information in a railway into stop s, trip s and shape*/
8 ProcessRailway(GTFSSystem, railway, r-tree );
9 OutputGTFSFeed(GTFSSystem ); // output a GTFS feed using a CSV writer
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Table of Contents
1 Introduction2 Background3 Procedure
Data ExtractionRepair System
Oder RepairGap RepairClassificationConnection RepairTopological Sort
Generating GTFS Feed4 Evaluation5 Further Research
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Evaluation - Part I
Conclusions:
using A* ⇐⇒ usingDijkstraOrder Repair isefficient in removinggaps.
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Evaluation - TRAVIC I
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Evaluation - TRAVIC II
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Evaluation - OTP I
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Evaluation - OTP II
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Evaluation - Part II
Conclusions:
result in TRAVIC is very good.result in OTP is ok. For some routes is fine.
Analysis:
We take the condition into account, that some stops belongto a big station. So the GTFS feed contains this information.OTP doesn’t recognize this information in the GTFS feed. Itleads to a not good result.
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Table of Contents
1 Introduction2 Background3 Procedure
Data ExtractionRepair System
Oder RepairGap RepairClassificationConnection RepairTopological Sort
Generating GTFS Feed4 Evaluation5 Further Research
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Further Research
1 Extending network by collecting all the ways near to at leastone of the railways
2 Improving the efficiency of classification by precomputingtraining data
3 Identifying neighbors with regard to the existence of a real gap
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Thank you for your attention!
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data
IntroductionBackground
ProcedureEvaluation
Further Research
Question?
Zhiwei Zhang Public-Transit Data Extraction From OpenStreetMap Data