assessing partial observability of ... -...
TRANSCRIPT
![Page 1: Assessing partial observability of ... - scale-freeback.euscale-freeback.eu/wp-content/uploads/2018/09/VITI-ERC-SCALE-2018.pdf Assessing partial observability of link flow inference](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e039df7d7e249551631d02a/html5/thumbnails/1.jpg)
www.mobilab.lu
Assessing partial observability of link flow
inference problems at large scale networksFrancesco VITI & Marco Rinaldi - University of Luxembourg
ERC SCALE 2018 β 10-11 September 2018, Grenoble
![Page 2: Assessing partial observability of ... - scale-freeback.euscale-freeback.eu/wp-content/uploads/2018/09/VITI-ERC-SCALE-2018.pdf Assessing partial observability of link flow inference](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e039df7d7e249551631d02a/html5/thumbnails/2.jpg)
www.mobilab.lu
Outline
Introduction
Network Sensor Location Problems
Full and partial observability concepts
A new metric for assessing partial observability
NSP metric
Ranking partial observability solutions
Examples
The impact of route set generation in large scale applications
Hypergraph
Exact and approximate solutions
Examples
Conclusions & future research directions
2
![Page 3: Assessing partial observability of ... - scale-freeback.euscale-freeback.eu/wp-content/uploads/2018/09/VITI-ERC-SCALE-2018.pdf Assessing partial observability of link flow inference](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e039df7d7e249551631d02a/html5/thumbnails/3.jpg)
www.mobilab.lu
Introduction
Information from traffic sensors crucial to various applications, e.g.
State estimation
OD flow estimation
Model calibration
Traffic management
β¦
β’ Problem: Obtaining full information not realistic in real-sized networks
need to strategically position the (limited) amount of sensors
The quality of traffic information depends on number and type of
sensors and where they are positioned
network sensor location problem, NSLP
3
![Page 4: Assessing partial observability of ... - scale-freeback.euscale-freeback.eu/wp-content/uploads/2018/09/VITI-ERC-SCALE-2018.pdf Assessing partial observability of link flow inference](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e039df7d7e249551631d02a/html5/thumbnails/4.jpg)
www.mobilab.lu
Network Sensor Location
Problem (NSLP)
Formal definition:
βFind an optimal number and location of sensors to
maximize the available information on a networkβ
Analogy with a sudoku game: different initial states (positions
of numbers) determine difficulty (estimation reliability) and
uniqueness (solution determinacy)
Need rules to solve them
Link-Route-OD relationships
Local consistency (e.g., flows at nodes)
4
![Page 5: Assessing partial observability of ... - scale-freeback.euscale-freeback.eu/wp-content/uploads/2018/09/VITI-ERC-SCALE-2018.pdf Assessing partial observability of link flow inference](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e039df7d7e249551631d02a/html5/thumbnails/5.jpg)
www.mobilab.lu
The NSLP formulation
Determining link/route information based on simple algebraic relations
Fundamental relationships used
OD flows f,
route flows h Οπββ πΏππβπ = π£π , βπ β β
Οπββ ππ€πβπ = ππ€ , βπ€ β π²
ππ =
β π
link flows v
Location rules
Mostly based on widely accepted statements, i.e.
Select those variables that can tell something about unknown variables
(information coverage)
Try and get as much flow information as possible (information capturing)
Donβt waste sensors to get information you have already (information
independence)
5
![Page 6: Assessing partial observability of ... - scale-freeback.euscale-freeback.eu/wp-content/uploads/2018/09/VITI-ERC-SCALE-2018.pdf Assessing partial observability of link flow inference](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e039df7d7e249551631d02a/html5/thumbnails/6.jpg)
www.mobilab.lu
Location rules
Heuristic rules in literature (Yang and Zhouβs, 1998, Larsson et al., 2010, Cipriani et al.,
2006, Yang et al., 2006, Gentili and Mirchandani, 2012, Castillo et al., 2014)
1. OD/route/link coverage
2. Maximum (link/route/OD) flow fraction
3. Maximum intercepting/net (link/route/OD) flow
4. Link/route independence
No general methodology exists that applies all rules at once
Two families of NSLP approaches (Gentili& Mirchandani, 2012)
Observability problems β exploiting topological supply relations
Flow estimation problems β exploting demand flow relations
![Page 7: Assessing partial observability of ... - scale-freeback.euscale-freeback.eu/wp-content/uploads/2018/09/VITI-ERC-SCALE-2018.pdf Assessing partial observability of link flow inference](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e039df7d7e249551631d02a/html5/thumbnails/7.jpg)
www.mobilab.lu
Observability vs. Flow
estimation
Observability problems
Need only relations between link,
route and OD variables (topological
characteristics)
Specifies coverage requirements
Set the positions of the known
numbers, in relation to the unknown
ones
Assess solution determinacy
Flow-estimation problems
Optimal solutions related to estimation
model adopted
Prior information necessary
Different rules to determine quality of
information a priori
Set the values of numbers in relation to
those unknown
Assess solution reliability
7
![Page 8: Assessing partial observability of ... - scale-freeback.euscale-freeback.eu/wp-content/uploads/2018/09/VITI-ERC-SCALE-2018.pdf Assessing partial observability of link flow inference](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e039df7d7e249551631d02a/html5/thumbnails/8.jpg)
www.mobilab.lu
Full observability problem
Determine the minimum amount/combination of sensors to be placed
such that all flow information is known
Obvious solution: equip all links with sensors
Better solution: exploit topological relationships to reduce number of variables
necessary
8
3 1 2
4 3 5
v v v
v v v
![Page 9: Assessing partial observability of ... - scale-freeback.euscale-freeback.eu/wp-content/uploads/2018/09/VITI-ERC-SCALE-2018.pdf Assessing partial observability of link flow inference](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e039df7d7e249551631d02a/html5/thumbnails/9.jpg)
www.mobilab.lu
Full observability problem
Full observability: find set of linearly independent variables:
ππ = πππ
Different solution methods:
Pivoting (Castillo et al., 2008);
Gaussian Elimination (Hu et al., 2009);
Node-based (Ng, 2012);
Topological tree (He, 2013);
Non-planar Holes (Castillo et al., 2014).
Derive new relations π using matrix transformations
π£1π£2π£3π£4π£5
=
1 0 1 00 1 0 11 1 1 11 1 0 00 0 1 1
β1β2β3β4
![Page 10: Assessing partial observability of ... - scale-freeback.euscale-freeback.eu/wp-content/uploads/2018/09/VITI-ERC-SCALE-2018.pdf Assessing partial observability of link flow inference](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e039df7d7e249551631d02a/html5/thumbnails/10.jpg)
www.mobilab.lu
Full observability: Pivoting
π£1π£2π£3π£4π£5
=
1 0 1 00 1 0 11 1 1 11 1 0 00 0 1 1
β1β2β3β4
β1π£2π£3π£4π£5
=
1 0 β1 00 1 0 11 1 0 11 1 β1 00 0 1 1
π1β2β3β4
β1β2π3π£4π£5
=
1 0 β1 00 1 0 β11 1 0 01 1 β1 β10 0 1 1
π1π2β3β4
β1β2π3β3π5
=
0 β1 1 10 1 0 β11 1 0 01 1 β1 β11 1 β1 0
π1π2π4β4
Problem 1 : Full observability solutions only theoretical in real-sized networks
(~60% of links need to be measured)
Problem 2: Exact solutions, not unique permutation-dependent
![Page 11: Assessing partial observability of ... - scale-freeback.euscale-freeback.eu/wp-content/uploads/2018/09/VITI-ERC-SCALE-2018.pdf Assessing partial observability of link flow inference](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e039df7d7e249551631d02a/html5/thumbnails/11.jpg)
www.mobilab.lu
Research questions
Research questions:
Is there a way of assessing partial observability solutions?
What is the added value of placing or removing an extra sensor?
What if only a limited number of sensors is available for budget reasons?
What would be the # of sensors needed to guarantee an acceptable level of
under-determinedness?
Are full observability solutions all the same in terms of potential partial
observability solutions?
β1β2π3β3π5
=
0 β1 1 10 1 0 β11 1 0 01 1 β1 β11 1 β1 0
π1π2π4β4
π£1π£2π£3π£4π£5
=
1 0 1 00 1 0 11 1 1 11 1 0 00 0 1 1
β1β2β3β4
![Page 12: Assessing partial observability of ... - scale-freeback.euscale-freeback.eu/wp-content/uploads/2018/09/VITI-ERC-SCALE-2018.pdf Assessing partial observability of link flow inference](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e039df7d7e249551631d02a/html5/thumbnails/12.jpg)
www.mobilab.lu
Partial observability
Goal Find an optimal* set of locations in full observability
solution(s) in terms of partial observability
* minimizing the magnitude of missing information, while
* respecting given budget constraints (e.g. maximum number of sensors)
Why starting from the full observability solution?
1. We can assure that links with sensors are linearly independent
2. Solutions provide a smallest set of linearly independent variables
3. The search space of solutions is reduced significantly
β1β2π3β3π5
=
0 β1 1 10 1 0 β11 1 0 01 1 β1 β11 1 β1 0
π1π2π4β4
![Page 13: Assessing partial observability of ... - scale-freeback.euscale-freeback.eu/wp-content/uploads/2018/09/VITI-ERC-SCALE-2018.pdf Assessing partial observability of link flow inference](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e039df7d7e249551631d02a/html5/thumbnails/13.jpg)
www.mobilab.lu
Towards a new metric
Rewrite ππ = πππ into
ππππππππππ
=π ππ π
ππππππππππ
Subdivide variables in those observed and those unobservedππππππ &πππ
ππππππ &π’ππππ
ππππ=
π ππ π
ππππππππππ
=πβ²ππβ²
ππβ²β²πβ²β²
πππ
ππππππ &πππ
ππππππ &π’ππππ
ππππ
Focus on the βsizeβ of the solution space of the reduced pivoted matrix and on its basic
dimensions knowing that
The reduced matrix Pβ will have infinite possible solutions due to under-
determinedness
The basis Bβ provides the βbasicβ vectors characterizing the infinite solutions, i.e. any
combination πΆπβ² belongs to the Null space
We want now
A metric that gives a scalar value to the degree of under-determinedness
It is normalized, i.e. it can be seen as βfraction of full information lostβ13
![Page 14: Assessing partial observability of ... - scale-freeback.euscale-freeback.eu/wp-content/uploads/2018/09/VITI-ERC-SCALE-2018.pdf Assessing partial observability of link flow inference](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e039df7d7e249551631d02a/html5/thumbnails/14.jpg)
www.mobilab.lu
Null-Space Metric of P, NSP
β’ Characterize the Null Space, i.e. set of vectors mapping to zero in the given
(sub)space measuring the degrees of freedom in solution space
Compare extent of the basis of the Null space of the reduced matrix wrt basis
of full matrix normalization of the solution spaces
New metric for assessing partial observability solutions based on the trace
function:
πππ =
π ππ π
π
πβ²πΉ
π ππ π πΉ
Frobenius norm:
related to the trace of the matrix, i.e. the maximum error extent in the null-space
related to the singular values of the reduced matrix
Basis of the Partial
Observability matrixProjection onto full set of
independent variables
Extent of the full matrix
14
![Page 15: Assessing partial observability of ... - scale-freeback.euscale-freeback.eu/wp-content/uploads/2018/09/VITI-ERC-SCALE-2018.pdf Assessing partial observability of link flow inference](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e039df7d7e249551631d02a/html5/thumbnails/15.jpg)
www.mobilab.lu
Geometric interpretation
NSP metric related to the
maximum size of the prism
derived from the basis
A
B
C
D
v3v1
v2
v4
v5
15
![Page 16: Assessing partial observability of ... - scale-freeback.euscale-freeback.eu/wp-content/uploads/2018/09/VITI-ERC-SCALE-2018.pdf Assessing partial observability of link flow inference](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e039df7d7e249551631d02a/html5/thumbnails/16.jpg)
www.mobilab.lu
Greedy algorithm(s)
Questions
1. How to optimally identify sensor placements?
2. What is the added value of placing or removing an extra sensor?
3. Given n sensors, where should these be installed to get maximum
information?
Two local search algorithms introduced:
Add
starts from an empty set of observed variables, and adds as variable
to be observed, the one that results in the biggest decrease of error
Remove
starts from the set observing all variables, and chooses the single
variable that causes the least decrease in information
![Page 17: Assessing partial observability of ... - scale-freeback.euscale-freeback.eu/wp-content/uploads/2018/09/VITI-ERC-SCALE-2018.pdf Assessing partial observability of link flow inference](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e039df7d7e249551631d02a/html5/thumbnails/17.jpg)
www.mobilab.lu
Proof of concept (1)
Test on simplified βcandyβ
network
Easier to illustrate properties
Has only two βpivotβ families to
evaluate
βCenterβ family, i.e. l3 is
observed
βStarβ family, where solution
does not contain I3
A
B
C
D
v3v1
v2
v4
v5
17
![Page 18: Assessing partial observability of ... - scale-freeback.euscale-freeback.eu/wp-content/uploads/2018/09/VITI-ERC-SCALE-2018.pdf Assessing partial observability of link flow inference](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e039df7d7e249551631d02a/html5/thumbnails/18.jpg)
www.mobilab.lu
Proof of concept (2)
Star
A
B
C
D
v3v1
v2
v4
v5 Max. Estimation Error
Observed: l1 & l4 Observed: l1 & l5 Observed: l4 & l5l1 0 0 ββ,β
l2 0 + ββ,β β 0 ββ,β + 0 β 0 0 + 0 β ββ,β
l3 0 + ββ,β ββ,β + 0 0
l4 0 ββ,β 0
l5 ββ,β 0 0
NSP 0.44 0.44 0.44
Max. Estimation Error
Observed: l1 Observed: l4 Observed: l5
l10 ββ,β ββ,β
l2 ββ,β + ββ,β β 0 ββ,β + 0 β [ββ,β] ββ,β + 0 β [ββ,β]
l3 ββ,β + [ββ,β] ββ,β + 0 0 + ββ,β
l4ββ,β 0 ββ,β
l5ββ,β ββ,β 0
NSP 0.79 0.64 0.64
18
![Page 19: Assessing partial observability of ... - scale-freeback.euscale-freeback.eu/wp-content/uploads/2018/09/VITI-ERC-SCALE-2018.pdf Assessing partial observability of link flow inference](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e039df7d7e249551631d02a/html5/thumbnails/19.jpg)
www.mobilab.lu
Proof of concept (3)
Center
A
B
C
D
v3v1
v2
v4
v5Max. Estimation Error
Observed: l1 & l5 Observed: l1 & l3 Observed: l3 & l5l1 0 0 ββ,β
l2 0 β ββ,β 0 0 β ββ,β
l3 ββ,β 0 0
l4 0 β ββ,β 0 β ββ,β 0
l5 0 ββ,β 0
NSP 0.53 0.47 0.47
Max. Estimation Error
Observed: l1 Observed: l3 Observed: l5l1 0 ββ,β ββ,β
l2 ββ,β β 0 0 β ββ,β ββ,β β [ββ,β]
l3 ββ,β 0 ββ,β
l4 ββ,β β [ββ,β] 0 β ββ,β ββ,β β 0
l5 ββ,β ββ,β 0
NSP 0.8 0.69 0.8
19
![Page 20: Assessing partial observability of ... - scale-freeback.euscale-freeback.eu/wp-content/uploads/2018/09/VITI-ERC-SCALE-2018.pdf Assessing partial observability of link flow inference](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e039df7d7e249551631d02a/html5/thumbnails/20.jpg)
www.mobilab.lu
Pivot βFamiliesβ: Ranking
Question: are pivots all the same to find optimal locations?
Answer: NO
And our NSP is able to provide different results for the same number of sensors
Singular Value Decomposition can be employed to rank pivots a-priori
ΟπΟπ π ππ as a measure of pivotβs information content
20
β’ βStarβ Family: 2.7979
β’ βCenterβ Family: 2.7321
![Page 21: Assessing partial observability of ... - scale-freeback.euscale-freeback.eu/wp-content/uploads/2018/09/VITI-ERC-SCALE-2018.pdf Assessing partial observability of link flow inference](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e039df7d7e249551631d02a/html5/thumbnails/21.jpg)
www.mobilab.lu
Pivot βFamiliesβ: different
information
21
More informative
Faster error decrease
Rank: 2.7321
Rank: 2.7979
![Page 22: Assessing partial observability of ... - scale-freeback.euscale-freeback.eu/wp-content/uploads/2018/09/VITI-ERC-SCALE-2018.pdf Assessing partial observability of link flow inference](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e039df7d7e249551631d02a/html5/thumbnails/22.jpg)
www.mobilab.lu
Observations
Tested different network sizes and it seems that
Ranking is consistently independent on the number of sensors, i.e. better ranked
families remain better ranked till full observability solution
Best families can be found already from first sensors placed!
βParallelβ network (14 links; 4 ODs)
![Page 23: Assessing partial observability of ... - scale-freeback.euscale-freeback.eu/wp-content/uploads/2018/09/VITI-ERC-SCALE-2018.pdf Assessing partial observability of link flow inference](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e039df7d7e249551631d02a/html5/thumbnails/23.jpg)
www.mobilab.lu
Test on large networks:
Rotterdam
Test on real-sized networks shows that
Most informative links are near nodes and centroids (good news!)
Nice spatial distribution (even if no such rule was imposed)
Not all full observability solutions contain the minimum number of sensors
(consistent with recent findings of Castillo, 2013)
Rank:
266.27
Rank:
342.74
!!!
Links 476
Nodes 243
ODs 1890
![Page 24: Assessing partial observability of ... - scale-freeback.euscale-freeback.eu/wp-content/uploads/2018/09/VITI-ERC-SCALE-2018.pdf Assessing partial observability of link flow inference](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e039df7d7e249551631d02a/html5/thumbnails/24.jpg)
www.mobilab.lu
Conclusions (intermezzo)
A new definition of partial observability problems that accounts for
full and partially observed variables was introduced
A novel metric able to assess information quantity based on
analysis of the extent of the Null-Space (NSP)
Local search algorithms developed to explore the solution space,
to determine optimal sensor locations
The methodology is generic as requires only the fundamental
topological relations between links, routes and eventually OD flows
different applications and extensions possible
Solutions on large networks strongly depend on route set
24
![Page 25: Assessing partial observability of ... - scale-freeback.euscale-freeback.eu/wp-content/uploads/2018/09/VITI-ERC-SCALE-2018.pdf Assessing partial observability of link flow inference](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e039df7d7e249551631d02a/html5/thumbnails/25.jpg)
www.mobilab.lu
Node vs. Route based
approaches
Node-based relations
β’ Flow conservation at nodes
β’ Local, simple laws
β’ No route enumeration necessary
β’ Misses complete information on link-path-OD relations
solutions require systematically more sensors
Route-based relationsβ’ Flow conservation on routes
β’ Connects links across whole network
β’ Needs some kind of route enumeration
6
![Page 26: Assessing partial observability of ... - scale-freeback.euscale-freeback.eu/wp-content/uploads/2018/09/VITI-ERC-SCALE-2018.pdf Assessing partial observability of link flow inference](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e039df7d7e249551631d02a/html5/thumbnails/26.jpg)
www.mobilab.lu
Observability and route
information (1)
Different route set compositions yield different full observability
solutions
Routes may be selected because of behavioral/flow capturing rules
(e.g. shortest paths)
| |
1 0 1 0
1 0 0 1
l RA
{ }odR r | | | |dep indep
Hypothesis: more efficient full (and partial) observability solutions can
be found with efficient route set generation.
7
![Page 27: Assessing partial observability of ... - scale-freeback.euscale-freeback.eu/wp-content/uploads/2018/09/VITI-ERC-SCALE-2018.pdf Assessing partial observability of link flow inference](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e039df7d7e249551631d02a/html5/thumbnails/27.jpg)
www.mobilab.lu
Observability and route
information (2)
Three different sorts of information
Non Redundant (linearly independent)
Redundant + Informative (linearly dependent, but useful to derive
independence relationships)
Pure Redundant (linearly dependent)
Strongly dependent on the chosen route set
| |
1 0 1 0
[ ]
1 0 0 1
l RA NR RI PR
8
![Page 28: Assessing partial observability of ... - scale-freeback.euscale-freeback.eu/wp-content/uploads/2018/09/VITI-ERC-SCALE-2018.pdf Assessing partial observability of link flow inference](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e039df7d7e249551631d02a/html5/thumbnails/28.jpg)
www.mobilab.lu
Example: parallel highway
network
Best full obs. route set: Best* partial obs. route set:
Β«Parallel highwayΒ» network
4 routes, fully diverse 9 routes, highly overlapping
* Non-unique solution
9
![Page 29: Assessing partial observability of ... - scale-freeback.euscale-freeback.eu/wp-content/uploads/2018/09/VITI-ERC-SCALE-2018.pdf Assessing partial observability of link flow inference](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e039df7d7e249551631d02a/html5/thumbnails/29.jpg)
www.mobilab.lu
Route set generation
Main idea:
Enumerate routes so to obtain the maximum linearly independent route set
Evaluate resulting Partial Observability Solutions
Challenges:
High dimensionality, combinatorial problem
Non-uniqueness of solutions
Non-convex condition (linear independence)
29
![Page 30: Assessing partial observability of ... - scale-freeback.euscale-freeback.eu/wp-content/uploads/2018/09/VITI-ERC-SCALE-2018.pdf Assessing partial observability of link flow inference](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e039df7d7e249551631d02a/html5/thumbnails/30.jpg)
www.mobilab.lu
Hypergraph approach (1)
Hypergraph
Express combinations of routes as vertices
Edges capture linear independence through bitwise logical operations
Example:
2 3 4
1 3 4
2 3 5
{ , , }
{ , }
{ ,
,
, }
v
A v v v
B v v
C v v v
(0,1,1,1,0)
(1,0,1,1,0)
(0,1,1,0,1)
A
B
C
Route combination A + B is independent iif:
,A B A B
(1,1,1,1,0)A B
A
B
C
30
![Page 31: Assessing partial observability of ... - scale-freeback.euscale-freeback.eu/wp-content/uploads/2018/09/VITI-ERC-SCALE-2018.pdf Assessing partial observability of link flow inference](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e039df7d7e249551631d02a/html5/thumbnails/31.jpg)
www.mobilab.lu
Hypergraph approach (2)
B CA B A C
A B C
A B C
Independence condition:
A
B
C
AA B C
B
B C
A C
A
B
C
31
![Page 32: Assessing partial observability of ... - scale-freeback.euscale-freeback.eu/wp-content/uploads/2018/09/VITI-ERC-SCALE-2018.pdf Assessing partial observability of link flow inference](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e039df7d7e249551631d02a/html5/thumbnails/32.jpg)
www.mobilab.lu
3 equivalent solutions
A B
B C
B C A C
BA A C
A
B
C
32
![Page 33: Assessing partial observability of ... - scale-freeback.euscale-freeback.eu/wp-content/uploads/2018/09/VITI-ERC-SCALE-2018.pdf Assessing partial observability of link flow inference](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e039df7d7e249551631d02a/html5/thumbnails/33.jpg)
www.mobilab.lu
Finding exact solutions (1)
Problem to be solved: constrained maximum clique in
the hypergraph|| ||
| |
{0,1}
(
(
A {
)
}
)
h h
h
V V
g ij
g V g
HG g
A J
A
a
I A
Q I
min
. .0
{0,1}
T
x HG
HG
x Q x f x
xPs t
x
Quadratic component,
captures adjacency
| |
max(|:
| ) 1
ii
i h
v
vf f
V
Linear component,
Captures importance
1 | |[ , ] { }
1:
| |
,...,hHG i iV
ik k i
ii i
P i
v
pp
vp v
p
Parenthood
constraints
33
![Page 34: Assessing partial observability of ... - scale-freeback.euscale-freeback.eu/wp-content/uploads/2018/09/VITI-ERC-SCALE-2018.pdf Assessing partial observability of link flow inference](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e039df7d7e249551631d02a/html5/thumbnails/34.jpg)
www.mobilab.lu
Finding exact solutions (2)
2R
2 (2 1)
2
R R
34
Exact solutions based on solving the max clique problem
computationally intractable even for small networks!
Real challenge: building the hypergraph
nodes to be generated
edges to be checked for eligibility
Quickly unfeasible with realistic network sizes, akin to brute
forcing.
![Page 35: Assessing partial observability of ... - scale-freeback.euscale-freeback.eu/wp-content/uploads/2018/09/VITI-ERC-SCALE-2018.pdf Assessing partial observability of link flow inference](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e039df7d7e249551631d02a/html5/thumbnails/35.jpg)
www.mobilab.lu
Full βcandyβ example (1)
12
Exact solutions based on solving the max clique problem
computationally intractable even for small networks!
8
| | 256
| | 32385
h
h
R
V
L
![Page 36: Assessing partial observability of ... - scale-freeback.euscale-freeback.eu/wp-content/uploads/2018/09/VITI-ERC-SCALE-2018.pdf Assessing partial observability of link flow inference](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e039df7d7e249551631d02a/html5/thumbnails/36.jpg)
www.mobilab.lu
Full βcandyβ example (2)
36
![Page 37: Assessing partial observability of ... - scale-freeback.euscale-freeback.eu/wp-content/uploads/2018/09/VITI-ERC-SCALE-2018.pdf Assessing partial observability of link flow inference](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e039df7d7e249551631d02a/html5/thumbnails/37.jpg)
www.mobilab.lu
Hypergraph generation:
heuristics
Vertex culling rules:
(Cul-1): Remove vertices if no new information is introduced w.r.t. parents
(Cul-2): Remove vertices if expected information is lower than best bound
Metaheuristics used to find efficient solutions (e.g. GA)
A B
BA
A B
BA
B CA B A C
A B C
D
14
![Page 38: Assessing partial observability of ... - scale-freeback.euscale-freeback.eu/wp-content/uploads/2018/09/VITI-ERC-SCALE-2018.pdf Assessing partial observability of link flow inference](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e039df7d7e249551631d02a/html5/thumbnails/38.jpg)
www.mobilab.lu
Impact of culling rules
Cul-1
Solution: Β« BCH Β»Cul-1 + Cul-2
Solution: Β« BC Β»
38
![Page 39: Assessing partial observability of ... - scale-freeback.euscale-freeback.eu/wp-content/uploads/2018/09/VITI-ERC-SCALE-2018.pdf Assessing partial observability of link flow inference](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e039df7d7e249551631d02a/html5/thumbnails/39.jpg)
www.mobilab.lu
Parallel Highways example
Hypergraph Statistics Solution statistics
# Vertices # Arcs# Vertices Max
Clique
Final route set size
[ind, full coverage]
Tot Memory usage (MB)
[RAM + Nodefiles]
Comp. Time
(s)% Gap
3583 5991142 405 6, 6 217.77 3600+278.7 4.11
39
![Page 40: Assessing partial observability of ... - scale-freeback.euscale-freeback.eu/wp-content/uploads/2018/09/VITI-ERC-SCALE-2018.pdf Assessing partial observability of link flow inference](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e039df7d7e249551631d02a/html5/thumbnails/40.jpg)
www.mobilab.lu
Test on large scale networks:
Route set generation methods
Yenβs K-Shortest path
Free parameter: k, how many paths per O/D
Intuition: higher k -> higher information content
Intuition 2: an upper bound to k must exist, after which adding routes to the
routeset will only bring redundant information*
Enumerate according to Yenβs K-Shortest paths
Extra check: if new path added is NOT independent, discard
Stop when k is met or no more independent paths can be found
Independency check: performed through matrix rank (can be computationally
cumbersome)
Castilloβs (2015) Independent paths
Enumerates routes such that all routes (and combinations thereof) are
independent from one another
Higher degree of information wrt. randomly chosen k-shortest paths
10
![Page 41: Assessing partial observability of ... - scale-freeback.euscale-freeback.eu/wp-content/uploads/2018/09/VITI-ERC-SCALE-2018.pdf Assessing partial observability of link flow inference](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e039df7d7e249551631d02a/html5/thumbnails/41.jpg)
www.mobilab.lu
Sioux Falls example
Generated Solution Statistics Solution statistics
# VerticesTot. GA
GenerationsFinal route set size
Tot Memory usage (MB)
[RAM]Comp. Time (s)
37 165 - 37 0.72 281.9
41
![Page 42: Assessing partial observability of ... - scale-freeback.euscale-freeback.eu/wp-content/uploads/2018/09/VITI-ERC-SCALE-2018.pdf Assessing partial observability of link flow inference](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e039df7d7e249551631d02a/html5/thumbnails/42.jpg)
www.mobilab.lu
Rotterdam example
Generated Solution Statistics Solution statistics
# VerticesTot. GA
GenerationsFinal route set size
Tot Memory usage (MB)
[RAM]Comp. Time (s)
144 541 - 150 98.26 12259.1
Links 476
Nodes 243
ODs 1890
![Page 43: Assessing partial observability of ... - scale-freeback.euscale-freeback.eu/wp-content/uploads/2018/09/VITI-ERC-SCALE-2018.pdf Assessing partial observability of link flow inference](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e039df7d7e249551631d02a/html5/thumbnails/43.jpg)
www.mobilab.lu
Vulnerability analysis
43
KSP
KISP
C1
MI
PivotingNetwork
Topology
MI
KSP
KISP
C
'
'
'
'
MI
KSP
KISP
C
β πππππ β|fail|Γ|πππ|β|πππππ|Γ|πππ|
MI
KSP
KISP
C
A
A
A
A
β|πΏ|Γ|π |
( , )G N L
Random Sensor
FailureLoss of
Information
Hypothesis: different full (and partial) observability solutions bear
different levels of resilience to sensor failure.
'||
||
||
||
T
F
F
NSB
P
')(B null
![Page 44: Assessing partial observability of ... - scale-freeback.euscale-freeback.eu/wp-content/uploads/2018/09/VITI-ERC-SCALE-2018.pdf Assessing partial observability of link flow inference](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e039df7d7e249551631d02a/html5/thumbnails/44.jpg)
www.mobilab.lu
Test Results (1)
Information level for different route set generation policies at 50 sensors
44
( ' )KISPNSP
( ' )KSPNSP
1)( 'CNSP
)( 'MINSP
![Page 45: Assessing partial observability of ... - scale-freeback.euscale-freeback.eu/wp-content/uploads/2018/09/VITI-ERC-SCALE-2018.pdf Assessing partial observability of link flow inference](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e039df7d7e249551631d02a/html5/thumbnails/45.jpg)
www.mobilab.lu
Test Results (2)
Loss of information upon 10% sensor failure (100 draws, 5x ~U(0,50))
45
( )NSP
![Page 46: Assessing partial observability of ... - scale-freeback.euscale-freeback.eu/wp-content/uploads/2018/09/VITI-ERC-SCALE-2018.pdf Assessing partial observability of link flow inference](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e039df7d7e249551631d02a/html5/thumbnails/46.jpg)
www.mobilab.lu
Conclusions
Different full observability solutions indeed bear different resilience to
sensor failure
Max. Independent Route Set policy
Achieves highest information content and density
Interestingly, also yields most resilient solution upon sensor failure
Future research topics:
Mixing different types of sensors (scanned links, FCDs,β¦)
Topology to data: impact on flow estimation techniques, comparison
between topology prediction and data validation
Potential for combination with flow capturing methods to be explored,
especially for state estimation
From observability to controllabilityβ¦
46
![Page 47: Assessing partial observability of ... - scale-freeback.euscale-freeback.eu/wp-content/uploads/2018/09/VITI-ERC-SCALE-2018.pdf Assessing partial observability of link flow inference](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e039df7d7e249551631d02a/html5/thumbnails/47.jpg)
www.mobilab.lu
References
Viti F., Rinaldi M., Corman F., TampΓ©re C.M.J. (2014). Assessing Partial
Observability in Network Sensor Location Problems. Transportation
Research Part B: Methodological, Vol. 70, pp. 65-89.
Rinaldi M., Viti F. (2017). Exact and Approximate Route Set Generation for
Resilient Partial Observability in Sensor Location Problems. Transportation
Research Part B: Methodological, Vol. 105, pp. 86-119.
Thank you!
{francesco.viti,marco.rinaldi}@uni.lu
47