an algebraic approach to practical and scalable overlay network monitoring yan chen, david bindel,...
Post on 20-Dec-2015
213 views
TRANSCRIPT
![Page 1: An Algebraic Approach to Practical and Scalable Overlay Network Monitoring Yan Chen, David Bindel, Hanhee Song, Randy H. Katz Presented by Mahesh Balakrishnan](https://reader038.vdocuments.us/reader038/viewer/2022110322/56649d415503460f94a1b45f/html5/thumbnails/1.jpg)
An Algebraic Approach to Practical and Scalable Overlay Network
Monitoring
Yan Chen, David Bindel, Hanhee Song, Randy H. Katz
Presented by Mahesh Balakrishnan
![Page 2: An Algebraic Approach to Practical and Scalable Overlay Network Monitoring Yan Chen, David Bindel, Hanhee Song, Randy H. Katz Presented by Mahesh Balakrishnan](https://reader038.vdocuments.us/reader038/viewer/2022110322/56649d415503460f94a1b45f/html5/thumbnails/2.jpg)
Motivation
Overlay networks Monitoring of end-to-end paths The need for a separate Monitoring Service Metrics: Latency... Loss Rate? The Goal: A Scalable Overlay Loss Rate
Monitoring Service
![Page 3: An Algebraic Approach to Practical and Scalable Overlay Network Monitoring Yan Chen, David Bindel, Hanhee Song, Randy H. Katz Presented by Mahesh Balakrishnan](https://reader038.vdocuments.us/reader038/viewer/2022110322/56649d415503460f94a1b45f/html5/thumbnails/3.jpg)
Existing Work…
Latency-only Schemes Clustering:
– Nodes are clustered together, and cluster representative is monitored
– Claim: Inaccurate for congestion detection
Co-ordinates:– Cannot give congestion information
![Page 4: An Algebraic Approach to Practical and Scalable Overlay Network Monitoring Yan Chen, David Bindel, Hanhee Song, Randy H. Katz Presented by Mahesh Balakrishnan](https://reader038.vdocuments.us/reader038/viewer/2022110322/56649d415503460f94a1b45f/html5/thumbnails/4.jpg)
Existing Work.
Network Tomography: Determining internal network properties from black-box measurements
Shavitt, et al. Algebraic approach Ozmutlu, et al. Selecting minimal set of paths to
cover all links
General Metric Systems: RON
![Page 5: An Algebraic Approach to Practical and Scalable Overlay Network Monitoring Yan Chen, David Bindel, Hanhee Song, Randy H. Katz Presented by Mahesh Balakrishnan](https://reader038.vdocuments.us/reader038/viewer/2022110322/56649d415503460f94a1b45f/html5/thumbnails/5.jpg)
Core Idea
Assumptions:– Access to link composition of paths– Ability to measure path (but not link) characteristics
From the possible n2 end-to-end paths, select a basis set of k paths (k << n2) to monitor.
The characteristics of all paths can be inferred from this basis set.
Centralized algorithm: all nodes send measurements to central node.
![Page 6: An Algebraic Approach to Practical and Scalable Overlay Network Monitoring Yan Chen, David Bindel, Hanhee Song, Randy H. Katz Presented by Mahesh Balakrishnan](https://reader038.vdocuments.us/reader038/viewer/2022110322/56649d415503460f94a1b45f/html5/thumbnails/6.jpg)
The Math
Eq 1: Represent paths as
vectors:
A
D
C
B
l1
l2
3p1
)1)(1(1 211 llp
0
1
1
v
)1log(
)1log(
)1log(
011
)1log()1log()1log(
3
2
1
211
l
l
l
llp
AD
BD
AC
![Page 7: An Algebraic Approach to Practical and Scalable Overlay Network Monitoring Yan Chen, David Bindel, Hanhee Song, Randy H. Katz Presented by Mahesh Balakrishnan](https://reader038.vdocuments.us/reader038/viewer/2022110322/56649d415503460f94a1b45f/html5/thumbnails/7.jpg)
System of Linear Equations
srG }1|0{
1 sRx
…
=
Path Matrix Link Rates Path Rates
1 rRb
![Page 8: An Algebraic Approach to Practical and Scalable Overlay Network Monitoring Yan Chen, David Bindel, Hanhee Song, Randy H. Katz Presented by Mahesh Balakrishnan](https://reader038.vdocuments.us/reader038/viewer/2022110322/56649d415503460f94a1b45f/html5/thumbnails/8.jpg)
Example Network
111
100
011
G
A
D
C
B
l1
l2
3p1
AB
AC
BC
bGx k = Number of essential paths 1 < k <= sG is rank deficient: k < s
![Page 9: An Algebraic Approach to Practical and Scalable Overlay Network Monitoring Yan Chen, David Bindel, Hanhee Song, Randy H. Katz Presented by Mahesh Balakrishnan](https://reader038.vdocuments.us/reader038/viewer/2022110322/56649d415503460f94a1b45f/html5/thumbnails/9.jpg)
More Math
k = # of essential paths
= rank (G) k <= s Usually G is rank-
deficient: k < s Select k linearly
independent paths to monitor:
bxG G
One-time QR Decomposition: O(rk2) time… O(n4)!
Inferring other paths: O(k2)
=…k
s
![Page 10: An Algebraic Approach to Practical and Scalable Overlay Network Monitoring Yan Chen, David Bindel, Hanhee Song, Randy H. Katz Presented by Mahesh Balakrishnan](https://reader038.vdocuments.us/reader038/viewer/2022110322/56649d415503460f94a1b45f/html5/thumbnails/10.jpg)
Assessment Criteria
Accuracy Scalability: How does k grow w.r.t n?
Other concerns:– centralized solution– compute time under churn– storage load
![Page 11: An Algebraic Approach to Practical and Scalable Overlay Network Monitoring Yan Chen, David Bindel, Hanhee Song, Randy H. Katz Presented by Mahesh Balakrishnan](https://reader038.vdocuments.us/reader038/viewer/2022110322/56649d415503460f94a1b45f/html5/thumbnails/11.jpg)
Effect of Topology on k growth
Star Topology, Strict Hierarchy: s = O(n), => k = O(n)
Clique: Each path (end host pair) contains a unique link, hence k = O(n2)
Hierarchy is good, Dense Connectivity is bad Conjecture: k = O(nlogn) for the internet What if only a small % of end nodes are on
overlay?
![Page 12: An Algebraic Approach to Practical and Scalable Overlay Network Monitoring Yan Chen, David Bindel, Hanhee Song, Randy H. Katz Presented by Mahesh Balakrishnan](https://reader038.vdocuments.us/reader038/viewer/2022110322/56649d415503460f94a1b45f/html5/thumbnails/12.jpg)
Linear Regression Tests
Synthetic Hierarchical Real
![Page 13: An Algebraic Approach to Practical and Scalable Overlay Network Monitoring Yan Chen, David Bindel, Hanhee Song, Randy H. Katz Presented by Mahesh Balakrishnan](https://reader038.vdocuments.us/reader038/viewer/2022110322/56649d415503460f94a1b45f/html5/thumbnails/13.jpg)
Handling Change
Path Addition: O(k2) Path Removal: O(k2) [Naïve : O(rk2) Node Addition: O(nk2) Node Removal: O(nk2)
– Cannot use path removal algorithm directly; path will be replaced using another path involving node
– Remove all paths, then look for replacements Cubic in n: Churn in large systems?
![Page 14: An Algebraic Approach to Practical and Scalable Overlay Network Monitoring Yan Chen, David Bindel, Hanhee Song, Randy H. Katz Presented by Mahesh Balakrishnan](https://reader038.vdocuments.us/reader038/viewer/2022110322/56649d415503460f94a1b45f/html5/thumbnails/14.jpg)
Routing Changes
End-to-end internet paths are generally stable
Traceroute Topology checked on a daily basis, in
presence of drastic loss rate changes If path has changed at certain links, other
paths with that link are checked as well
![Page 15: An Algebraic Approach to Practical and Scalable Overlay Network Monitoring Yan Chen, David Bindel, Hanhee Song, Randy H. Katz Presented by Mahesh Balakrishnan](https://reader038.vdocuments.us/reader038/viewer/2022110322/56649d415503460f94a1b45f/html5/thumbnails/15.jpg)
Load Balancing/Topology Measurement Errors
Paths in G are randomly reordered before basis set is selected
Untraceable paths/segments are modeled as single links; they always get selected in basis
Router aliases – one physical link presented as several virtual links – all virtual links get similar loss rates
![Page 16: An Algebraic Approach to Practical and Scalable Overlay Network Monitoring Yan Chen, David Bindel, Hanhee Song, Randy H. Katz Presented by Mahesh Balakrishnan](https://reader038.vdocuments.us/reader038/viewer/2022110322/56649d415503460f94a1b45f/html5/thumbnails/16.jpg)
Evaluation: Simulation
Three synthetic BRITE topologies: Barabasi-Albert, Waxman, hierarchical
One ‘real’ router topology (Mercator) Methodology:
– Loss Distribution: Good = 0-1%, Bad = 5-10%– Loss Model: Bernoulli, Gilbert
Simulate loss for selected paths, infer for other paths
![Page 17: An Algebraic Approach to Practical and Scalable Overlay Network Monitoring Yan Chen, David Bindel, Hanhee Song, Randy H. Katz Presented by Mahesh Balakrishnan](https://reader038.vdocuments.us/reader038/viewer/2022110322/56649d415503460f94a1b45f/html5/thumbnails/17.jpg)
Accuracy: Synthetic Topology
All Configurations under 0.008, 1.18
![Page 18: An Algebraic Approach to Practical and Scalable Overlay Network Monitoring Yan Chen, David Bindel, Hanhee Song, Randy H. Katz Presented by Mahesh Balakrishnan](https://reader038.vdocuments.us/reader038/viewer/2022110322/56649d415503460f94a1b45f/html5/thumbnails/18.jpg)
Accuracy: Real Topology
![Page 19: An Algebraic Approach to Practical and Scalable Overlay Network Monitoring Yan Chen, David Bindel, Hanhee Song, Randy H. Katz Presented by Mahesh Balakrishnan](https://reader038.vdocuments.us/reader038/viewer/2022110322/56649d415503460f94a1b45f/html5/thumbnails/19.jpg)
Accuracy
Real Topology
Synthetic Hierarchical Topology
![Page 20: An Algebraic Approach to Practical and Scalable Overlay Network Monitoring Yan Chen, David Bindel, Hanhee Song, Randy H. Katz Presented by Mahesh Balakrishnan](https://reader038.vdocuments.us/reader038/viewer/2022110322/56649d415503460f94a1b45f/html5/thumbnails/20.jpg)
Running Time
3 seconds for 100 nodes, 21 minutes for 500!
![Page 21: An Algebraic Approach to Practical and Scalable Overlay Network Monitoring Yan Chen, David Bindel, Hanhee Song, Randy H. Katz Presented by Mahesh Balakrishnan](https://reader038.vdocuments.us/reader038/viewer/2022110322/56649d415503460f94a1b45f/html5/thumbnails/21.jpg)
Load Balancing
![Page 22: An Algebraic Approach to Practical and Scalable Overlay Network Monitoring Yan Chen, David Bindel, Hanhee Song, Randy H. Katz Presented by Mahesh Balakrishnan](https://reader038.vdocuments.us/reader038/viewer/2022110322/56649d415503460f94a1b45f/html5/thumbnails/22.jpg)
Effect of Churn/Routing Change
Path Addition: 125 msec Path Removal: 445 msec Node Addition: 1.18 sec Node Removal: 16.9 sec What about n >> 60?
Node Deletion
Node Addition
Network Link Removal
![Page 23: An Algebraic Approach to Practical and Scalable Overlay Network Monitoring Yan Chen, David Bindel, Hanhee Song, Randy H. Katz Presented by Mahesh Balakrishnan](https://reader038.vdocuments.us/reader038/viewer/2022110322/56649d415503460f94a1b45f/html5/thumbnails/23.jpg)
PlanetLab Experiments
51 hosts, each from different organization Each node sends a UDP packet to every
other host in each trial 300 trials of 300 msec each Receiver counts packets for loss rate Traceroute used for topology measurement
![Page 24: An Algebraic Approach to Practical and Scalable Overlay Network Monitoring Yan Chen, David Bindel, Hanhee Song, Randy H. Katz Presented by Mahesh Balakrishnan](https://reader038.vdocuments.us/reader038/viewer/2022110322/56649d415503460f94a1b45f/html5/thumbnails/24.jpg)
PlanetLab Results
Cumulative coverage/FP Cumulative error (Worst Run)
Average Abs. Error = 0.0027, Average Error Factor 1.1
![Page 25: An Algebraic Approach to Practical and Scalable Overlay Network Monitoring Yan Chen, David Bindel, Hanhee Song, Randy H. Katz Presented by Mahesh Balakrishnan](https://reader038.vdocuments.us/reader038/viewer/2022110322/56649d415503460f94a1b45f/html5/thumbnails/25.jpg)
Effect of traffic on loss rates
Sensitivity Analysis done at night, on empty networks
Threshold at 12.8 Mbps
Why do this?
![Page 26: An Algebraic Approach to Practical and Scalable Overlay Network Monitoring Yan Chen, David Bindel, Hanhee Song, Randy H. Katz Presented by Mahesh Balakrishnan](https://reader038.vdocuments.us/reader038/viewer/2022110322/56649d415503460f94a1b45f/html5/thumbnails/26.jpg)
Conclusion
Algebraic Method for inferring loss rates of all paths from a basis set
Quite Accurate Reasonable load imposed on each node But is it really scalable? Centralized solution, cubic dependence on n
for handling node addition/removal