Tomography-based Overlay Network MonitoringYan Chen, David Bindel, Randy H. KatzEECS Department, University of California at Berkeley {yanchen,dbindel,randy}@EECS.Berkeley.EDU A tomography-based overlay network monitoring system Given n end hosts, characterize O(n2) paths with a basis set of O(nlogn) paths Selectively monitor O(nlogn) paths to compute the loss rates of the basis set, then infer the loss rates of all other paths Both simulation and PlanetLab experiment results promising: avg absolute error 0.0027ConclusionsWork in Progress Network diagnostics Provide it as a continuous service on PlanetLab More efficient numerical iterative methods for path selectionhttp://www.cs.berkeley.edu/~yanchen/research/wnmms

Motivation

Applications of end-to-end distance monitoringOverlay routing/location - P2P systems VPN management/provisioning Requirements for E2E monitoring systemScalable & efficient: small amount of probing trafficAccurate: capture congestion/failures

Static estimation: Global Network Positioning (GNP)Dynamic monitoringLoss rates: RON (n2 measurement)Latency: IDMaps, Dynamic Distance Maps, IsobarLatency similarity under normal conditions doesnt imply similar losses !Network tomographyInferring the characteristics of links rather than E2E pathsLimited measurements -> under-constrained system

Existing Work

1

Path Matrix and Path Space

Path loss rate p, link loss rate l

Totally s links, path vector v

A

p

B

lj

Path matrix GPut all r = O(n2) paths togetherPath loss rate vector b

3

Problem Formulation

Given n end hosts on an overlay network and O(n2) paths, how to select a minimal subset of paths to monitor so that the loss rates/latency of all other paths can be inferred.

Key idea: select a basis set of k paths that completely describe all O(n2) paths (k O(n2)) Select and monitor k linearly independent paths to compute the loss rates of basis setInfer the loss rates of all other pathsApplicable for any additive metrics, like latency

2

Sample Path Matrix

x1 - x2 unknown => set of vectorsform null spaceTo separate identifiable vs. unidentifiable components: x = xG + xN

All E2E paths are in path space, i.e., GxN = 0

4

Algorithms

Select k = rank(G) linearly independent paths to monitorUse rank revealing decomposition, e.g., QR with column pivotingLeverage sparse matrix: time O(rk2) and memory O(k2)E.g., 10 minutes for n = 350 (r = 61075) and k = 2958Compute the loss rates of other paths

Time O(k2) and memory O(k2)

Topology measurement errors toleranceCare about path loss rates than any interior linksRouter aliases => assign similar loss rates to all the alias linksIncomplete route => add direct virtual linksTopology changesAdd/remove/change one path incurs O(k2) time

6

How Much Measurement Saved ?

Internet has moderate hierarchical structure [TGJ+02]If a pure hierarchical structure (tree): k = O(n)If no hierarchy at all (worst case, clique): k = O(n2)Internet should fall in between

For reasonably large n, (e.g., 100), k = O(nlogn)

7

Evaluation

8

Extensive simulation and PlanetLab experimentsSimultaneous loss rate measurements through UDP streamingSimultaneous tracerouteAveragely, 248 out of 2550 paths have no or incomplete routing informationNo router aliases resolvedExperiments: 6/24 6/27100 experiments @ peak hoursOn average k = 872 out of 2550 paths

Areas and Domains (51 hosts)# of hosts

US (40).edu33

.org3

.net2

.gov1

.us1

Interna-tional (11)Europe (6)France1

Sweden1

Denmark1

Germany1

UK2

Asia (2)Taiwan1

Hong Kong1

Canada2

Australia1

Loss rate distribution

AccuracyAbsolute errorAverage 0.0027 for all paths, 0.0058 for lossy pathsRelative error factorCDF of 95 percentile for each experiment

9

9

lossrate[0, 0.05)lossy path [0.05, 1.0] (4.1%)

[0.05, 0.1)[0.1, 0.3)[0.3, 0.5)[0.5, 1.0)1.0

%95.9%15.2%31.0%23.9%4.3%25.6%

Intuition through Topology Virtualization

Virtual links: minimal path segments whose loss rates uniquely identifiedCan fully describe all pathsxG : similar forms as virtual links

Weidong thinks arch-e paper size is the right oneLook under page settings to find the size of this doc seems to fit!Use slides as 205% x 205%