1

Passive Network Tomography Using Bayesian Inference

Lili QiuJoint work with

Venkata N. Padmanabhan and Helen J. WangMicrosoft Research

Internet Measurement Workshop 2002Marseille, France

2

Motivation

C&W

AT&T

WebServer

Sprint

UUNet

Qwest Earthlink

AOL

It’s so slow!

Diagnosisengine

Ethernet

Why is itso slow?

3

Network DiagnosisDiagnosis

engine

Netmon/tcpdump

traces

Network topology

Troublespots location

Diagnosis results:Qwest access link: 63.232.180.230->63.232.33.134Peering between UUNET and AOL: 64.45.216.154->172.139.89.74

4

Network Diagnosis (Cont.) Goal: Determine internal network

characteristics using passive end-to-end measurements Primary focus: identifying lossy links

Applications Trouble shooting Server selection Server placement Overlay network path construction

5

Previous Work Active probing to infer link loss rate

multicast probes striped unicast probes

Pros & cons accurate since individual loss events identified expensive because of extra probe traffic

S

A B

S

A B

6

Problem Formulation

l1

l8l7l6

l2

l4 l5

l3

server

clientsp1 p2 p3 p4 p5

(1-l1)*(1-l2)*(1-l4) = (1-p1)

(1-l1)*(1-l2)*(1-l5) = (1-p2)…(1-l1)*(1-l3)*(1-l8) = (1-p5)

Challenges:• Under-constrained system

of equations• Measurement errors

7

Gibbs Sampling D

observed packet transmission and loss at the clients

ensemble of loss rates of links in the network Goal

determine the posterior distribution P(|D) Approach

Use Markov Chain Monte Carlo with Gibbs sampling to obtain samples from P(|D)

Draw conclusions based on the samples

8

Gibbs Sampling (Cont.) Applying Gibbs sampling to network

tomography 1) Initialize link loss rates arbitrarily 2) For j = 1 : warmup

for each link i compute P(li|D, {li’}) where li is loss rate of link i, and {li’} = kI lk

3) For j = 1 : realSamples for each link i

compute P(li|D, {li’}) Use all the samples obtained at step 3 to

approximate P(|D)

9

Performance Evaluation

Simulation experiments Trace-driven validation

10

Simulation Experiments Advantage: no uncertainty about link loss rate! Methodology

Topologies used: randomly-generated: 20 - 3000 nodes, max degree = 5-50 real topology obtained by tracing paths to microsoft.com

clients randomly-generated packet loss events at each link

A fraction f of the links are good, and the rest are “bad” LM1: good links: 0 – 1%, bad links: 5 – 10% LM2: good links: 0 – 1%, bad links: 1 – 100%

Link loss processes: Bernoulli and Gilbert Goodness metrics:

Coverage: # correctly inferred lossy links False positive: # incorrectly inferred lossy links

11

Random topologies Gibbs sampling for a 1000-node random topology (d = 10, f = 0.5)

0

100

200

300

400

500

600

0 200 400 600 800 1000

# li

nk

s

"# correctly identified lossy links""# true lossy links""# false positive"

Confidence estimate for gibbs sampling works welland can be used to rank order the inferred lossy links.

12

Trace-driven Validation Validation approach

Divide client traces into two: tomography and validation Tomography data set loss inference Validation set check if clients downstream of the inferred lossy

links experience high loss Experimental setup

Real topologies and loss traces collected from traceroute and tcpdump at microsoft.com during Dec. 20, 2000 and Jan. 11, 2002

Results For the small subset of inferences that could be validated, all the

inferences are correct Likely candidates for lossy links:

links crossing an inter-AS boundary links having a large delay (e.g. transcontinental links) links that terminate at clients

13

Summary Passive network tomography is feasible

Gibbs sampling yields a high coverage (over 80%), and a low false positive rate (below 5-10%)

We have developed two other inference techniques which trade-off accuracy for speed (more details in “Server-based Inference of Internet Performance”, to appear in INFOCOM’03)

Future work: make loss inference in real time Acknowledgements:

Chris Meek, David Wilson, Christian Borgs, Jennifer Chayes, David Heckerman