trajectory sampling for direct traffic observation
Post on 13-Feb-2016
39 Views
Preview:
DESCRIPTION
TRANSCRIPT
Trajectory Sampling forDirect Traffic Observation
Matthias Grossglauser
joint work with Nick Duffield
AT&T Labs – Research
Traffic Engineering
overload!
Two large flows
Traffic Engineering
overload!New egress point
for first flow
Multi-homed customer
Traffic Engineering
overload!
OSPF shortest path splitting
Traffic Engineering• Goal: domain-wide control & management to
– Satisfy performance goals– Use resources efficiently
• Knobs:– Configuration & topology: provisioning, capacity
planning– Routing: OSPF weights, MPLS tunnels, BGP policies,
…– Traffic classification (diffserv), admission control,…
• Measurements are key: closed control loop– Characterize demand: what’s coming in?– Observe network state: how is the network
reacting? (low-level adaptivity!)– Check performance: what’s the customer’s QoS?
Traffic Matrix vs. Path Matrix
• Traffic matrix– # bytes from ingress i to egress j
• Path matrix– Spatial flow of traffic through domain– # bytes for every path from i to j
flow 1 flow 2 flow 3 flow 4
Flow Measurement
• IP flow abstraction– Set of packets with “same” src and dest IP
addresses– Packets that are “close” together in time (a
few seconds)• Cisco NetFlow
– Router maintains a cache of statistics about active flows
– Router exports a measurement record for each flow
Inferring the Path Matrix from the Traffic Matrix
Network State Uncertainty• Hard to get an up-to-date snapshot of…• …routing
– Large state space– Vendor-specific implementation– Deliberate randomness– Multicast
• …element states– Links, cards, protocols,…
• …element performance– Packet loss, delay at links
missing “down” alarms spurious down
noise
missing alarms
Direct Traffic Observation• Goal: direct observation
– No network model & state estimation• Basic idea:
– Sample packets at each link– Sampling decision based on hash over packet
content– Consistent sampling trajectories– Labels based on second hash function
• Exploit entropy in packet content to obtain statistically representative set of trajectories
Sampling and Labeling
• Fields of interest collected only once• Multicast: trajectory is a tree
Fields Included in Hashes
Collisions: Identical Packets
Sampling and Labeling Hashes
• x: subset of packet bits, represented as binary number
• Sampling hash– h(x) = x mod A– Sample if h(x) < r– r/A: thinning factor
• Labeling hash– g(x) = x mod M
• Make appropriate choice of A, M– predictable patterns should “mix” well
Pseudo-Random Sampling• Goal: infer metrics of interest from
trajectory samples– E.g., what fraction of traffic of
customer x on a link y?• Question: is sample set statistically
representative?– Obvious for “really random” sampling– Distribution of a field in the sampled
subset = real distribution?– In other words: does the complement
of the field provide enough entropy?
Quality of Deterministic Sampling
• Experiment: statistical test to check if sampled and full distributions are close– Chi-square statistic to verify independence
hypothesis– Hypothesis: sampled distribution consistent
with full distribution
– Confidence level C(T) for hypothesis, where C is cdf of with I-1 degrees of freedom2
jn j bin in samples # :
nnnnmmmmmmmm
I
I
I
...
...
...
21
111211
000201
1
0 1
2)(
i
I
j ij
ijij
mmm
T
jmij bin in d(un)sample packets # :
Chi-square Test on Source AddressIf , then accept hypothesis 1)(TC
Bitwise Independence• 2x2 contingency table formed by
– sampling decision– l-th bit of packet
Optimal Sampling
• Fix amount of measurement traffic c per time period
• Problem:– n: number of samples in sampling period– M: alphabet size, m=log2(M) bits/label– nm: total amount of measurement traffic [bits]– Goal: maximize # unique labels, subject to nm<c
• Result:– optimal alphabet size M*=c log(2)– optimal number of samples n*=M*/log(M*)– example: c=1Mb/period
Label Collisions and Trajectory Ambiguity
Ambiguity cont.
• Rule for acyclic subgraphs + unicast packets:– unambiguous if each connected component of the subgraph is
• (a) a source tree• (b) a sink tree without loss
InferenceExperiment
• Experiment: infer from trajectory samples– Estimate fraction of traffic from customer– Source address customer– Source address sampling + label
• Fraction of customer traffic on backbone link:
b on labels unique #cb, on common labels unique #ˆ
Estimated Fraction (c=1000bit)
Estimated Fraction (c=10kbit)
Sampling Device
MPLS: simple additional logic to look “behind” label stack
Sampling Device Implementation
• Interface vs. processing speed– OC-192: 10 Gbps– State of the art DSP:
• Proc: 600M MACs x 32 bit: 20 Gbps• I/O: 300MHz x 256 bit: 70 Gbps
– Moore’s law vs. interface speed growth• Vendor interest: cisco, juniper, avici
Summary• Advantages
– Trajectory sampling estimates path matrix…and other metrics: loss, link delay
– Direct observation: no routing model + network state estimation
– No router state– Multicast (source tree), DDoS (sink tree)– Control over measurement overhead– Small measurement delay
• Disadvantages– Requires support on linecards
• Open questions & research problems– Collection, storage, querying (in progress)– Management interface
top related