a non-intrusive, wavelet-based approach to detecting network performance problems polly huangeth...
TRANSCRIPT
![Page 1: A Non-intrusive, Wavelet-based Approach To Detecting Network Performance Problems Polly HuangETH Zurich Anja FeldmannU. Saarbruecken Walter WillingerAT&T](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515065f550346c77d8b45e2/html5/thumbnails/1.jpg)
A Non-intrusive, Wavelet-based Approach To Detecting Network Performance Problems
Polly Huang ETH ZurichAnja Feldmann U. SaarbrueckenWalter Willinger AT&T Labs-Research
![Page 2: A Non-intrusive, Wavelet-based Approach To Detecting Network Performance Problems Polly HuangETH Zurich Anja FeldmannU. Saarbruecken Walter WillingerAT&T](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515065f550346c77d8b45e2/html5/thumbnails/2.jpg)
Road Map
Motivation and rationaleMechanism detailsConclusion and outlook
![Page 3: A Non-intrusive, Wavelet-based Approach To Detecting Network Performance Problems Polly HuangETH Zurich Anja FeldmannU. Saarbruecken Walter WillingerAT&T](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515065f550346c77d8b45e2/html5/thumbnails/3.jpg)
Performance Problem
Web
TCP
Network
Link/Physical
Web
TCP
Network
Link/Physical
Google.com
congestionroutingserver
else
Internet
Web
TCP
NetworkLink/Physical
congestionroutingproxyelse
![Page 4: A Non-intrusive, Wavelet-based Approach To Detecting Network Performance Problems Polly HuangETH Zurich Anja FeldmannU. Saarbruecken Walter WillingerAT&T](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515065f550346c77d8b45e2/html5/thumbnails/4.jpg)
Current State
Active probing Ex: traceroute, ping Disturbing - injecting unnecessary traffic Biasing - distort metrics of interest
‘Heisenberg’ effects
Passive measurements Ex: Cisco NetFlow, IP Accounting, other packet-
level measurment give much information Do not infer problems inside the network
![Page 5: A Non-intrusive, Wavelet-based Approach To Detecting Network Performance Problems Polly HuangETH Zurich Anja FeldmannU. Saarbruecken Walter WillingerAT&T](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515065f550346c77d8b45e2/html5/thumbnails/5.jpg)
What Would Be Cool
PassiveTrigger alerts in real time For problems due to
Server load Congestion Routing error
Common Symptoms Delay and drop
![Page 6: A Non-intrusive, Wavelet-based Approach To Detecting Network Performance Problems Polly HuangETH Zurich Anja FeldmannU. Saarbruecken Walter WillingerAT&T](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515065f550346c77d8b45e2/html5/thumbnails/6.jpg)
TCP’s Closed-loop Control
Delays/drops reflected in RTT/RTO estimations RTT: round trip time RTO: retransmission timeout
Quality of Network Path Values of RTT/RTO estimations Amounts of RTT/RTO samples
Can be measured passively
![Page 7: A Non-intrusive, Wavelet-based Approach To Detecting Network Performance Problems Polly HuangETH Zurich Anja FeldmannU. Saarbruecken Walter WillingerAT&T](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515065f550346c77d8b45e2/html5/thumbnails/7.jpg)
Detailed Estimation
Methodology A hash table of all data packets
observed One RTT sample per data-ack pair One RTO sample per data-data pair
Slow ~ #packets/observation period especially with high date rate
connections (the likely trouble makers)
![Page 8: A Non-intrusive, Wavelet-based Approach To Detecting Network Performance Problems Polly HuangETH Zurich Anja FeldmannU. Saarbruecken Walter WillingerAT&T](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515065f550346c77d8b45e2/html5/thumbnails/8.jpg)
Objectives
Passive measurement Non-intrusive
Infer quality of network paths Detecting network performance
problemEfficiently (so can be done in real
time) Wavelet-based technique
![Page 9: A Non-intrusive, Wavelet-based Approach To Detecting Network Performance Problems Polly HuangETH Zurich Anja FeldmannU. Saarbruecken Walter WillingerAT&T](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515065f550346c77d8b45e2/html5/thumbnails/9.jpg)
Road Map
Motivation and rationaleMechanism detailsConclusion and outlook
![Page 10: A Non-intrusive, Wavelet-based Approach To Detecting Network Performance Problems Polly HuangETH Zurich Anja FeldmannU. Saarbruecken Walter WillingerAT&T](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515065f550346c77d8b45e2/html5/thumbnails/10.jpg)
Wavelet-based Technique
Theoretical ground Wavelet transform Energy plots (or scaling plots) Interpreting energy plots
WIND, the problem detection tool Features & examples Detection methodology Validation effort
![Page 11: A Non-intrusive, Wavelet-based Approach To Detecting Network Performance Problems Polly HuangETH Zurich Anja FeldmannU. Saarbruecken Walter WillingerAT&T](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515065f550346c77d8b45e2/html5/thumbnails/11.jpg)
Theoretical Ground
FFT Frequency decomposition fj, Fourier coefficient Amount of the signal in frequency j
WT: wavelet transform Frequency (scale) and time decomposition dj,k, wavelet coefficient Amount of the signal in frequency j, time k
![Page 12: A Non-intrusive, Wavelet-based Approach To Detecting Network Performance Problems Polly HuangETH Zurich Anja FeldmannU. Saarbruecken Walter WillingerAT&T](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515065f550346c77d8b45e2/html5/thumbnails/12.jpg)
Wavelet Example
0-1
1
00 00 00 00 11 11 11 11
s1
s2
s3
s4
d1
d2
d3
d4
0 0 0 0 2 2 2 2 0 0 0 0 0 0 0 0
0 0 4 4 0 0 0 0
0 8 0 0
8 8
![Page 13: A Non-intrusive, Wavelet-based Approach To Detecting Network Performance Problems Polly HuangETH Zurich Anja FeldmannU. Saarbruecken Walter WillingerAT&T](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515065f550346c77d8b45e2/html5/thumbnails/13.jpg)
Self-similarity
Energy function Ej = Σ(dj,k)2/Nj
Self-similar process Ej = 2j(2H-1) C <- the magic!!
log2 Ej = (2H-1) j + log2C
linear relationship between log2 Ej and j
![Page 14: A Non-intrusive, Wavelet-based Approach To Detecting Network Performance Problems Polly HuangETH Zurich Anja FeldmannU. Saarbruecken Walter WillingerAT&T](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515065f550346c77d8b45e2/html5/thumbnails/14.jpg)
Self-similar Traffic
![Page 15: A Non-intrusive, Wavelet-based Approach To Detecting Network Performance Problems Polly HuangETH Zurich Anja FeldmannU. Saarbruecken Walter WillingerAT&T](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515065f550346c77d8b45e2/html5/thumbnails/15.jpg)
Effect of Periodicity
self-similar
Internet Traffic
![Page 16: A Non-intrusive, Wavelet-based Approach To Detecting Network Performance Problems Polly HuangETH Zurich Anja FeldmannU. Saarbruecken Walter WillingerAT&T](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515065f550346c77d8b45e2/html5/thumbnails/16.jpg)
Adding Periodicity
packets arrive periodically, 1 pkt/23 msec
coefficients cancel out at scale 410 00 00 00 10 00 00 00
s1
s2
s3
s4
d1
d2
d3
d4
1 0 0 0 1 0 0 0 1 0 0 0 1 0 0 0
1 0 1 0 1 0 1 0
1 1 1 1
2 0
![Page 17: A Non-intrusive, Wavelet-based Approach To Detecting Network Performance Problems Polly HuangETH Zurich Anja FeldmannU. Saarbruecken Walter WillingerAT&T](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515065f550346c77d8b45e2/html5/thumbnails/17.jpg)
Simulation TrafficSingle RTT
![Page 18: A Non-intrusive, Wavelet-based Approach To Detecting Network Performance Problems Polly HuangETH Zurich Anja FeldmannU. Saarbruecken Walter WillingerAT&T](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515065f550346c77d8b45e2/html5/thumbnails/18.jpg)
Simulation TrafficCongestion
![Page 19: A Non-intrusive, Wavelet-based Approach To Detecting Network Performance Problems Polly HuangETH Zurich Anja FeldmannU. Saarbruecken Walter WillingerAT&T](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515065f550346c77d8b45e2/html5/thumbnails/19.jpg)
Interpreting Energy Functions
Abrupt knees at RTT time scale RTO time scale
Knee shifts RTT/RTO time changes
Low energy level (after normalization) congestion low traffic volume
![Page 20: A Non-intrusive, Wavelet-based Approach To Detecting Network Performance Problems Polly HuangETH Zurich Anja FeldmannU. Saarbruecken Walter WillingerAT&T](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515065f550346c77d8b45e2/html5/thumbnails/20.jpg)
WIND - The Detection Tool
Wavelet-based Inference for Network Detection
Based on libpcap and tcpdumpOn-line mode (efficient)
Per packet: compute dj,k
Per observation period: output Ej
On a subnet basisOff-line mode
Detailed RTT/RTO estimation
![Page 21: A Non-intrusive, Wavelet-based Approach To Detecting Network Performance Problems Polly HuangETH Zurich Anja FeldmannU. Saarbruecken Walter WillingerAT&T](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515065f550346c77d8b45e2/html5/thumbnails/21.jpg)
Real TrafficBy Subnets
![Page 22: A Non-intrusive, Wavelet-based Approach To Detecting Network Performance Problems Polly HuangETH Zurich Anja FeldmannU. Saarbruecken Walter WillingerAT&T](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515065f550346c77d8b45e2/html5/thumbnails/22.jpg)
Real TrafficBy Periods
![Page 23: A Non-intrusive, Wavelet-based Approach To Detecting Network Performance Problems Polly HuangETH Zurich Anja FeldmannU. Saarbruecken Walter WillingerAT&T](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515065f550346c77d8b45e2/html5/thumbnails/23.jpg)
Real TrafficBy Periods
![Page 24: A Non-intrusive, Wavelet-based Approach To Detecting Network Performance Problems Polly HuangETH Zurich Anja FeldmannU. Saarbruecken Walter WillingerAT&T](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515065f550346c77d8b45e2/html5/thumbnails/24.jpg)
Detecting Methodology
Reference function Smoothed average
Difference Area below the reference function Weighted sum by scale
Flagged interesting Top 10% deviations
![Page 25: A Non-intrusive, Wavelet-based Approach To Detecting Network Performance Problems Polly HuangETH Zurich Anja FeldmannU. Saarbruecken Walter WillingerAT&T](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515065f550346c77d8b45e2/html5/thumbnails/25.jpg)
Pick Out Interesting Ones26, 30, 31
![Page 26: A Non-intrusive, Wavelet-based Approach To Detecting Network Performance Problems Polly HuangETH Zurich Anja FeldmannU. Saarbruecken Walter WillingerAT&T](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515065f550346c77d8b45e2/html5/thumbnails/26.jpg)
Validation By
WIND off-line mode Detailed RTT/RTO estimations Volume
Similar heuristics (area difference) CCDF of RTT/RTO Ratio of RTO/RTT Volume
![Page 27: A Non-intrusive, Wavelet-based Approach To Detecting Network Performance Problems Polly HuangETH Zurich Anja FeldmannU. Saarbruecken Walter WillingerAT&T](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515065f550346c77d8b45e2/html5/thumbnails/27.jpg)
Validate period 26, 30, 31
CCDF of RTO: pick out period 23, 26, 31
CCDF of RTT:pick out period 29, 30, 31
80-90% are validated interesting80-90% are validated interesting
![Page 28: A Non-intrusive, Wavelet-based Approach To Detecting Network Performance Problems Polly HuangETH Zurich Anja FeldmannU. Saarbruecken Walter WillingerAT&T](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515065f550346c77d8b45e2/html5/thumbnails/28.jpg)
Road Map
Motivation and rationaleMechanism detailsConclusion and outlook
![Page 29: A Non-intrusive, Wavelet-based Approach To Detecting Network Performance Problems Polly HuangETH Zurich Anja FeldmannU. Saarbruecken Walter WillingerAT&T](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515065f550346c77d8b45e2/html5/thumbnails/29.jpg)
Summary
Detect problems using energy plots If self-similar, clean linear relationship If periodic, getting knees If problems, knee shifts or low energy level
WIND: the online/offline analysis tool
Passive Efficient
![Page 30: A Non-intrusive, Wavelet-based Approach To Detecting Network Performance Problems Polly HuangETH Zurich Anja FeldmannU. Saarbruecken Walter WillingerAT&T](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515065f550346c77d8b45e2/html5/thumbnails/30.jpg)
Outlook
Full-fledged diagnosing tool More sophisticated heuristics Use of traceroute data
Illustrative examples Using the tool (beta release) Using the methodology
![Page 31: A Non-intrusive, Wavelet-based Approach To Detecting Network Performance Problems Polly HuangETH Zurich Anja FeldmannU. Saarbruecken Walter WillingerAT&T](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515065f550346c77d8b45e2/html5/thumbnails/31.jpg)
Questions?
http://www.tik.ee.ethz.ch/~huang