Multiscale Traffic Processing Techniques for Network Inference and Control

R. Baraniuk R. Nowak E. Knightly R. Riedi

V. Ribeiro S. Sarvotham A. Keshavarz R. King

NMS PI meeting Monterey November 2004

SPiN Signal Processing in Networking

Rice University spin.rice.edu

Spatio-Temporal Available Bandwidth Estimation

On-line localization of the tight link in a path

Effort 1

Rice University spin.rice.edu

Key Definitions

• Available bandwidth: left-over capacity on link• Tight link: link with least available bandwidth • Goal:

• locate tight link in space and over time• using end-to-end probing

Rice University spin.rice.edu

Applications

• Network monitoring - locating hot spots

• Network aware applications- Server selection- Route selection

• Science: where do Internet tight links occur and why?

Rice University spin.rice.edu

Methodology

iiBA min

Path available bandwidth

Sub-path available bandwidth

imiBmA

1min],1[

Methodology:• For m>tight link, A[1,m] remains constant

Rice University spin.rice.edu

Packet Tailgating

• Packet train contains:– Large packets stressing, with m hops life time– Small packets tailgating, full life time

• Purpose:– Large packets “measure” bandwidth via their delay– Small packets transport this timing information to the

receiver

Rice University spin.rice.edu

Lite-probing: pathChirp• real world tool

– Queuing delay cross traffic – Averaged excursions available

resources• Light weight

– Probe at various rates simultaneously• …converges in a handful of RTTs

Departure pattern

Queuing against departure

Methodology

Number of chirps

12 chirps

Real world experiments

Estimation against true x-trafficInternet experiment

Rice University spin.rice.edu

Bandwidth: a Probabilistic entity• Available bandwidth depends on temporary

congestion level of potential tight linksUIUC – Rice

Probability of being tight linkEstimates taken 10 mins apart

REAL WORLD EXPERIMENTSUIUC (J. Hou)– Rice

Available sub-path bandwidth

spacetimeavail

Rice University spin.rice.edu

STAB: Spatio Temporal available Bandwidth

• STAB detects new tight link and reduced available bandwidth around 250 secs into simulation

ns2 Simulation setting:Double web farm in ns2(420 clients, 40 servers)

Estimate

Truth

Rice University spin.rice.edu

GUI: ease of configuration• Running on windows• Instrumental for distribution and transfer

Rice University spin.rice.edu

GUI in action

• See Demo

Connection-level Analysis and Modeling of Network Traffic

understanding the cause of burstsdetect changes of network state

Effort 2

Rice University spin.rice.edu

Bursts and DominanceConnection level separation:

– Detrimental bursts caused by the ONE strongest connection– ….called Alpha connections

Origin of Alpha:– High rate today from small RTT (round trip time)– Not congestion controlled

Beta connections: – All the rest– Well controlled

Overall traffic Residual trafficBeta

1 Strongest connectionAlpha

= +Mean

99%

Rice University spin.rice.edu

Burst Model• Alpha traffic = High rate ON-OFF source

– bottleneck at the receiver (TCP advertised window) Rate determined by RTT

– Current state of measured traffic• Analysis: Queues will explode with TCP’s ability to

achieve large rates (HSTCP, BIC)

Beta (top) + Alpha Variable Service Rate Queue-tail Weibull (as for self-similar traffic) unless

• rate of alpha traffic larger than available bandwidth • and duration of alpha ON period heavy tailed

Rice University spin.rice.edu

Free parameters

Total Alpha Beta

Duration - Rate

Duration - Size

Size - Rate

X

• Beta users: rate determines file size• Alpha users are “free”

Scheme RD:Rate Durationindependent

Scheme SD:Size Durationindependent

Scheme SR: Size Rate

independentTotal

Alpha

Beta

Real TraceSIMULATION

Rice University spin.rice.edu

Network-User Driven Traffic modelMixture fit of alpha-beta

• Alpha: free to choose files RATE DISTRIBUTIONS• Beta: patience factor FILE SIZE DISTRIBUTIONS

• RAPID PERFORMANCE ASSESSMENT

• TCP Control: manages only BETA traffic effectively Congestion and admission control

Original trace (Bellcore) Alpha (SR) + Beta (RD)

Rice University spin.rice.edu

Effort 3

Model based Protocols

Rice University spin.rice.edu

Prediction and what if scenarios• High-performance protocols pushed

– HSTCP– STCP – XCP– FAST-TCP– BI-TCP

• RTT bias– alpha-beta differentiation between flows more pronounced

for STCP and HSTCP (which have large RTT bias).• Need for RTT-fair high-performance TCP

Rice University spin.rice.edu

TCP Africa• Hybrid two modes

– Fast mode: (absence of congestion)• Rapid, opportunistic increase of window (rate)

– Slow mode: (presence of congestion)• Linear (slow) increase in congestion avoidance

– Congestion inference: • Current average RTT – minimal RTT (Vegas-type)

Adaptive and Fair Rapid Increase Congestion Avoidance

Rice University spin.rice.edu

TCP Africa

• Hybrid two modes– Fast mode: (absence of congestion)

• Rapid, opportunistic increase of window (rate)– Slow mode: (presence of congestion)

• Linear (slow) increase in congestion avoidance– Congestion inference:

• Current average RTT – minimal RTT (Vegas-type)• Induces LOSSES infrequently (like Reno)• Combines aggressiveness of HSTCP with

reliability and low loss induction of Reno

Rice University spin.rice.edu

RTT Fairness• Against peers with different RTT• HSTCP: low RTT overwhelms

• Africa: RTT bias is comparable to Reno

Rice University spin.rice.edu

Safety • Degrade performance of other flows • …as compared to normal conditions• ns2 sim: Reno over 100Mbps link• …and with 1 Gbps

HSTCP poor Africa acceptableHSTCP poor Africa acceptable

Rice University spin.rice.edu

Impact and Transfer

Rice University spin.rice.edu

Software

• STAB • pathChirp• Alpha-Beta decomposition

• User Interface on Windows (GUI)– 80% completed

• Free, available at spin.rice.edu

Rice University spin.rice.edu

PublicationsEnable transition to DoD contractors

– Note and understand– Write own code

• IEEE Internet Computing Magazine– pathChirp and STAB

• Computer Networks – Special issue on LRD traffic– Alpha-Beta / Network-User driven traffic model

• IEEE Signal Processing – special issue on SPiN

• IEEE SP Magazine– Special issue on Complexity in Networking– Network modeling, MWM, role of multifractal scaling

• InfoCom 2005– TCP Africa

Rice University spin.rice.edu

Tech Transfer and Integration• GUI: Pathchirp Running on Windows• Raytheon: Doug Fowler discussing transitions pathChirp• Sensor networks: Steve Beck (BAE Austin)• Hitachi

– David Diep makes pathChirp IPv4 and IPv6 compatible• Computer Sciences Corporation (DoD contractor)

– Steve Tsang uses MWM for DSN VoIP User Interface• GridLab Project (Verstoep)

– Deployed pathChirp for Grid computing measurements• SPAWAR (consulted Phuong Nguyen)• J9 (consulted Jasom Boyer)• GaTech (Riley-Fujimoto)

– On-line pathChirp inference in integrated demo to detect UDP storms• UC Riverside (Faloutsos)

– On-line Traffic estimation / demystify LRD• UIUC (Hou) and ISI (Heidemann)

– Integration of probing schemes into network – simulators JavaSim and ns-2

• SLAC (Cottrell) – Large scale monitoring using pathChirp

Rice University spin.rice.edu

Ongoing work• pathChirp: chirp-web

– Tight links on high speed networks– Anomaly detection through chirp-web

• : Network/user-driven traffic model– Through simulation and measurements assess

impact of protocols, applications, clientele, end-host server

– Parameters from network and user specifications

• High-speed protocols and congestion control– continue to integrate advanced modeling/probing

    techniques into new protocols