A Survey of Recent High Speed TCP VariantsPhilip Fisher-OgdenU.C. Davis

March 16th, 2007

Outline1. Why do we need high speed TCP variants?

2. Three recent variants Compound TCP CUBIC TCP-Adaptive Reno

3. Recent Evaluations

4. Q&A

Standard TCP - advantages Widely used Standardized Promises, promises

Guaranteed, in-order delivery Avoids congestion collapse

Evolved: Tahoe, Reno/NewReno, SACK

Standard TCP - disadvantages Underutilization of the bandwidth in fast long

distance networks (FLDnets) Low window size resiliency to packet loss in

FLDnets

Prior High Speed TCPs Effectively use available bandwidth Unfriendly – “doesn’t play nicely with

others” Unfair to different RTT flows Examples: HSTCP, STCP, BIC-TCP

Goals TCP friendly

Minimize bandwidth-stolen RTT fair

Co-existing flows with different RTTs are treated fairly

Efficient use of available bandwidth

Compound TCP

Compound TCP - Motivation Loss-based - uses packet loss as an indicator of

congestion and modifies the increase/decrease congestion avoidance TCP parameters.

Delay-based - infers congestion and bottleneck queue size from changes in the RTT and modifies the transfer rates to mitigate the effects of congestion.

Combine both approaches Focus on efficiency & friendliness

Compound TCP - Design Maintains a delay window and a congestion

window – uses both to determine send window

Estimates the bottleneck queue size: Diff = (Expected – Actual) * baseRTT

Has a threshold, γ, at which it says “the network is congested”

Compound TCP Delay Window CalculationsDelay window Formula Meaning

dwnd(t+1) = dwnd(t) + (α * win(t)k – 1)+, if diff < γ In the increase phase, CTCP scales up the win (a composite of cwnd and dwnd) by α * win(t)k.

(dwnd(t) – ζ * diff )+, if diff ≥ γ When a bottleneck queue is detected, dwnd decreases proportional to the estimated bottleneck size diff.

(win(t) * (1 - β) – cwnd/2)+, if loss is detected (3 dup acks)

win(t) * (1 - β) is the desired window size after loss occurs, so dwnd is set to the desired size minus the portion that cwnd already provides.

Note: (...)+ = max(..., 0)

Compound TCP - Design Builds on loss-based (standard) TCP Reverts to standard TCP at low window sizes Uses gamma-auto tuning to dynamically

adjust γ, the estimated number of packets that indicate a bottleneck in the queue, based on the network configuration

Compound TCP - EvaluationProduction FLD Network Link Tests (comparing CTCP & TCP)

Test Results

Throughput CTCP improves the throughput by 28% to 52% over regular TCP.

TCP friendliness With gamma auto-tuning, CTCP keeps good TCP friendliness (<20% bandwidth stolen) even in the presence of multiple concurrent flows.

Reverse traffic CTCP improves the throughput over TCP even in the presence of many reverse flows.

CUBIC

CUBIC - Motivation Enhance BIC!

Maintain BIC’s scalability & stability Simplify the window control Improve BIC’s friendliness Use real-time, rather than ACK-clocked, updates

to window Improve the detection of the “TCP Region”

CUBIC - Design Based on BIC:

CUBIC - Design Window growth function:

CUBIC – Design Aspects Stability – The window grows slowly around Wmax. Scalability – It experiences fast “probing” growth

away from Wmax. Intra-protocol fairness - Two competing CUBIC

flows will converge to fair share window sizes. Fairness – The window growth rate is time

dependent and RTT independent, allowing for fairer sharing.

CUBIC – Design Aspects Standard TCP outperforms CUBIC’s window

growth function in short RTT networks. CUBIC emulates the time-independent TCP

window adjustment algorithm so that it can select the greater of the two windows (emulated versus cubic) to use.

CUBIC - EvaluationNS-2 Tests (comparing TCP, CUBIC, BIC, HSTCP, STCP, & HTCP)

Test Results

TCP Friendliness in Short-RTT Networks

CUBIC TCP maintained the best overall fair-share throughput ratio with regular TCP, and even had less throughput than regular TCP in the higher speed networks.

TCP Friendliness in Long-RTT Networks

As the link speed increased the throughput ratios of high speed to standard decreased. CUBIC TCP showed the best friendly ratio to regular TCP across all bottleneck link bandwidths in this experiment.

Stability The CoV measurements showed that CUBIC TCP provides good stability in the presence of network perturbations and varying router buffer sizes.

TCP-Adaptive Reno (AReno)

TCP-AReno - Motivation Enhance TCP-Westwood-BBE Improve on TCP-Reno’s response function by

incorporating a congestion estimate that is based on RTT measurements.

Identify if a packet loss was due to congestion or not.

TCP-AReno - Design Congestion Estimate

RTTcong = the RTT that would indicate a congestion event.

Classifies packet losses as due to congestion or not

RTT values vary between RTTmin and RTTcong. The distance of recent RTT measurements from RTTcong and RTTmin determines the level of congestion in the network.

TCP-AReno - Design Congestion Window Increase

TCP-AReno - Design Congestion Window Reduction

TCP-AReno - EvaluationLaboratory Tests (comparing TCP-AReno & TCP-Reno)

Test Results

Congestion windows of lone flows

TCP-AReno is able to increase its congestion window when the link was under-utilized and then limit the increase when the window approached or exceeded the congestion point. TCP-AReno distinguishes between packet loss events that were congestion based or not, and then adapts its congestion window reduction accordingly.

Congestion windows of co-existing flows

TCP-AReno averaging 227.4Mbps (vs. 232.1) and TCP-Reno averaging 18.5Mbps (vs. 18.3). TCP-AReno is resilient to random packet losses and is able to attain high throughput without negatively affecting TCP-Reno flows.

Throughput of a lone flow under various bottleneck capacities

TCP-AReno is effective in high-speed and lossy environments, obtaining 10x more throughput than TCP-Reno when the packet loss rate is 10-4.

Recent Evaluations

Recent EvaluationsLinux beats Windows!

TCP variant Operating System

BIC Current Linux

CUBIC Possibly future Linux

Compound TCP (CTCP) Possibly future Windows Vista

TCP Any other operating system

Recent EvaluationsLinux beats Windows!

BIC/CUBIC are over aggressive and steal bandwidth.

Serious unfairness

Recent EvaluationsExperimental Evaluation of Cubic-TCP

Conducted partially in response to “should CUBIC be adopted in Linux?”

CUBIC suffers from a slow convergence of congestion windows.

Controversial! A rebuttal was issued by Injong Rhee (CUBIC co-author)

Recent EvaluationsExperimental Evaluation of Cubic-TCP

Example of slow convergence:

Recent EvaluationsAssessing Interactions among Legacy & High-Speed TCPs

Evaluated efficiency, fairness, & friendliness

Recent EvaluationsAssessing Interactions among Legacy & High-Speed TCPs

Delay-based control are not effective in improving RTT-fairness due to the slow-start behavior of short flows that induces RTT measurement problems

TCP-AReno was modified to avoid having the short flow problem impact its delay based mechanism.

TCP-AReno outperformed all others. Compound TCP came close, but experienced the

delay-based control problem.

SummaryProtocol Approach Advantages Disadvantages

Compound TCP Combines a delay based component with TCP Reno’s loss based component.

Outperforms the link utilization of HSTCP and TCP Reno while still maintaining fairness to TCP Reno flows.

The delay based component works best when in a steady state, but its effectiveness is reduced by the slow start behavior of lots of short flows.

CUBIC Builds on BIC by simplifying the window response function and improving its TCP friendliness and RTT fairness through RTT-independent congestion window updates.

Expands the TCP region to utilize standard TCP when it performs well. Maintains BIC’s scalability and stability regardless of RTT.

Suffers from long convergence times between competing CUBIC flows.

TCP-Adaptive Reno Incorporates a congestion estimation component that allows it to differentiate between packet losses from congestion or not, while also using RTT measurement to dynamically adjust its window response function.

Offers the best link utilization, RTT fairness, and TCP friendliness of all the high speed variants.

The most recent modifications have not yet been evaluated by an outside party.

Q&A