High Performance Networking with

Little or No Buffers

Yashar Ganjali

High Performance Networking GroupStanford University

yganjali@stanford.eduhttp://www.stanford.edu/~yganjali

Joint work with:Guido Appenzeller, Ashish Goel, Tim Roughgarden, Nick McKeown

May 5, 2005

May 5, 2005 2

MotivationNetworks with Little or No

Buffers Problem

Internet traffic is doubled every year Disparity between traffic and router

growth (space, power, cost) Possible Solution

All-Optical Networking Consequences

Large capacity large traffic Little or no buffers

May 5, 2005 3

Which would you choose?

DSL Router 1 DSL Router 2

$504 x 10/100 Ethernet

1.5Mb/s DSL connection

1Mbit of packet buffer

$554 x 10/100 Ethernet

1.5Mb/s DSL connection

4Mbit of packet buffer

Bigger buffers are better

May 5, 2005 4

What we learn in school

Packet switching is good Long haul links are expensive Statistical multiplexing allows efficient

sharing of long haul links Packet switching requires buffers Packet loss is bad Use big buffers Luckily, big buffers are cheap

May 5, 2005 5

Statistical Multiplexing

Observations1. The bigger the buffer, the lower the packet loss.2. If the buffer never goes empty, the outgoing line

is busy 100% of the time.

May 5, 2005 6

What we learn in school

Queueing Theory

kkXP

EX

1

11

M/M/1

X

Buffer size

Loss rate

Observations1. Can pick buffer size for a given loss rate.2. Loss rate falls fast with increasing buffer size.3. Bigger is better.

May 5, 2005 7

What we learn in school

Moore’s Law: Memory is plentiful and halves in price every 18 months. 1Gbit memory holds 500k packets

and costs $25.

Conclusion: Make buffers big. Choose the $55 DSL router.

May 5, 2005 8

Why bigger isn’t better

Network users don’t like buffers Network operators don’t like buffers Router architects don’t like buffers We don’t need big buffers We’d often be better off with smaller

ones

May 5, 2005 9

Backbone Router Buffers

Universally applied rule-of-thumb: A router needs a buffer size:

• 2T is the two-way propagation delay• C is capacity of bottleneck line

Context Mandated in backbone and edge routers. Appears in RFPs and IETF architectural guidelines.. Usually referenced to Villamizar and Song: “High

Performance TCP in ANSNET”, CCR, 1994. Already known by inventors of TCP [Van Jacobson, 1988] Has major consequences for router design

CTB 2

CRouterSource Destination

2T

May 5, 2005 10

Review: TCP Congestion Control

Only W packets may be outstanding

Rule for adjusting W If an ACK is received: W ← W+1/W If a packet is lost: W ← W/2

May 5, 2005 11

Review: TCP Congestion Control

Only W packets may be outstanding

Rule for adjusting W If an ACK is received: W ← W+1/W If a packet is lost: W ← W/2

Source Dest

maxW

2maxW

t

Window size

May 5, 2005 12

Buffer Size in the Core

ProbabilityDistribution

B

0

Buffer Size

W

May 5, 2005 13

Backbone router buffers

It turns out that The rule of thumb is wrong for a core

routers today

Required buffer is instead of CT 2n

CT 2

May 5, 2005 14

2T C

n

Simulation

Required Buffer Size

May 5, 2005 15

Validation

Theoretical results validated by: Thousands of ns2 simulations Network lab (Cisco routers) at University

of Wisconsin Stanford University dorm traffic Internet2 experiments

Ongoing work with network operators and router vendors…

May 5, 2005 16

Impact on Router Design

10Gb/s linecard with 200,000 x 56kb/s flows Rule-of-thumb: Buffer = 2.5Gbits

• Requires external, slow DRAM Becomes: Buffer = 6Mbits

• Can use on-chip, fast SRAM• Completion time halved for short-flows

40Gb/s linecard with 40,000 x 1Mb/s flows Rule-of-thumb: Buffer = 10Gbits Becomes: Buffer = 50Mbits

May 5, 2005 17

How small can buffers be?

Imagine you want to build an all-optical router for a backbone network…

…and you can build a few dozen packets in delay lines.

Conventional wisdom: It’s a routing problem (hence deflection routing, burst-switching, etc.)

Our belief: First, think about congestion control.

May 5, 2005 18

TCP with ALMOST No Buffers

Utilization of bottleneck link = 75%

May 5, 2005 19

Two Concurrent TCP Flows

May 5, 2005 20

TCP Throughput with Small Buffers

TCP Throughput vs. Number of Flows

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

0 200 400 600 800 1000Number of Flows

Th

rou

gh

pu

t

May 5, 2005 21

TCP Reno Performance

Buffer Size = 10; Load = 80%

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 1000 2000 3000 4000 5000 6000

Bottleneck Capacity Mbps

Th

rou

gh

pu

t

May 5, 2005 22

The chasm between theory and practice

M/M/1

11

X

kkXP

EX

1

= 50%, EX = 1 packet = 75%, EX = 3 packets

= 50%, P[X>10] < 10-3

= 75%, P[X>10] < 0.06

Theory (benign conditions) Practice

Typical OC192 router linecard buffers over 2,000,000 packets

Can we make the traffic arriving at the routersPoisson “enough” to get most of the benefit?

May 5, 2005 23

Ideal Solution

If packets are spaced out perfectly; and The starting times of flows are chosen

randomly; We only need a small buffer for contention

resolution.

May 5, 2005 24

Pacing We need to break bursts

Modify TCP: Instead of sending packets when your receive ACKS send packets with a fixed rate of CWND/RTT.

Rely on network properties:• Access links throttle the flows to low rate• Core:Acess > 1000:1• TCP’s window size is limited today.

If these properties make the flow look like poisson with only 5-10 packets of buffering we can get 70-80% throughput.

May 5, 2005 25

What we know so farabout very small buffers

ArbitraryInjectionProcess

If Poisson Processwith load < 1

CompleteCentralized

Control

Any rate > 0need unbounded

buffers

Theory Experiment

Need buffersize of approx:

O(logD + logW)i.e. 20-30 pkts

D=#of hopsW=window size

TCP Pacing:Results as goodor better than forPoisson

Constant fractionthroughput withconstant buffers

[Leighton 1999]

May 5, 2005 26

CWND: Reno vs. Paced TCP

May 5, 2005 27

TCP Reno: Throughput vs. Buffer Size

May 5, 2005 28

Paced TCP: Throughput vs. Buffer Size

May 5, 2005 29

Early resultsCongested core router with 10 packet buffers.Average offered load = 80%RTT = 100ms; each flow limited to 2.5Mb/s

router

source

server

source

10Gb/s

>10Gb/s >10Gb/s

May 5, 2005 30

Slow access links, lots of flows, 10 packet buffers

router

source

server

source

10Gb/s

5Mb/s 5Mb/s

Congested core router with 10 packet buffers.RTT = 100ms; each flow limited to 2.5Mb/s

May 5, 2005 31

Conclusion

We can reduce 1,000,000 packet buffers to 10,000 today.

We can probably reduce to 10-20 packet buffers: With many small flows, no change needed With some large flows, need pacing in the

access routers or at the edge devices. Need more experiments.

May 5, 2005 32

Experiments

Performance measurement with Small (thousands of packets); and Tiny (tens of packets) buffers

Metrics: Link utilization (goodput/throughput) Drops Buffer occupancy Etc.

Data Gathered for minutes to days High load (50-70% utilization) is better

Thank you!Thank you!

Questions?Questions?