A Rant on Queues
Van Jacobson
July 26, 2006
MIT Lincoln LabsLexington, MA
vj-ll-jul06
• Unlike the phone system, the Internet supports communication over paths with diverse, time varying, bandwidth.
• This means we often have to connect a fire hose to a soda straw.
2
vj-ll-jul06
• Unlike the phone system, the Internet supports communication over paths with diverse, time varying, bandwidth.
• This means we often have to connect a fire hose to a soda straw.
• This kind of plumbing needs an adapter. The adapter is called a queue.
2
vj-ll-jul06
How a real queue works
3
vj-ll-jul06
Sender injects a window’s worth of packets
Sender Receiver
4
vj-ll-jul06
Packets reach high to low bandwidth transition
Sender Receiver
5
vj-ll-jul06
First ack returns and releases next data packet
Sender Receiver
6
vj-ll-jul06
Steady-state reached
Sender Receiver
7
vj-ll-jul06
• The amount of data that has to be in transit to run at 100% utilization is the bottleneck bandwidth times the sender-receiver-sender round-trip delay (this is called the bandwidth*delay product).
• The bottleneck has to have this much buffer to handle the start up transient.
• What happens if it doesn’t?
8
vj-ll-jul06
•••••••••••••••••••••••••••••••••••••••
••• ••••• ••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••
••••••••••••••••••••••••••••••••••••••••••••••••••••• •••••••••••
••••••••••••••••••• ••••
••••••••••••••••••••••••••••
••••••••••••••••••••••••••••••••••
••••••••••••• ••••••••••••••••••••••••••••••••••••••••••••••••••••••••••
••••••••••••••••••••••••••••••••••••••••••
•••••••••••••••••••••••••••••• •••••••
••••••••••••••••••••• •••••••
••••••••••••••••••••••••••••••••••
•••••••••••••••••••••••••••••••••••
•• ••••••••••••••••••••••••••••••••••••••••••••••••••• ••••••••••
•••••••••••••••••••••••••••••••• •••••
•••••••••••••••••••••••••• •••••
•••••••••••••••••••••••••••••••••
••••••••••••••••••••••••••••••••••••••••
•••••••••••••••••••••••••••••••••••••• ••••••••••••••
••••••••••••••••••••••••••••••••••••••
••••••••••••••••••••••••••••••••••••••••••••
Satnet Test -- Dec 11, 1988
Time (sec)
Seq
uenc
e N
umbe
r (K
B)
0 200 400 600 800 1000 1200 1400
020
4060
8010
012
014
016
0
(averaged 100 Bytes/sec on a 8 KByte/sec link)
9
vj-ll-jul06
•• ••••••••••••
••
•
••
••••••••••••
••••••••••• •••
•••••••••••
•
•••••
•• •••• •••
•••••••••
•••••••••• •••
••••• •••
••••••••••
•
Satnet Test -- Dec 11, 1988 (Detail)
Time (sec)
Seq
uenc
e N
umbe
r (K
B)
100 120 140 160 180 200
1015
2025
3035
40
10
vj-ll-jul06
How does the queue behave vs. time?
Sender Receiver
11
vj-ll-jul06
Queue behavior at the fast-to-slow transition
Time
Que
ue le
ngth
12
vj-ll-jul06
Queue behavior at the fast-to-slow transition
Time
Que
ue le
ngth
20 packet window
30 packet window
12
vj-ll-jul06
Queue behavior with ack-per-window receiver
Time
Que
ue le
ngth
13
vj-ll-jul06
Three minor (and completely standard) variations in protocol implementation give
three wildly different average queue lengths.
I.e., the average queue lengthcontains no informationabout demand or load.
14
vj-ll-jul06
A mathematical digression ...
15
vj-ll-jul06
∫[A(t) − D(t)]
• A queue is the integral of the difference between an arrival process & a departure process.
• For packet network queues, D is usually deterministic and A is some sort of random mixture process.
16
vj-ll-jul06
The Poisson arrival process is beloved by academics
the world over
• It describes a low density, IID, uniform random collection of stuff.
• ‘Uniform random’ means the interarrival time
is distributed as e-λt.17
vj-ll-jul06
0 20 40 60 80 100
010
2030
4050
Demand (% of output capacity)
Ave
rage
Que
ue L
engt
hFor a Poisson process, queue length is a function of demand
18
vj-ll-jul06
0 5 10 15 20 25
05
1015
2025
In fact, inverse queue length is
a linear function of 1/λ
19
vj-ll-jul06
For years people haveused queue length as
a proxy for loadin network controls
• In theory this shouldn’t work at all
• In practice it sometimes sort of works because the internet protocols are robust in the face of foolishness.
20
vj-ll-jul06
Theory failures
• There can be lots of Poisson traffic in the network but not at a bottleneck.
• Poisson requires independent, random uniform traffic but at a bottleneck real traffic is highly correlated and not at all uniform.
• The correlations are intrinsic since reliability requires a sender-receiver-sender loop and traffic in a loop is never poisson.
21
vj-ll-jul06
Poisson model isn’t even representative
• Queue length is the integral of an arrival rate (with deterministic departures).
• Queue length for a Poisson process is proportional to arrival rate.
• Implies family where ∫F(t) ∝ F(t)
• Exponential (Poisson process) is the only member of this family.
22
vj-ll-jul06
• Poisson models fail because they’re memoryless but congested router queues are all about memory.
• A much better mathematical model for real queues is a random process with memory. I.e., a random walk or Brownian motion.
• Its behavior is the polar opposite of poisson (see Feller, vol.1, chap.3).
23
vj-ll-jul06
Two simulation runs of the simplest random walk
(Bernoulli trials)
0 2000 4000 6000 8000 10000
050
100
150
200
Round
Sum
0 2000 4000 6000 8000 10000
!50
050
Round
Sum
24
vj-ll-jul06
Probability density distribution of the simplest random walk
!100 !50 0 50 100
1.0
1.5
2.0
2.5
3.0
Value after 100 iterations
Pro
babi
lity
(%)
25
vj-ll-jul06
And now back to packets ...
26
vj-ll-jul06
CS profs try to instill intuition based on this picture of traffic:
Measured traffic usually looks more like:
Where does all the structure come from?27
vj-ll-jul06
• Part of the structure comes from the stability of high rate, bursty arrivals on the upstream side of the queue -- one conversation is unlikely to insert a packet into another’s burst.
• This tendency to preserve bursts is then driven by some of the non-linear dynamics of the net ...
28
vj-ll-jul06
The Internetat 50,000 feet
the Internet
Sender Receiver
packet
ack
29
The ends are just mirrors, all the dynamics result from the (usually non-linear) behavior of the net.
vj-ll-jul06
How shared links make bursts
• Picture two conversations sharing a congested gateway as two separate train tracks with one common section.
• When a blue train waits for red trains to go through the shared section, the blue trains behind it catch up (get more clumped)
• If the merge rules are efficient (service each color to exhaustion), the system clumps exponentially fast.
30
vj-ll-jul06
R1
R2
G1 G2
S1
S2
• Clumping creates a ‘horizon problem’ that tends to bias traffic managers.
• For satellite systems, the traffic time structure can interact with superframe epoch structure in unfortunate ways (e.g., ACTS hopping beams).
• Increasing delay or epoch length makes things worse (windows increase to compensate).
31
vj-ll-jul06
Suggestions
• Queue length is meaningless (but long term min can be useful).
• Try have at least a bandwidth*delay of buffer.
• Don’t let it stay full.
• Try for a ‘flow-thru’ architecture to minimize packet time-structure disturbance
32
vj-ll-jul06
Suggestions (cont.)
• Never introduce additional delay (apps will just try to fill it with packets):
‣ Let apps do their own FEC; avoid link layer Reed-Solomon and ARQ.
‣ Use smooth, simple downlink schedulers
‣ Use predictive and anticipatory uplink schedulers
33