end-to-end multicast congestion control and avoidance jiang li advisor: shivkumar kalyanaraman
TRANSCRIPT
End-to-End Multicast Congestion Control and Avoidance
Jiang Li
Advisor: Shivkumar Kalyanaraman
Outline
• Overview of multicast and multicast congestion management
• Solution series– LE-SBCC – ORMCC– GMCC– MCA+
• Simulations & Experiments
• Future work
Multicast
• Efficient One-to-Many Data Distribution– Unicast: one copy for EACH receiver– Multicast: one copy for ALL receivers
Unicast Multicast
Source
Receiver 1 Receiver 2 Receiver 1 Receiver 2
Source
Router Router
When / Where to Use Multicast?• Whenever / wherever efficient one-to-many delivery
of same data is needed• Applications:
– News/sports/stock/weather updates– Distance learning– Configuration, routing updates, service location– Teleconferencing (audio, video, shared whiteboard, text
editor)– Distributed interactive gaming or simulations– Email distribution lists– Content distribution; Software distribution– Web-cache updates – Database replication
Research on Multicast
• Routing
• Reliable transmission
• Congestion management
• Security
• Address allocation
• And more
Multicast Congestion Management
• Adapt sending rate to available bandwidth– Single-rate– Multi-rate
Receiver 1 Receiver 2
Source
Router
1Mbps 2Mbps
1Mbps 1Mbps2Mbps
Challenges
• Multiple paths– Complex congestion pattern– Heterogeneous bandwidth
• Dynamically changing
…
Bottlenecks
Source
Receivers
Router
`0.5Mbps
1Mbps
2Mbps
Potential Problems
• Drop-to-Zero– React to congestion more than necessary
0 Mbps
Sending rate
Congestion !Cut rate
Congestion !Cut rate
Congestion !Cut rate
Rate increase
Congestion !Cut rate
Congestion !Cut rate
4Mbps3Mbps
2Mbps1Mbps
2Mbps
1Mbps 0 Mbps
Source
Router
btnk1 btnk3btnk2
Potential Problems (cont’d)
• Feedback implosion
…
Source
Receivers
Router
Potential Problems (cont’d)
• Large number of states and large computation complexity– O(1) number of states wanted– O(1) computation time wanted
• Unfriendliness to other existent congestion control protocols (e.g. TCP)– Break others or be broken by others
End-to-end
Solution Categories
• End-to-end vs. third-party assisted– End: source, receiver– Third-party: router, proxy etc
Router
Proxy (uncontrollable by end users)
Source
Receiver
Receiver
(Third party)
(Third party)
(End)
(End)
(End)
Easy to deploy. Our choice.
Network
Congestion Control vs. Congestion Avoidance
• Congestion control– Reactive– Congestion is managed
when packets are lost
• Congestion avoidance– Proactive– Congestion is managed
when queue is being built up at bottlenecks– Low avg. queue length, high b/w utilization
Queue length
Buffer size
Time
Cong. Avoid. Cong. Ctrl
Packet drop begins
t0
A Series of Solutions
• LE-SBCC (single-rate congestion control)– Purely source-based, compatible with many
multicast transport protocols
• ORMCC (single-rate congestion control)– O(1) state and computation complexity– Scalable to large groups
• GMCC (multi-rate congestion control)– Adaptive to receiver heterogeneity
• MCA+ (single-rate congestion avoidance)– Responsive to incipient congestion
Outline
• Overview of multicast and multicast congestion management
• Solution series– LE-SBCC – ORMCC– GMCC– MCA+
• Simulations
• Future work
LE-SBCC
• Loss-Event Oriented Source-Based Multicast Congestion Control
• Motivations– Compatible with all multicast transport
protocols (e.g. RMTP, PGM)– Easiest deployment
• Trade-offs– For small to medium sessions– Single-rate
Characteristics
• Based on source– Source does most of the work, e.g.
• Filtering feedback packets• Adjusting sending rate
• Minimum receiver support– Single-bit feedback
• Can be piggybacked by ACK or NAK• Commonly available in transport protocols
Can be deployed by upgrading source only
Problem
• How to react to congestion?
Source
Router
Rcvr 1 Rcvr 2 Rcvr 3 Rcvr 4 Rcvr 5
Idea
• React to the most congested receiver
Source
Router
Rcvr 1 Rcvr 2 Rcvr 3 Rcvr 4 Rcvr 5
7050 30 60 40
Reacts to 70 feedback packets
• React to a number of feedback packets approx. equal to those from the most congested receiver.
Feedback Filter Cascade
RateadaptationLI2LE MaxLPRF ATFFeedback
packets
Source
Router
Rcvr 1 Rcvr 2 Rcvr 3 Rcvr 4 Rcvr 5
7050 30 60 40
RTT (Round Trip Time):biased towards the most
congested receiver
RTTEstimator
Filter 1: LI2LE
• Pass at most one feedback packet per RTT per receiver• Loss indication (LI): an original feedback packet• Loss event (LE): a packet passing the filter
RTT
Filtering receiver 1’sfeedback
LI LE
Filtering receiver 2’sfeedback RTT
LI LE
t0t1
< RTT
Filter 2: Max-LPRF
• Passes every LE with the probability:
i
i
i
X
Xp
maxSource
Router
Rcvr 1 Rcvr 2 Rcvr 3 Rcvr 4 Rcvr 5
X1=70
X2=50 X3=30 X4=60 X5=40
250
705
1
1
i
iX
Xp
Approx. 70 LEs
Filter 3: ATF
• Enforce at most one rate cut per RTT.
RTT Time
Loss events
Sendingrate
Ignored
Rate cut
t0 t0+RTT
Accepted for rate cut
Rate Adaptation
• AIMD (additive increase/multiplicative decrease)
• Other (e.g. TFRC, binomial)
RTT RTT RTT RTT RTT RTT
Time
Sending rate
s/RTT
s: packet size
V
V/2
Outline
• Overview of multicast and multicast congestion management
• Solution series– LE-SBCC – ORMCC– GMCC– MCA+
• Simulations & Experiments
• Future work
ORMCC
• LE-SBCC– O(N) state at the source (N: # receivers)– Subject to feedback implosion
• ORMCC– O(1) state and computation complexity– Statistics-based feedback suppression– Proprietary support required from receivers
Source Functions
• Dynamically maintains a congestion representative (CR) (the slowest receiver)– Using TRAC (Throughput Rate At Congestio
n)• Receiving rate during congestion epochs
– Slowest receiver has the lowest average TRAC
• Only accepts CR’s input for rate adaptation (AIMD)
• Detect & recover from loss of CR– Omitted here
Receiver Functions
• Measures TRAC– Only during congestion epochs– Average over several packets or a short perio
d to avoid oscillation
• Sends feedback with TRAC– Suppresses if its average TRAC < average C
R TRAC – σ• σ : std. deviation of CR TRAC• Average CR TRAC andσare multicast by the sourc
e
Outline
• Overview of multicast and multicast congestion management
• Solution series– LE-SBCC – ORMCC– GMCC– MCA+
• Simulations & Experiments
• Future work
GMCC (Generalized Multicast Congestion Control)
• LE-SBCC & ORMCC– Single-rate
• GMCC: A multi-rate scheme– Various receivers receive data at
different rates. – Can run in single-rate mode– Leverage ORMCC and greatly simplify the
scheme
Receiver 1 Receiver 2
Source
Router
1Mbps 2Mbps
1Mbps 1Mbps2Mbps
Previous Multi-Rate Schemes• Receiver-based schemes
– The source sends data in each layer without regarding to network situation
– Receivers increase/decrease their receiving rates by constantly join/leave layers
• Coarse control, heavy router burden
Join interval
Leave
Leave intervalReceiverSource
Data
Layer 1Layer 2Layer 3Layer 4
~
Join
1 layer = 1 multicast group
(Very short) (Very short)
Previous Multi-Rate Schemes (cont’d)
• SMCC– Source adaptation– Static layering (pre-defined maximum rate for
each layer)
ReceiverSource Join interval Leave interval
~
Data
Layer 1Layer 2Layer 3Layer 4
(Dynamic) (Dynamic)
Join Leave
single-ratesingle-ratesingle-ratesingle-rate
GMCC
• Independent single-rate congestion control in each layer
• Dynamic layering (no rate limit for each layer)
Join interval Leave intervalSource Receiver
Data
Layer 1Layer 2
Layer 3Layer 4
(Dynamic) (Dynamic)
Join Leave
~
Key Ideas
• “Unsatisfied” receivers join a new layer.
Receiver 1 Receiver 2
Source
Router
1Mbps 2Mbps
1Mbps1Mbps
1Mbps=2Mbps
Unsatisfied: much less congested than the most congested one (in a laye
r)
Key Ideas (cont’d)
• No receiver is allowed to be the most congested in more than one layer.
Receiver 1 Receiver 2
Source
Router
1Mbps 2Mbps
1Mbps1Mbps
0.5Mbps0.5Mbps
0.33Mbps0.33Mbps
0.33Mbps0.5Mbps0.33Mbps
Receiver Operations
• Join– A receiver joins a higher layer if it sees much
less congestion than CR in its top joined layer.
• CR: The most congested receiver
• Leave– A receiver leaves its top joined layer if it is the
most congested in more than one layers.
JoinLess congested
Most congested
Leave
Layer 1Layer 2
Layer 1
Layer 3Layer 2
Layer 1
Layer 3Layer 2
Layer 1Layer 2
Source Operations
• Control on and off of each layer– Only the top layer can be deactivated– A previously unnoticed problem
• Do single-rate multicast congestion control in each layer– Very similar to ORMCC
Layer 1
Layer 3Layer 2
Receivers
Outline
• Overview of multicast and multicast congestion management
• Solution series– LE-SBCC – ORMCC– GMCC– MCA+
• Simulations & Experiments
• Future work
MCA+
• Multicast congestion avoidance with feedback suppression
• MCA+ = ORMCC + incipient congestion detection– Using accumulation to detect incipient
congestion
MCA+: Accumulation• A flow’s accumulation is a time-shifted,
distributed sum of the queued bits in all nodes along its path.
• Can be measured end-to-end– Showed theoretically by colleague’s work
Accumulations:
Red flow: 5 packets
Blue flow: 4 packets
Yellow flow: 9 packets
Routers
Packets in queues
t1
t2 t3 t4
Data
MCA+: Accumulation Measurement
• Use control packets as pivot points• (λ: input rate, μ: output rate)tt
Time
PropagationDelay
CP0
CP0
CPi
CPi
N packets sent (λt)
M packets received (μt)Accumulation
(N – M packets)
Sender side
Receiver side
MCA+: Accumulation & Congestion
• Accumulation ≥ 2 packets → congestion– By receivers.– Possible one packet accumulation even when
underloaded– More noise, higher threshold– Accumulation ≤ 1 packet, synchronize (begin
another measurement period).
• Feedback and rate adaptation– Very similar to ORMCC
Outline
• Overview of multicast and multicast congestion management
• Solution series– LE-SBCC – ORMCC– GMCC– MCA+
• Simulations & Experiments
• Future work
Problems to Solve -- Review
• Drop-to-Zero
• TCP-friendliness
• State and computation complexity
• Feedback implosion
Simulations: Drop-to-Zero Avoidance Test
…Bottlenecks
Source
Receivers
Router
`
• Topology
Drop-to-Zero Avoidance Test (cont’d)
Multicast flow rateThroughput rates of two randomly chosen
unicast flows
• LE-SBCC, using ns-2 – Multicast rate ≈ Unicast rate
Simulations: Drop-to-Zero Avoidance Test (cont’d)
• ORMCC, 10000 receivers, using ROSS
Multicast ORMCC
Unicast ORMCC
Simulations: Drop-to-Zero Avoidance Test (cont’d)
• ORMCC vs PGMCC & TFMCC, 500 receivers, using ROSS
ORMCC
PGMCCTFMCC
Unicast PGMCC
Simulations: Drop-to-Zero Avoidance Test (cont’d)
• MCA+
Bottleneck Utilization Improvement of MCA+
• Average queue length
Bottleneck Utilization Improvement of MCA+
• Bandwidth utilization
Throughput Improvement of GMCC
Throughput Improvement of GMCC(cont’d)
• 6 groups of bottlenecks: 0.2 Mbps ~ 1.2Mbps
Outline
• Overview of multicast and multicast congestion management
• Solution series– LE-SBCC – ORMCC– GMCC– MCA+
• Simulations & Experiments
• Future work
Future Work
• Application of GMCC to multicast video streaming
• Application of GMCC to bulk data transmission
• Multi-rate application layer multicast congestion control
Thank you
&
Estimated # of Multicast Receivers on the Internet
ORMCCDetect/Recover from CR Loss
• Recover from CR absence– Measure response time (TR)
since hypothetical bottleneck queue build-up (tq)
• If no feedback from CR for TR since tq
– Claim CR absence– Requests feedback from all
receivers
Buffer size
Queuelength
Packet loss begins
Time
Time
Rate
TRAC
TR
tq
Feedback from
CR arrives
GMCCCompare Degree of Congestion
• Throughput Attenuation Factor (TAF)– A: Individual throughput attenuation factor
• 1 - (output/input) ( )• Severity of congestion
– B: Congestion occurrence rate• Number of packet loss epochs / Total sent pack
ets• Frequency of congestion
– TAF = A • B. Higher TAF, more congested
Input
Output
Input - output
Packets sent
Packets arrived 10
2
at congestion
Epoch 1 Epoch 2
Probabilistic Inter-layer Bandwidth Switching
• Hidden bandwidth problem
2Mbps
1Mbps10
Mbps
10Mbps
Receivers
Rcvr 1 Rcvr 2 Rcvr 3
Source
Probabilistic Inter-layer Bandwidth Switching (cont’d)
Layer 2traffic volume
Layer 1traffic volume
2Mbps
1Mbps10
Mbps
10Mbps
Receivers
Rcvr 1 Rcvr 2 Rcvr 3
Source
Test period
• No change to total traffic volume
• Difference in test period
Posttest
period
Simulations: TCP-Friendliness Test
• Topology
…
…
TCP
Multicast
Simulations: TCP-Friendliness Test (cont’d)
• LE-SBCC
Simulations: TCP-Friendliness Test (cont’d)
• ORMCC
Effect of Feedback Packet Loss• ORMCC, 5% loss
CR Switch Test
• ORMCC, every CR stops response after 20 seconds
Linux Experiment (ORMCC)
• Topology
Linux Experiment (ORMCC)
• Result
Effectiveness of Layering
Source1Mbps
GMCC receiver 1
GMCC receiver 2
Unicast receivers
5Mbps
Layering & Throughput Rate (1)
Throughput rate of receiver 2Receiver 2 joined layer 0
Receiver 2 stayed in layer 1Receiver 2 joined layer 1
Effectiveness of Layering (cont’d)
Source1Mbps
GMCC receiver 1
GMCC receiver 2
Unicast receivers
10Mbps
Layering & Throughput Rate (2)
Throughput rate of receiver 2
Receiver 2 joined layer 0
Receiver 2 stayed in layer 1Receiver 2 joined layer 1
GMCC’s Response to Dynamic Traffic
• Topology
GMCC’s Response to Dynamic Traffic (cont’d)
• Throughput dynamics
GMCC PIBS Test
• Topology
GMCC PIBS Test Result
Related Research
• End-to-end single-rate– DeLucia’s scheme, PGMCC, TFMCC, MDP-C
C
• End-to-end multi-rate– MLDA, HALM– RLM,RLC,PLM,FLID-DL,FGLM,STAIR etc
• Third-party assisted– Single-rate: e.g. Chiu’s work– Multi-rate: e.g. Kar’s work