YE PU & YAN SUN

Reliable IP Multicast:status and selected topics

A CSE620 Presentation

04/18/23 10:15 PM

Reliable Multicast

Overview

Introduction Reliable Multicast Protocols Case Studies Multicast congestion control Routing for Multicast The MBone and the Internet2 Summary

04/18/23 10:15 PM

Reliable Multicast

Introduction

Why Multicast?– In many emerging

applications, one sender will transmit to a group of receivers simultaneously

Why Reliable?– Audio/Video applications

do not require reliability– Many other exciting

applications do, e.g. remote WB, collaborative VR, data dissemination

Unicasting

Multicasting

04/18/23 10:15 PM

Reliable Multicast

Reliable Multicast: Basic Questions• What is the "right" definition of reliable multicast?

• Is there a baseline(e.g., reliable delivery of all data)? • should ordering/causality be part of the networking semantics of reliable

multicast?

• where to draw the line between network- and application-level functionality? • Design Approaches

• How important is scalability (large number of participants)? • Are there fundamental differences from one setting to another (1-many vs

many-many) that require different approaches? • Are separate designs, each optimized for a different scenario, the way to go? • Can one protocol (or protocol framework) fit all requirements? Will n protocols

(or framworks) fit k (n<k) scenarios? • Framework

• Is there a value (and if so, what is it) of developing a common framework (a la RTP) in which various reliable multicast protocols can be built

• What should that framework look like?

• In terms of IETF, is there any part (which one) that should be standardized? -- ACM SIGCOMM Multicast Workshop, Stanford, August 27, 1996

04/18/23 10:15 PM

Reliable Multicast

Reliability Mechanism: Who’s Responsible?

Sender Initiated • Sender is responsible for packet loss

detection• Based on positive acknowledgements

(ACKs) • ACK implosion at large scale multicast, poor

scalability

Receiver Initiated• Receiver is responsible for packet loss

detection• Based on negative acknowledgements

(NACKs)• Alleviates ACK implosion, better

performance• Potentially NACK implosion

ACK Implosion

NACK Trigger

NACK Implosion

04/18/23 10:15 PM

Reliable Multicast

Loss Recovery: What did ja say?Loss Recovery: Detection and Retransmission of lost packets

Global Recovery• Repair are multicasted to the entire group• Efficient where loss is often concentrated at the backbone gateway

Local Recovery• Try to recover from packet loss without going all the way to the

source• Response multicast within a scope just large enough to reach each

affected receiver

Forward Error Control (FEC)• Retransmit error-correcting codes instead of original packet data• Simultaneously repair packet losses with a single packet

04/18/23 10:15 PM

Reliable Multicast

Feedback Control

Feedback Control: Mechanism that restricts the amount of feedback generated by multicast group

Structure BasedRely on a designated receiver (DR) to process and filter feedback traffic

Timer BasedDelay retransmission request for a random time interval, uniformly distributed between the current time and one-way trip time to the source

04/18/23 10:15 PM

Reliable Multicast

Multicast Protocols by 1997

Protocol DataPropagation

ReliabilityMechanism

RepairRequest

FeedbackControl

Retrans-mission

FlowControl

Locus ofControl

Ordering GroupManagement

Target Application

RBP B-cast ACK/NACK U-cast - U-cast - Central Total Explicit General SolutionMTPMTP-2

M-cast NACK U-cast - M-cast Rate Central Total Explicit General Solution

RMP M-cast ACK/NACK M-cast - U-cast Window Central Total Explicit General SolutionXTP U-cast/M-cast ACK/NACK U-cast - U-cast/M-cast Rate Central Sequence

NumberExplicit General Solution

URGC B-cast NACK U-cast - U-cast - Central Total Explicit General SolutionRTP M-cast Rate-adjust by

Feedback- Probabilistic

Polling- Rate Distributed - - Interactive Multimedia

SRM M-cast NACK M-cast Timer-based M-cast - Distributed - Explicit Interactive MultimediaLBRM M-cast ACK/NACK U-cast Structure-based U-cast/M-cast - Central - Explicit Interactive MultimediaRAMP M-cast NACK U-cast - U-cast/M-cast Rate Distributed Sequence

NumberExplicit Interactive Multimedia

(Optical GigabitNetwork)

TRM M-cast NACK M-cast Timer-based M-cast Window Distributed SequenceNumber

Implicit Interactive Multimedia

Muse M-cast NACK U-cast - U-cast - Distributed - Implicit News ArticlePropagation

MDP M-cast NACK M-cast Timer-based M-cast Rate Distributed SequenceNumber

Implicit Data (File) Distribution

AFDP M-cast NACK U-cast - M-cast Rate Central SequenceNumber

Explicit Data (File) Distribution

TMTP M-cast ACK/NACK U-cast Structure-based M-cast Window Distributed - Implicit Data DisseminationRMTP M-cast ACK U-cast Structure-based U-cast/M-cast Window Distributed - Implicit Data DisseminationMFTP U/M/B-cast ACK/NACK U-cast (ACK)

M-cast (NACK)Timer-based M-cast Window Distributed - Explicit Data Dissemination

STORM M-cast NACK U-cast Structure-based U-cast Window Distributed - Implicit Data Dissemination

04/18/23 10:15 PM

Reliable Multicast

Case Study: SRM

SRM: Scalable Reliable Multicast

• Originally designed for wb• Currently operational over the MBone• Receiver-reliable, NACK-based• Any receiver can multicast NACK or repair packet

04/18/23 10:15 PM

Reliable Multicast

SRM Loss Recovery Principle

Data-driven Recovery(sequence gap detection)

Control-driven Recovery(session message sequence)

• Source assign unique sequence number• NACK generated when missing data detected

04/18/23 10:15 PM

Reliable Multicast

SRM: Source Path Message

• Each member multicasts periodic session messages that report the sequence number state for active sources

• Receivers detect the loss of the last packet in a burst

• Members also use session messages to determine the current participants of session

• Average session message bandwidth: 5% of data bandwidth

04/18/23 10:15 PM

Reliable Multicast

SRM NACK Suppression

• NACK is multicast to the entire group• Receiver in need of that data can suppress its own NACK• Simultaneous detection of packet loss: random delay and

receiver with smallest delay wins

04/18/23 10:15 PM

Reliable Multicast

SRM: Loss Recovery Algorithm

Loss Detection1. set the backoff parameter =1

2. upon miss data D from host S, choose a random delay t on 2[C1d(S), (C1+C2)d(S)]

3. schedule a request packet, REQD, for transmission in seconds

4. if we receive REQD from some other host before seconds, then set

= +1 and restart the request timer

5. otherwise, if data, D, or the repair reply, REPD, is received before seconds, cancel

REQD

6. otherwise, send REQD after seconds

Retransmission1. upon receipt of REQD from host A, if D is locally available, choose a random delay

on [C1d(A),(C1+C2)d(A)]

2. schedule the repair packet REPD for transmission in seconds

3. if REPD is received before seconds, then cancel the repair timer

4. otherwise, send REPD after seconds

04/18/23 10:15 PM

Reliable Multicast

Case Study: RMTP

RMTP: Reliable Multicast Transport Protocol

• Designed for file dissemination(single-sender)• Deployed in AT&T’s billing network• Based on a hierarchical structure• A special Designated Receiver (DR) is responsible

for sending ACKs to sender

04/18/23 10:15 PM

Reliable Multicast

RMTP: Network Topology

• Receivers grouped into local region

• Source multicasts packets to receivers

• Receivers unicast periodical ACK to its AP/DR

• DR provides local repair if data is • available• DR unicasts its own ACK to

parent to consolidation of traffic to the next DR in hierarchy

• Source determines retransmission based on status send by DR

04/18/23 10:15 PM

Reliable Multicast

Send Sequence Space

send_nextsw in_lb

send w indow

packet sent but not yet acknow ledged

avail_w in

RMTP: ACK Processing & Retransmission

A receiver’s receive window

A sender’s send window

04/18/23 10:15 PM

Reliable Multicast

RMTP: Formation of Local Region

• RMTP assumes there is some information about the approximate location of receivers

• Some receivers and servers are chosen as DR• Each DR periodically sends a special packet

SEND_ACK_TOME in which TTL field is set to a pre-determined value(say 64)

• Each receiver chooses the DR whose SEND_ACK_TOME has the largest TTL value

04/18/23 10:15 PM

Reliable Multicast

Case Study: PGM

PGM: Pragmatic General Multicast

• Router supported to provide scaling• Provide no notion of membership• NACK based, with suppression

04/18/23 10:15 PM

Reliable Multicast

PGM: Data Packet Types

ODATA: original content dataNACK: selective negative acknowledgementNCF: NACK confirmationRDATA: retransmission data(repair)SPM: source path message

TSI: Each PGM packet contains a Transport Session Identifier (TSI) to identify the session and source of data

04/18/23 10:15 PM

Reliable Multicast

PGM: NACK/NCF Dialogue

• NACK + random delay is unicast from router upstream towards source

• PGM-aware router keeps forwarding NACKs until it sees a NCF or RDATA

• Only one NACK is forwarded for every packet loss

• Source multicast NCFs to the whole group to provide NACK reliability

04/18/23 10:15 PM

Reliable Multicast

PGM: Source Path Message

• SPMs are multicast downstream interleaved with ODATA• PGM-aware routers use SPM to determine unicast path

forwarding NACKs• Receivers use SPM to determine the last PGM aware router

to forward NACK

04/18/23 10:15 PM

Reliable Multicast

PGM: Retransmission

Sender• Retransmit immediately after getting a NACK

Router • Maintain retransmission states for every interface that

received NACK• Only forward retransmission on one interface per NACK

04/18/23 10:15 PM

Reliable Multicast

PGM-aware Router Features

• Routers intercept SPMs and use them to establish source path state for the corresponding source and group

• Routers forward only the first copy of any NACK they receive to the upstream PGM-aware router to constrain NACK forwarding

• Routers discard exact duplicates of any NACK for which they already have repair state

• Routers use NACKs to maintain repair state consisting a list of interfaces upon which a given NACK was received, and return the RDATA only on these interface

• Routers can also optionally redirect NACKs to a designated local retransmitter (DLR) rather than the source

04/18/23 10:15 PM

Reliable Multicast

Congestion Control

Why Congestion Control?• Needs to use available bandwidth fairly among multiple best-effort

flows over a shared link

TCP Congestion Control• Multiplicative decrease at the indication of congestion• Linear increase when there is no congestion• Encourage fair sharing of bandwidth• No safeguard against aggressive flows (end to end feedback

controlled)

Multicast without CC• Non TCP compatible flows can lock out competing TCP flows• Simultaneous congestion collapses• Need end to end feedback based TCP compatible congestion control

mechanism

04/18/23 10:15 PM

Reliable Multicast

Control Metrics

Fairness - How it shares bandwidth with other connections, and how it discriminates against connections of different lengths. This is the closest thing to the "performance" of a connection

Safety - How wide of a range of operating conditions can the algorithm support without causing the network to go in to an unstable operating range

Responsiveness - How fast an algorithm adapts to changes in the network load

Variability (or accuracy) - How consistent is the performance of thealgorithm in the face of a given environment? i.e. what is the variance in throughputs?

Scalability - How do these metrics scale in the face of large scale groups?

04/18/23 10:15 PM

Reliable Multicast

Control Approaches

Window-based: “Slow start” TCP-style sliding window algorithm

Rate-adaptive: Adjust transmission rate upon receipt of NACKs

Forward Error Correction (FEC): Rarely used due to encoding/decoding overhead

04/18/23 10:15 PM

Reliable Multicast

MTCP: Hierarchical Congestion Control

Hierarchical Congestion Reports• Internal tree nodes sender's agent

(SA) • receivers send feedback to their

SAs• SAs send a summary of the

congestion level of their children to their parents

M R 1

S o urce

G ro upM em ber

G ro upM em ber

G ro upM em ber

G ro upM em ber

G ro upM em ber

M R 9 M R 10

M R 3M R 2 M R 4

M R 5

M R 8M R 7M R 6

SA

SA

ACKs

ACKs andsum m ary

ACKs andsum m ary

04/18/23 10:15 PM

Reliable Multicast

MTCP: Hierarchical CC (cnt’d)

Window Based Control• Send controls its rate based on its summary

Congestion Window Adjustment (when CWND goes down) • RTD timeout • Fast retransmission (in conjunction with selective acknowledgment) • Three NACKs for the same packet reduces the window (note that not

every loss causes CWND to go down by half)• Based on TCP Vegas scheme (I.e., long RTT causes it to go down)

04/18/23 10:15 PM

Reliable Multicast

Forward-error Correction Coding (FEC)

• "Simultaneous repair" utilize(n,k) block codes

• Packet stream is grouped into platoons of n packets each

04/18/23 10:15 PM

Reliable Multicast

FEC/ARQ

• On detected loss the receiver NACKs the platoon rather than the packet

• If each receiver indicates the number m of packets loss from that platoon, then the responder can merely send m of k parity packets.

SenderR eceiver R eceiver

04/18/23 10:15 PM

Reliable Multicast

Proactive FEC/ARQ

Proactive: Send some repairs before lossProactive factor:

• Sender sends round(k) packets

• Recevers NACKs to get add’l repairs

SenderR eceiver R eceiver

04/18/23 10:15 PM

Reliable Multicast

Multicast Routing

• Requires a significant amount of state and complexity in routers (requires at least per-group state information and often even per-source information) => Very slow deployment and use by Internet standards

• Dense Mode: Sender broadcasts traffic and triggers prune messages (DVMRP, PIM-DM)

• Sparse Mode: Group members explicitly sends join messages (MOSPF, CBT, PIM-SM)

Advantage Less routing state to keep (only routers on the multicast path keep) Explicit join: multicast traffic only flows across links leading to identified

receivers

Disadvantage Single-point-of-failure at RP Hot spot of multicast traffic at RP and non-optimal path on multicast tree

04/18/23 10:15 PM

Reliable Multicast

Multicast Routing in Early MBone

MBone on non-multicast capable Internet

1. MR3 and MR4, running the Multicast Router Daemon (mrouted), support IGMP. Mrouted encapsulates multicast datagrams in unicast datagrams to send, and decapsulates multicast datagrams from unicast datagrams it receives2. R1 and R2 are non-multicast enabled routers. They forward unicast encapsulated multicast packets just like any other unicast datagram

M ulticas tA p p licatio n(send er o rreceiver)

M ulticas tA p p licatio n(send er o rreceiver)

M R 3M R 4

R 1 R 2

04/18/23 10:15 PM

Reliable Multicast

DVMRP

• First protocol developed to support multicast routing

• Tree is constructed on demand using a “broadcast and prune”

• Reverse Path Forwarding (RPF) ensures no loops in the tree and only shortest paths included

• RPF uses unicast routing table• Does not scale to support

multicast groups that are sparsely distributed over a large network

1. the message reaches router 12. the message reaches routers 2,3, and 43. routers 3 and 4 exchange messages. Each one just drops the message, because

it didn’t arrive over the interface that gives the shortest path back to the source4. the message reaches router 7. Router 7 realizes it is a leaf router and there are

no group members on its subnet, so it sends a prune message back to router 6, the upstream router. Router 6, in turn, sends a prune message to router 4. Router 3 also sends a prune message to router 1

M R 6

M R 1

S o urcelo cal subnet

H o p s :1234

G ro upM em b er

G ro upM em b er

G ro upM em b er

G ro upM em b er

M R 3

M R 2M R 4 M R 5

M R 8

M R 7

04/18/23 10:15 PM

Reliable Multicast

MOSPF

• Intended for use within a single routing domain

• Dependent on the use of OSPF• Tree is only calculated when a

router receives the first data-gram in a stream

• All routers calculate exactly the same tree

• Does not scale well due to periodic flooding of group membership reports

1. MR 1 computes tree - knows members of group via IGMP and hence knows path to MR 4 is via MR 2, path to MR 8 is via MR 5, etc.

2. MR 2 computes tree - determines path to MR 4 is direct, path to MR 8 is via MR 5 and MR 3 computes tree - determines path to MR 9 is direct

3. MR 5 computes tree - determines path to MR 8 is direct

Note that the multicast transmission triggers this process (i.e. data driven process) and each router, when it receives a message, calculates exactly the same distribution tree as its predecessors and uses it to forward the message.

M R 1

M R 2

M R 4 M R 5

M R 3 M R9

M R 8M R 7

M R 6

G ro upM em ber

G ro upM em ber

G ro upM em ber

G ro upM em ber

04/18/23 10:15 PM

Reliable Multicast

Core Based Tree (CBT)

• a single tree that is shared by all members of the group, Multicast traffic for the entire group is sent and received over the same tree, regardless of the source

• significant savings in terms of the amount of multicast state information that is stored in individual routers

• concentration of traffic around the core

• load balancing might be achieved by using more than one core

04/18/23 10:15 PM

Reliable Multicast

PIM-SM

• Initial group-shared tree construction similar to CBT

• Supports both group-shared tree and shortest-path tree

• Relies on unicast routing tables to adapt to network topology changes

• Independent of the particular unicast routing protocol

1. The sender at Source 2 registers at the Rendezvous Point Multicast Router RPt

2. A receiver joins at Rpt; there is now a bigger shared tree3. The receiver is receiving lots of data from Source 2. The

receiver sends an explicit join to Source 2 to construct a shortest path route

M R

M RM R

M R

M R

M R

M R

M R

04/18/23 10:15 PM

Reliable Multicast

Interdomain Multicast RoutingNear-term Solution - PIM-SM/MBGP/MSDP:

• Multicast Border Gateway Protocol (MBGP): multicast route aggregation and abstraction as well as hop-by-hop policy routing is provided in unicast using the Border Gateway Protocol (BGP)

• Multicast Source Discovery Protocol (MSDP): works by having representatives in each domain announce to other domains the existence of active sources. MSDP is run in the same router as a domain's RP (or one of the RPs)

Long-term Solution - BGMP/MAAA:• Border Gateway Multicast Protocol (BGMP): first proposed as a long-term

solution to Internet-wide, inter-domain multicast. • Multicast Address Allocation Architecture (MAAA): consists of Multicast

Address-Set Claim (MASC) protocol (domain level), Address Allocation Protocol (AAP) (within a domain), and Multicast Address Dynamic Client Allocation Protocol (MADCAP) (for requesting addresses from a multicast Address Allocation Server (MAAS))

Alternative Solution - Root Addressed Multicast Architecture (RAMA)

04/18/23 10:15 PM

Reliable Multicast

The MBone

• A virtual network layered on top of the physical Internet to support routing of IP multicast packets

• Initially a test bed for multicast• Extensively exploits tunnels• Routing mainly with DVMRP

“MBONE is truly the start of mass-communication that may supplant television. Used well, it could become an important component of mass communication.” -- John December

04/18/23 10:15 PM

Reliable Multicast

The MBone

04/18/23 10:15 PM

Reliable Multicast

The Internet2

Internet2 is a collaboration among more than 100 U.S. universities to develop networking and advanced applications for learning and research. The design and implementation of a deployment strategy to provide a consistent and ubiquitous multicast service within the Internet2 community.

Internet2 Multicast-Peering Sites: Abilene, vBNS, NREN, DREN, Esnet, CANARIE, TEN-155/34 (DANTE), NORDUnet, SurfNet, APAN

Abilene is an advanced backbone network that connects regional network aggregation points, called gigaPoPs, to support the work of Internet2 universities as they develop advanced Internet applications.

vBNS maintains a native IP multicast service via a PIM sparse-dense-mode configuration among all vBNS Cisco routers. MBGP routing is used internally in combination with an MBGP default route representing MBone sources. vBNS belongs to MCI Worldcom

04/18/23 10:15 PM

Reliable Multicast

The Internet2 (cnt’d)

For Internet2, the plan has always been to try and do multicast “the right way” in so much as is possible given the currently available set of protocols. As a result, the multicast deployment plan is following guidelines set forth by the Internet2 Multicast Working Group.

Guidelines• all multicast deployed in Internet2 to be native and

sparse mode• No tunnels are allowed• All routers must support inter-domain multicast routing

using MBGP/MSDP.

04/18/23 10:15 PM

Reliable Multicast

Multicast on Abilene Network

04/18/23 10:15 PM

Reliable Multicast

Multicast on vBNS

04/18/23 10:15 PM

Reliable Multicast

Summary

• IP Multicast is emerging as an utterly important topic in the future Internet

• Achieving reliability: ACKs vs NACKs, Local Recovery, FEC, …

• Reliable multicast protocols: SRM, RMPT, and PGM• Multicast congestion control• Routing in multicast: DVMRP, MOSPF, CBT, PIM-SM• Interdomain Multicast and multicast deployment on

the MBone and the Internet2

04/18/23 10:15 PM

Reliable Multicast

ReferencesAlmeroth, K. C., The Evolution of Multicast: From the MBone to Inter-Domain Multicast to Internet2, Deployment, IPMI White Paper (www.stardust.com), 1999

Ballardie, A., RFC-2201: Core Based Trees (CBT) Multicast Routing Architecture, September 1997 Costello, A. M. and McCanne S., Search party: using randomcast for reliable multicast with local recovery, University of California at Berkeley Techanical Report UCB//CSD-98-1011, 1998

Estrin, D., Farinacci D., A. Helmy, Thaler D., Deering S., Handley M., Jacobson V., Liu C., Sharma P., Wei L., RFC-2362: protocol independent multicast-sparse mode (PIM-SM): protocol specification

Floyd, S., Jacobson V., Liu C., McCanne S., and Zhang L., A reliable multicast framework for light-weight sessions and application level framing, IEEE/ACM Transactions on Networking, Vol. 5, No. 6, 1997

IPMI, Reliable IP multicast - PGM overview, IPMI White Paper (www.stardust.com), 1998

http://www.tascnets.com/mist/doc/mcpCompare.html

http://netweb.usc.edu/multicast/

http://www.stardust.com/

http://www.starburstcom.com/

Katia Obraczka, Multicast transport mechanisms: a survey and taxonomy, IEEE Communications Magazine, January 1998

04/18/23 10:15 PM

Reliable Multicast

ReferencesMankin, A., Romanow A., Bradner S., and Paxson V., RFC-2357: IETF criteria for evaluating reliable multicast transport and application protocols, June 1998

McCanne, S., Scalable Multimedia Communication Using IP Multicast and Lightweight Sessions, IEEE Internet Computing, Vol. 3, No. 2, 1999

Moy, J., RFC-1584: multicast extensions to OSPF, March 1994

Paul, S., Sabnani K. K., Lin J. C., and Bhattacharyya S., Reliable Multicast Transport Protocol (RMTP), IEEE Journal on Selected Areas in Communications, Vol. 15 No. 3, 1997

Rekhter, Y., Li T., RFC-1771: a border gateway protocol 4 (BGP-4), March 1995

Waitzman, D. and Deering S., RFC1075: distance vector multicast routing protocol, November 1988