1 multicast forwarding and application state scalability in the internet tina wong dissertation...

1

Multicast Forwarding and Application State Scalability

in the Internet

Tina Wong

Dissertation SeminarComputer Science Division

University of California, BerkeleyOctober 16, 2000

2

Challenge

“… in the long run, the biggest issue facing multicast deployment is likely to be the scalability of multicast forwarding state as the number of multicast groups increases.”

--Thaler and Handley 2000

The memory required to store multicast forwarding entries at a router with 32 interfaces is 1024 TB for IPv6, assuming 50% address space utilization

--Radoslavov, Govindan and Estrin 1999

3

Outline

• Introduction, background, motivation• Multicast state scaling trends in Internet • Preference clustering protocol• Application-driven tunable reliability• Conclusions and future work

4

IP Multicast

• Efficient point-to-multipoint delivery mechanism

• Packets travel on common parts of the network only once

S

R R R

R

R

5

Multicast Routing

S

R R R

Broadcast

DVMRP• Per-source reverse shortest

path tree• Broadcast-and-prune• MBone

6

Multicast Routing

S

R R R

Prune



7

Multicast Routing

S

R R R

Forward Data



8

Multicast Routing

• PIM-Dense Mode / Sparse Mode– Unidirectional shared tree– Explicit joins– Core location a problem

• Core Based Trees (CBT)– Bi-directional shared tree– More optimal data paths– Few routing vendors support

9

Multicast Forwarding State

• Router maintains membership state to achieve forwarding

• State scales linearly with number of concurrent groups

• No natural aggregation

• Number of concurrent multicast groups limited by router memory

• Heartbeat messages to maintain state incur processing costs

oif0 oif1 oif2

iif0

s = 0.0.0.0/0G = 224.0.1.2/32iif0, oif1, oif2

10

Motivation

Lots of simultaneously active multicast groups on the Internet?

• Many small, group-based applications– Few participants form a single multicast group– E.g. internet video conferencing, games, events

notifications, etc

• Few large-scale applications– Lots of users form many multicast groups– E.g. Content delivery, stock quotes, DIS, etc

11

Related Work

• Multicast state reduction– Leaky and non-leaky state aggregation– Tunneling in backbone (MPLS, DCM)– Non-branching state elim (DTM, REUNITE)

• Application-level multicast– End Sytsem Multicast, YOID, Scattercast

12

Contributions

• Comprehensive analysis on multicast state– Understand scaling trends in the Internet– Predict future growth– Estimate potentials for reduction– Apply to network provisioning, protocol and

application design

• Mechanisms for network and end-host state scalability in large-scale applications– Interest-based content delivery– Application-driven loss recovery

13

Outline

• Introduction, background, motivation• Multicast state scaling trends in Internet• Preference clustering protocol• Application-driven tunable reliability• Conclusions and future work

14

Questions: Scaling Trends

Much research and engineering effort into making IP multicast widely deployed...

• How do multiplying peering agreements among parallel backbone networks affect multicast state scalability?

• How do rising subscriptions to individual applications increase multicast state?

• What are the state scaling properties when more and more applications use multicast?

15

Questions: State Concentration

An intuition: multicast state scalability is most critical at “core” routers…

• How concentrated is multicast state at “core” routers?

• How much benefit from tunneling?

“Core”

16

Questions: State Reduction

An intuition: delivery trees of sparse multicast groups tend to have large number of non-branching routers...

• How prominent are non-branching routers?

• Are these routers stateful?

S

R

R

R R

17

Basic Model

Local state• Fraction of concurrent

multicast groups

True local state• Local state with only

multicast forwarding

Independent of address space size and number of concurrent groups

5 concurrent groupsLocal state = 2/5True local state = 1/5

oif0 oif1 oif2

iif0

18

Methodology

• Simulations– Extends upon SGB package

• Parameters– Topology – Session density– Membership model

19

Topology

• 4 AS graphs from Nov97 to Jan00– Connectivity among Internet autonomous

systems– Study multicast state at inter-domain level– Over 3 year timespan

• Mbone graph from Feb99– Study multicast state at intra-domain level

• Generated graphs– TIERS– Transit-stub

20

Session Density

• Graphs have different number of nodes, from 1000 to 6474– Session density instead of absolute size– 0.1% to 0.9%, 1% to 9%, 10% to 90%– E.g., session with 0.1% density in AS-Jan00

with 6500 nodes involves 7 domains– E.g., session with 10% density in Mbone

with 4200 nodes involves 420 routers

21

Membership Taxonomy

Topological Correlationwithin one group

Subscription correlationacross multiple groups

NO

NO

YES

YES

1

random distrclusters

2

affinity/disaffinity

3

interest4

layeredinterest

5 6

22

Experiments

• For each experiment, fix topology, session density and membership model– (1) Pick a set of nodes with these parameters– (2) Build shortest path tree rooted at a random

node from this set– Repeat (1) & (2) 1000 times– Calculate local state and true local state on each

node in topology

• All combinations of parameters used, yielding 945 experiments and results!

24

Answers: Scaling Trends 1

• How do multiplying peering agreements among parallel backbone networks affect multicast state scalability?

– More state at a handful of core routers– Offset by reduced state in majority of

routers

25

Topological Properties

26

Hypothesis

• In a more connected network– Trees have larger fanouts and shorter

heights– Only a few highly peered routers involved in

most concurrent multicast trees

27

Hypothesis

• In a less connected network– Trees have smaller fanouts and taller

heights– Backbone routers share responsibility of

multicast forwarding -- “load balancing”?

28

Path Lengths

29

Node Degrees

AS-Nov97 MBone

30

Past and Future ScalingTrends

• Implication– If Internet continues to evolve as it has been,

multicast memory requirements at most of border routers actually decline, all things remain equal

• Evidence– Peering increases for past 3 years– Maximum domain degree from 605 to 1459, roughly

50% expansion each year– Slight decrease in state for majority of nodes– Slight increase for rest of nodes

32

Answers: Scaling Trends 2

• How do rising subscriptions to individual applications affect multicast state?

– Follows power law• fraction of stateful routers grows proportional to

some constant power of multicast group size

– Exponents within each membership for the Internet similar over past 3 years

– Predictive of future state growth

35

Answers: State Concentration

• How concentrated is multicast state at the “core” routers?

– State concentration does not follow “10/90” rule even when session density is 0.1%

– Application-driven membership significantly impact state distribution and concentration

– Tunneling useful for multicast applications with very sparse and spread-out membership

36

Answers: State Reduction

• How prominent are non-branching routers? Are these routers stateful?

– Very prominent– Up to 2 orders of magnitude reduction is

possible even at top 10% most stateful nodes

– Substantial even at 90% session density– Promising approach

37

Outline


38

Large-scale Applications

• Large-scale applications: many receivers, many sources, rich data types, UI

• Multicast uses one data stream to satisfy potentially heterogeneous receivers

• Lead to Preference Heterogeneity– Users differ in interest on application data– E.g. Content delivery, news dissemination,

stock quotes, network games, DIS, etc

39

Example: Stock Quotes Service

...

AABC BACU CABL DAGR EACO FACO

www.StockCentral.com

Amy

INTCDELLCSCOMSFT

Bob

AAPLAMZNEWEBMSFTGABCQCOM

Cathy

PWBCSISIYHOO

...

40

Example: Network GamesA player's position in virtual environmentdrives its preferences on entity updates

41

Preference Heterogeneity

• Assign each logical data stream a unique multicast address ?

+No superfluous data

–Multicast routing state scalability

–Multicast address allocation and scarcity

–End-host connection maintenance

• 100% reliability not necessary– Different levels of reliability desired– Help to reduce NACK implosion

42

The Clustering Concept

completeheterogeneity

completesimilarity

UNICAST MULTICAST

CLUSTER

approximately similar sources and receivers into like groups

many smallgroups

43

Preference Clustering Protocol

• Clustering algorithm– On-line and adaptive to changes in preferences– Customizable to different application and data

types

• Signaling protocol– Coordinate clustering within an application– Scalable, fault tolerant and reliable through

decentralization, soft state and sampling

• API • Detailed evaluation

44

App-Level Tunable Reliability

• Consider application semantics in loss recovery decisions– Meta-data to describe data content– Temporal: statistics on update frequency– Semantic: magnitude or importance of

change– Policy-driven by individual receivers

45

Outline


46

Conclusions

• Comprehensive study on multicast state scalability– Scaling trends confirmed with past 3 years– State distribution and concentration– Potentials for reduction

• Mechanisms to accommodate problem for large-scale applications– Customizable and adaptive preference

clustering protocol– Tunable reliable multicast protocol

47

Future Directions

• Compare and contrast methodologically IP multicast and application-level multicast– Params: Topology, session density,

membership– Apps: Few-to-few, one-to-many– Metric: Bandwidth, latency, complexity, etc

• Placement of service agents in Internet– Spawning of new agents – Coalescing based on topology, user

population, network measurements

1 multicast forwarding and application state scalability in the internet tina wong dissertation...

Documents