a first-principles approach to understanding the internet’s router-level topology 2004 acm sigcomm...

A First-Principles Approachto Understanding the Internet’s

Router-level Topology

2004 ACM SIGCOMM

Portland, OR

• Evaluate performance of protocols• Protect Internet• Resource provisioning• Understand large scale networks

Why Topology

Challenges

• Large Size• Real topologies are not publicly available• Incredible variability in many aspects

Trends in Topology Modeling

Observation Modeling Approach

• Real networks are not random, but have obvious hierarchy.

• Structural models (GT-ITM Calvert/Zegura, 1996)

• Long-range links are expensive • Random graph models (Waxman, 1988)

• Internet topologies exhibit power law degree distributions (Faloutsos et al., 1999)

• Degree-based models replicate power-law degree sequences

A few nodes have lots of connections

Ran

k R(d)

Degree d

Source: Faloutsos et al. (1999)Power Laws and Internet Topology

• Router-level graph & Autonomous System (AS) graph• Led to active research in degree-based network models

Most nodes have few connections

R(d

) =

P (

D>

d) x

#no

des

Degree-Based Models of Topology

• Preferential Attachment– Growth by sequentially adding new nodes– New nodes connect preferentially to nodes having

more connections– Examples: Inet, GPL, AB, BA, BRITE, CMU powe

r-law generator

Degree-Based Models of Topology

• Expected Degree Sequence– Based on random graph models that skew

probability distribution to produce power laws in expectation

– Examples: Power Law Random Graph (PLRG), Generalized Random Graph (GRG)

• Preferential Attachment (PA)– Growth by sequentially adding new nodes– New nodes connect preferentially to nodes having

more connections– Examples: Inet, GPL, AB, BA, BRITE, CMU powe

r-law generator

Features of Degree-Based Models

• Degree sequence follows a power law (by construction)• High-degree nodes correspond to highly connected

central “hubs”, which are crucial to the system• Achilles’ heel: robust to random failure, fragile to specific

attack • “scale-free” in complex networks

Preferential Attachment Expected Degree Sequence

Our Approach• Consider the explicit design of the Internet

– Annotated network graphs (capacity, bandwidth)– Technological and economic limitations– Network performance– Heuristic optimized tradeoffs (HOT)

• Seek a theory for Internet topology that is explanatory and not merely descriptive.– Explain high variability in network connectivity– Ability to match large scale statistics (e.g. power

laws) is only secondary evidence

Trends in Topology ModelingObservation Modeling Approach

• Real networks are not random, but have obvious hierarchy.

• Structural models (GT-ITM Calvert/Zegura, 1996)

• Long-range links are expensive • Random graph models (Waxman, 1988)

• Internet topologies exhibit power law degree distributions (Faloutsos et al., 1999)

• Degree-based models replicate power-law degree sequences

• Physical networks have hard technological (and economic) constraints.

• Optimization-driven models topologies consistent with design tradeoffs of network engineers

100

101

102

Degree

10-1

100

101

102

103

Ban

dwid

th (

Gbp

s)

15 x 10 GE

15 x 3 x 1 GE

15 x 4 x OC12

15 x 8 FE

Technology constraint

Total Bandwidth

Bandwidth per Degree

Router Technology ConstraintCisco 12416 GSR, circa 2002

high BW low degree high

degree low BW

0.01

0.1

1

10

100

1000

10000

100000

1000000

1 10 100 1000 10000degree

To

tal R

ou

ter

BW

(M

bp

s)

cisco 12416

cisco 12410

cisco 12406

cisco 12404

cisco 7500

cisco 7200

linksys 4-port router

uBR7246 cmts(cable)

cisco 6260 dslam(DSL)

cisco AS5850(dialup)

approximateaggregate

feasible region

Aggregate Router Feasibility

Source: Cisco Product Catalog, June 2002

core technologies

edge technologies

older/cheaper technologies

Rank (number of users)

Con

nect

ion

Spe

ed (

Mbp

s)

1e-1

1e-2

1

1e1

1e2

1e3

1e4

1e21 1e4 1e6 1e8

Dial-up~56Kbps

BroadbandCable/DSL~500Kbps

Ethernet10-100Mbps

Ethernet1-10Gbps

most users have low speed

connections

a few users have very high speed

connections

high performancecomputing

academic and corporate

residential and small business

Variability in End-User Bandwidths

Heuristically Optimal Topology

Hosts

Edges

Cores

Mesh-like core of fast, low degree routers

High degree nodes are at the edges.

SOX

SFGP/AMPATH

U. Florida

U. So. Florida

Miss StateGigaPoP

WiscREN

SURFNet

Rutgers U.

MANLAN

NorthernCrossroads

Mid-AtlanticCrossroads

Drexel U.

U. Delaware

PSC

NCNI/MCNC

MAGPI

UMD NGIX

DARPABossNet

GEANT

Seattle

Sunnyvale

Los Angeles

Houston

Denver

KansasCity

Indian-apolis

Atlanta

Wash D.C.

Chicago

New York

OARNET

Northern LightsIndiana GigaPoP

MeritU. Louisville

NYSERNet

U. Memphis

Great Plains

OneNetArizona St.

U. Arizona

Qwest Labs

UNM

OregonGigaPoP

Front RangeGigaPoP

Texas Tech

Tulane U.

North TexasGigaPoP

TexasGigaPoP

LaNet

UT Austin

CENIC

UniNet

WIDE

AMES NGIX

PacificNorthwestGigaPoP

U. Hawaii

PacificWave

ESnet

TransPAC/APAN

Iowa St.

Florida A&MUT-SWMed Ctr.

NCSA

MREN

SINet

WPI

StarLight

IntermountainGigaPoP

Abilene BackbonePhysical Connectivity(as of December 16, 2003)

0.1-0.5 Gbps0.5-1.0 Gbps1.0-5.0 Gbps5.0-10.0 Gbps

Cisco 750X

Cisco 12008

Cisco 12410

OC-3 (155 Mb/s)OC-12 (622 Mb/s)GE (1 Gb/s)OC-48 (2.5 Gb/s)10GE (10 Gb/s)

CENIC Backbone (as of January 2004)

dc1

dc2

dc3

hpr

dc1

dc3

hpr

dc2

dc1

dc1 dc2

hpr

hpr

SACOAK

SVL

LAX

SDG

SLOdc1

FRGdc1

FREdc1

BAKdc1

TUSdc1

SOLdc1

CORdc1

hprdc1

dc2

dc3

hpr

AbileneLos Angeles

AbileneSunnyvale

Corporation for Education Network Initiatives in California (CENIC)

1

10

100

1000

10000

100000

1 10 100degree

tota

l BW

(Mbp

s)12410

12008

750X

12410 Feasible Region

12008 Feasible Region

750X Feasible Region

CENIC Router Configurations, Jan. 2004

Two Different Perspectives

• Degree-based perspective– Match aggregate statistics– Suggest high-degree central hubs

• First principles perspective– Technology and economic constraints – Performance– Suggest fast, low-degree core routers– Consistent with physical design of real networks

How to reconcile these two perspectives?

Same Degree Distribution

PA

PLRG

Same Degree Distribution

HOT

Same Degree Distribution

PA PLRG/GRG

HOT Abilene-inspired Sub-optimal

Network PerformanceGiven realistic technology constraints on routers, how well is the

network able to carry traffic?

Step 1: Constrain to be feasible

Abstracted Technologically Feasible Region

1

10

100

1000

10000

100000

1000000

10 100 1000

degree

Ban

dw

idth

(M

bp

s)

kBxts

BBx

ijrkjikij

ji jijiij

,..

maxmax

:,

, ,

Step 3: Compute max flow

Bi

Bj

xij

Step 2: Compute traffic demand

jiij BBx

PA PLRG/GRGHOT

Structure Determines Performance

P(g) = 1.19 x 1010 P(g) = 1.64 x 1010 P(g) = 1.13 x 1012

Likelihood-Related Metric

• Easily computed for any graph• Depends on the structure of the graph, not the

generation mechanism• Measures how “hub-like” the network core is

j

connectedji

iddgL ,

)(Define the metric (di = degree of node i)

For graphs resulting from probabilistic construction (e.g. PLRG/GRG),

LogLikelihood (LLH) L(g)

Interpretation: How likely is a particular graph (having given node degree distribution) to be constructed?

Lmax

l(g) = 1P(g) = 1.08 x 1010

P(g) Perfomance (bps)

PA PLRG/GRGHOT Abilene-inspired Sub-optimal

0 0.2 0.4 0.6 0.8 1

1010

1011

1012

l(g) = Relative Likelihood

Lmax

l(g) = 1P(g) = 1.08 x 1010

P(g) Perfomance (bps)

PA PLRG/GRGHOT Abilene-inspired Sub-optimal

0 0.2 0.4 0.6 0.8 1

1010

1011

1012

l(g) = Relative Likelihood

Points to Emphasize

• Same Degree distribution can have different core structures

• Same Core structure can have different degree distributions

100 101 102100

101

102

Node Degree

No

de

Ran

k

100 101 102100

101

102

Node Degree

No

de

Ran

k

100 101 102100

101

102

No

de

Ran

k

Node Degree

Abilene-inspired core

Uniform high BW users

Low variability deg dist.

Abilene-inspired core

High variability at edges

Power-law deg distribution

Abilene-inspired core

Uniform low BW users

low variability deg dist.

Conclusions1. Probabilistic degree-based generation

mechanisms are likely to produce networks that have poor performance and are very unlikely to generate realistic networks.

2. Models of router-level topology should consider inherent technological and economic tradeoffs in network design, and they should consider issues such as performance over simple connectivity.

3. Realistic router-level topology generators will require additional work to incorporate other key features (e.g. geography, population density) into the framework

Future Work

• Validation– rely on cooperative ISPs to validate router technology

constraints– combine with heuristics and supplementary

measurements– develop annotated prototype ISP topology generator

A Tier 1 Router Bandwidth-Degree (Courtesy: Matt Roughan)

Future Work

• Validation– rely on cooperative ISPs to validate router technology

constraints– combine with heuristics and supplementary measure

ments– develop annotated prototype ISP topology generator– Contact: David Alderson <alderd@cds.caltech.edu>

• HOT-inspired framework for protocol stack– consider layering as optimization decomposition– study properties of `natural' decompositions– Contact: Lun Li <lun@cds.caltech.edu>