1

A Comparative Study of Architectural Impact on BGP Next-hop Diversity

15th IEEE Global Symposium, March 2012

Jong Han Park1, Pei-chun Cheng2, Shane Amante3, Dorian Kim4, Danny McPherson5, Lixia Zhang2

1AT&T Labs2University of California, Los Angeles

3Level-3 Communications Inc.4NTT Communications Inc.

5Verisign Inc.

2

Why this work?• Large ISPs have deployed i-BGP Route Reflections

for scalability• Hierarchical Route Reflection has a common

perception for reducing path diversity• Is the perception correct?• What factors have most impact on path diversity?• This measurement study aims to answer the above

two questions

3

Next-Hop Diversity: Definitions

• Next-hop POP: the city in which the next-hop router is located for a given destination prefix

• Next-hop AS: the neighboring AS used to reach a given destination prefix

POP1

POP2

POP3

ASX AS1

AS2

NH Router1

NH Router2

NH Router3

AS3

ASO

Prefix p

4

But not all existing paths visible

5

Data collection settings

• Data collected from 2 large ISPs with different i-BGP architectures– ISPFM: full-mesh i-BGP backbone

– ISPRR: hierarchical Route Reflection i-BGP backbone

ISPRRISPFM

iBGP routeriBGP node type: BGP confederation

sub-AS sub-AS

1st level reflector 2nd level reflector3rd level reflector

iBGP node type:

Reflector to Client Peer

Backbone sub-AS (full-mesh)

BGP data collection POPs

sub-AS

6

Next-hop diversity of the 2 ISPs

• Dataset: routing table snapshot on June 03, 2010

• ISPFM (ISPRR) has ~66% (50%) prefixes with next-hop POP diversity >=9 (7)

• ISPFM (ISPRR) has ~10% (34%) of all prefixes with only one next-hop POP

• ISPFM has higher overall next-hop diversity than ISPRR

ISPFM ISPRR

7

Investigating the causes for the differences• Estimating available paths based on BGP dynamics

[Oliveira’09]

– Identify prefixes with at least one route flap (i.e., becoming completely unreachable from all routers and becoming reachable again)

– Record all paths with associated BGP attribute values

• Simulate BGP best path selection at the AS level– IN: available paths to reach a given prefix– OUT: amount of eliminated paths at each step of BGP best path

selection

**[Oliveira’09] Quantifying Path Exploration in the Internet by Oliveira et al, Transactions on Networking 2009

8

Examples of path elimination in simulation

ASX

AS2

AS1

AS3

ASOPrefix p

BGP best path selection

1. Highest Local preference2. Shortest AS_PATH length3. Lowest Origin4. Lowest MED5. Lowest IGP cost6. More …

Path diversity loss due toTopology dependent factors

PATH 1PATH 2

PATH 3

PATH 4

LOCAL_PREF AS_PATH ORIGIN MEDPATH 1 100 AS1-ASO 0 0

PATH 2 100 AS1-ASO 0 50

PATH 3 80 AS1-ASO 0 0

PATH 4 100 AS2-AS-3-ASO 0 50

Available Next-hop POP = 4-Local_Pref, Next-hop POP = 3-ASPathLen, Next-hop POP = 2-Origin, Next-hop POP = 2-MED, Next-hop POP = 1

9

Simulation results

• Dataset: updates from all backbone routers during the 1st week of June 2010

• Both ISPs suffer a significant next-hop diversity reduction– The first 2 criteria reduces up to 42%

• Only minor reduction (less than 2.9%) due to topology dependent factors

10

Well-engineered RR placement min. diversity loss

• Not all regions (or mega-POPs) are equal– Place more backbone RRs in the regions with higher prefix injection density

• In case of ISPRR

– For more than 50% of prefixes, RRs exist in the nearest POPs– The current RR placement is nearest for more than 85% of prefixes after

eliminating paths using the top 4 BGP path selection criteria

Tier-1 backbone RRs (continent)

Tier-2 backbone RRs (regions or mega-POPs)

Tier-3 RRs (POPs)

B1

PEs

B2B3

Prefix p

Available

11

Summary• Engineering efforts going on aiming to increase

path diversity• Lack of quantitative study on existing path

diversity and impacting factors• Our measurement/simulation results based on

iBGP data from 2 large ISPs shed new insights:– Routing policies: main factor– Impact of different iBGP architectures insignificant • Can be further mitigated by well-engineered i-BGP topology

12

Questions?

Thank you!