On the Impact of Layer-2 on Node Degree Distribution

Pascal Mérindol, Benoit Donnet, Jean-Jacques Pansiot, Olivier Bonaventure

Melbourne - 2th November 2010

Internet Measurement Conference 2010

IMC 2010 - On the Impact of Layer-2 on Node Degree Distribution

Agenda

‣ Topology Discovery Background‣ Data Collection : mrinfo-rec‣ Problem Statement‣ Revisiting the node degree distribution ‣ Domain Specificities‣ Model key distributions

Topology Discovery

IMC 2010 - On the Impact of Layer-2 on Node Degree Distribution

‣ Internet seen as a dynamic graph✓ of IP interfaces

➡ traceroute, route_record✓ of routers

➡ alias resolution : ally, iffinder, ...✓ of Autonomous Systems

➡ IP to AS mapping (routeview project), router to AS mappping ?

‣ Goals✓ IP network models & simulations✓ ground truth input for topology generation

R1

R0

R2switch

1.1.0.2

1.1.1.1

1.1.1.2

1.1.0.1

1.1.2.3

1.1.2.1

R5

R6

1.1.2.2

1.1.3.1

1.1.0.2 [version 12.4]1.1.0.2 → 1.1.0.1 [1/0/pim/querier]1.1.2.3 → 1.1.2.1 [1/0/pim/querier]1.1.2.3 → 1.1.2.2 [1/0/pim/querier]1.1.3.1 → 0.0.0.0 [1/0/pim/leaf]

Topology Discovery

IMC 2010 - On the Impact of Layer-2 on Node Degree Distribution

mrinfo‣ Topology discovery using mrinfo

➡ Uses IGMP messages✓ ASK_NEIGHBORS✓ NEIGHBORS_REPLY

➡ Output✓ All multicast interfaces of a given router✓ All multicast neighbors/links

‣ mrinfo applied recursively➡ mrinfo-rec✓ probe all neighbors✓ daily based

IMC 2010 - On the Impact of Layer-2 on Node Degree Distribution

mrinfo-rec

Pajek

Sprint - 2006

while (--ncount >= 0 && p < ep ) {

/*NEIGHBOR ADDRESS*/ register u_int32 neighbor = *(u_int32*)p; p += 4;

‣ Limitations➡ multicast scope➡ IGMP filtering➡ non compliant routers

‣ Advantages ➡ network friendly probing : 1 probe injected per router➡ aliasing : no need to gather IP interfaces➡ forwarding independent : backup links visible [IMC2009]➡ layer-2 vision : distinguish the IP logical layer over MAC

IMC 2010 - On the Impact of Layer-2 on Node Degree Distribution

The dataset‣ More than 4 years of daily collected data

➡ ~10000 routers➡ ~100000 IPs➡ ~300-800 AS

‣ Raw data towards IP network graphs➡ Cleaning process (tunnels, private IP, disabled IP, ...)➡ Router to AS mapping [PAM2010]➡ Layer-2 Inference➡ Graph selection : one per AS per month

✓ dimensions (|V=R+S|,|E|)✓ connectivity✓ influence of L2 hardware✓ ...

V = set of vertex/nodes E = set of edgesR = set of routersS = set of layer-2 nodes

Num

ber o

f nod

es w

ith k

link

s

Number of links (k)

Many nodes witha few links

A few hubs withlarge number of links

Topology Discovery

IMC 2010 - On the Impact of Layer-2 on Node Degree Distribution

Problem : MAC vs. IP layerR1

R2

R3

R4

R1

R2

R3

R4

mrinfo, layer-2 view: L2 graph traceroute, layer-3 view: IP graph

‣ The node degree distribution follows a heavy tailed power law ?

‣ What is the impact of L2 on scale free & preferential attachment models ?

Ethernet Switches & Shared Risk Link Groups

(SRLG)

IP logical view

IMC 2010 - On the Impact of Layer-2 on Node Degree Distribution

Problem : Global Overview

LAN

MAN

WAN

0.0.0.0 ← 10.0.0.1

0.0.0.0 ← 192.168.0.1

← point-to-point connection

← point-to-multipoint connection

IP-only layer vision

L3 routerL2 switch

L1 hub

L1 bus

IMC 2010 - On the Impact of Layer-2 on Node Degree Distribution

‣ Using our dataset, we can infer several kinds of L2 deviceL2 inference validation

‣ Focusing on Ethernet Broadcast networks, we check our inference validity using three rules :

➡ symmetry, querier election and subnet mask.‣ Three states: coherent - incoherente - incomplete

‣ Most of L2 inferred devices seems to reveal Ethernet broadcast devices such as switches

minimal covering prefix for p2p and p2mp connections proportion of Ethernet switches

Revisiting the node degree distribution‣ We construct two graphs of the same network

➡ the L2 inferred graph (L2-aware) and the L3 graph (L2-agnostic)

t(x): degree distribution of L2 and L3 nodes in the L2 graphr’(x): degree distribution of L3 nodes in the L3 graph

Overall networks analysis

‣The power law distribution is much less heavy tailed considering an L2 view !

R1

R2

R3

R4

R1

R2

R3

R4

t(1)=4, t(4)=1

r’(3)=4

IMC 2010 - On the Impact of Layer-2 on Node Degree Distribution

‣What impact on other graph/routing characteritics ?

Revisiting the node degree distribution

IMC 2010 - On the Impact of Layer-2 on Node Degree Distribution

✓α : proportion of routers connected to at least one L2 device ✓β - L2 : proportion of p2p connections going through an L2 device in the L2 aware network✓β - L3 : proportion of p2p connections going through an L2 device in the L2 agnostic network

‣ ~10-30 % of L2 nodes...

...but resulting in two strongly different graphs :

➡ The number of edges/connections in the L3 graph is much more larger than in the L2 graph,

➡ Between 40% and 75% of routers are connected to an L2 device,

➡ Most of the L3 connections rely on L2 Ethernet switches... ...even in the L2 graph, almost half of router-to-router connections goes through an L2 node !

Telecom Italia

Per AS analysis: a uniform observation ?

Sprint IUNET

Level 3

IMC 2010 - On the Impact of Layer-2 on Node Degree Distribution

t(x): degree distribution of L2 and L3 nodes in the L2 graphr’(x): degree distribution of L3 nodes in the L3 graph

R1

R2

R3

R4

R1

R2

R3

R4

Why such a great tail shift ?

IMC 2010 - On the Impact of Layer-2 on Node Degree Distribution

‣How a small (L2-aware) degree router can become a large L3 degree router ?

L2 aware degree distributionL3 degree distribution

‣ Must look at specific distributions on the L2-aware graph

➡ r : router degree distribution

➡ s : switch degree distribution

➡ b : L2 degree distribution of routers (#L2 neighbors)

➡ p : degree distribution of routers connected to L2 devices («subdistribution» of r)

multi-graph problem

R

RR

3 7

A closer look at key distributionsr - L3 node degree

b - #L2 neighbors

p - routers connected to L2 neighbors

s- L2 node degree

IMC 2010 - On the Impact of Layer-2 on Node Degree Distribution

Sprint IUNET

Telecom ItaliaLevel 3

Practical IP networks design

IMC 2010 - On the Impact of Layer-2 on Node Degree Distribution

‣ MAN access distributes IP connectivity at low cost

➡ quite typical accross our dataset ...

‣ Redundancy may exacerbate this phenomenon

‣ VLAN can introduce an additional MAC logical layer (lower-bound analysis)

‣ Access/core networks ?Level 3 - AS3356

FRANKFURT MAN ACCESSASBR

SWITCH

ROUTER

NEIGHBOR AS

VLAN ?

Sprint

Telecom Italia

IUNET

A large degree router has more L2 neighbors

IMC 2010 - On the Impact of Layer-2 on Node Degree Distribution

p: degree distribution of L3 nodes connected to b switches in the L2 graphb: number of L2 neighbors

Level 3

Towards a model ?

IMC 2010 - On the Impact of Layer-2 on Node Degree Distribution

‣How to model this shift ?‣What are key factors ?

L2 aware degree distributionL3 degree distribution

RR

‣ «Replay» the collected data using key distributions... ...and one of their possible correlations

➡ r (router degree distribution), s (switch degree distribution)

➡ b (L2 degree distribution of routers = #L2 neighbors)

➡ p×b : correlation between p and b distributions

‣ Is this sufficient ? What about AS specificities ?

Towards a model ?

IMC 2010 - On the Impact of Layer-2 on Node Degree Distribution

Sprint

Level 3 Telecom Italia

data: degree distribution of L3 nodes in the L2-agnostic graphmodel: degree distribution of L3 nodes in the generated graph

IUNET

Conclusion

IMC 2010 - On the Impact of Layer-2 on Node Degree Distribution

‣ mrinfo-rec is a useful tool for three reasons➡ describe a connected multicast topology at the router level (no need for alias resolution)

➡ can discover backup links (no forwarding dependence)

➡ able to natively infer L2 devices (hybrid bipartite graph)

‣ Layer-2 switches strongly impact the node degree distribution➡ most of IP router-to-router connections go through Broadcast Ethernet switches !

➡ such switches are generally connected to a large number of routers

➡ correlation between the L2-aware degree of a router and its number of L2 neighbors

Ongoing Work

IMC 2010 - On the Impact of Layer-2 on Node Degree Distribution

‣ Improve Active Topology Discovery➡ using the best of mrinfo, traceroute and alias resolution techniques

➡ MERLIN : A new measurement platform to MEasure the Router Level of the INternet

‣ Various Analysis on other graph/routing metrics➡ clustering coefficient/assortativity : understand the dynamic of attachement preferences

➡ path diversity : measure the physical resilience of IP networks

Questions ?

IMC 2010 - On the Impact of Layer-2 on Node Degree Distribution

✓ Pascal Mérindol, Benoit Donnet, Jean-Jacques Pansiot, Matthew Luckie, Young Huyn. MERLIN: MEasure the Router Level of the INternet

Université catholique de Louvain, Technical Report 2010-3, September 2010.

✓ Jean-Jacques Pansiot, Pascal Mérindol, Benoit Donnet, and Olivier Bonaventure. Extracting Intra-Domain Topology from mrinfo ProbingIn Proc. Passive and Active Measurement Conference (PAM), April 2010.

✓ Pascal Mérindol, Virginie Van den Schrieck, Benoit Donnet, Olivier Bonaventure and Jean-Jacques Pansiot. Quantifying ASes Multiconnectivity using Multicast InformationIn Proc. ACM/USENIX Internet Measurement Conference (IMC), November 2009.

‣http://inl.info.ucl.ac.be/content/mrinfo‣http://svnet.u-strasbg.fr/mrinfo/