Shivlu Jain www.mplsvpn.info

Ethernet VPN Layer 2 Scalability

Shivlu Jain

7/30/2012

Shivlu Jain www.mplsvpn.info

Introduction

MPLS (Multi-Protocol Label Switching) is matured technology & has widely been opted by most of the

service providers across the globe. Initially it has been deployed for fast switching but due to its

scalability, resiliency & protocol agnostic nature made it more successful across the network. MPLS not

only provides the wan connectivity but also acts as a platform for service providers to offer different

kind of services which can further be used for monetization purpose.

VPLS (Virtual Private LAN Services) is one of the service offering in MPLS which helps to provide the

extension of broadcast domain from one to multiple sites over the wan. VPLS became more popular

after the outburst of data center interconnects. The utmost reason for the extension of layer 2 domains

is workload mobility (Migration of Virtual machines from one data center to another), high availability

clusters, and geographical redundancy.

Current Challenges with VPLS

1. Scaling of thousands of MAC addresses (Single VM requires single mac address):- Virtualization

applications are fueling the need of the mac-address in the network. A single server which can

host hundreds of virtual machines and every machine consume one mac address which clearly

justifies the scaling requirement of mac-address tables.

2. Optimal forwarding of multicast:- Multicast LSP can be formed in conjunction with VPLS but

limited to point to multipoint which consumes more network resources as there is no defined

set of parameters in VPLS to create multipoint to multipoint multicast LSPs.

3. MultiHoming:- VPLS supports Active/standby BGP multi homing model. MultiHoming with all

active attached circuits is not possible. In contract, customer can utilize only 50% of the links in

lieu of 100% payment.

4. C-Mac (Customer Mac) Transparency:- Current VPLS solution doesn’t support the transparency

of customer mac address.

5. Fast Convergence for C-Mac Flushing:- In case of failure of virtual machines or physical servers,

network re-convergence will occur which may lead to the mac flushing problems.

Shivlu Jain www.mplsvpn.info

Proposed Solution

Ethernet Virtual Private Network (E-VPN) is the proposed solution to overcome the issues highlighted by

VPLS. E-VPN uses the existing MPLS/IP backbone to transport the layer 2 connectivity among the various

data centers which are part of same VPN. Being layer-2 extension, the solution treats the mac addresses

as routable addresses and uses the existing MP-iBGP protocol to carry the customer mac addresses. In E-

VPN, mac learning at the edge routers doesn’t occur in data plane but in the control plane consequences

more control could be applied in terms of the learning mechanism. The process is similar to the IPVPN as

mentioned in RFC 4364. The policy attributes specified in E-VPN are almost similar in MPLS VPN. RD and

RT remains the same, but instead of virtual routing forwarding instance we have now Ethernet VPN

Instance. The information about Ethernet TAG of EVI is advertised by the new BGP NLRI which is E-VPN.

Data Center

Cloud

MES1MES2P1P2

P3

MES2

Destination

100 20

EVPN IGP Label

MES2

Destination

100 21

EVPN IGP Label

IGP Label

MES2

Destination

100 22

EVPN

MES2

Destination

100 PHP

EVPN

MES2

Destination

100

EVPN

Traffic Forwarding From Host(H2,M2) To Host(H1,M1)

Data Center

Cloud

H2,M2 H1,M1

Source Destination

H1,M1

MPLS

CLOUD

Figure 1

In EVPN, the mac learning could be of two types:-

1. Local Mac Learning

2. Remote Mac Learning

In local mac learning process, MPLS Edge Switch (MES) must support the local mac learning process

through standard protocols. Once the local learning process gets complete, MES can advertise the

locally learn mac address to remote MES nodes via MP-iBGP. This process of receiving the remote mac

addresses of attached customer via MP-iBGP is known as remote mac learning process.

Shivlu Jain www.mplsvpn.info

Solution for MultiHoming and Avoiding Layer 2 Loops in EVPN

Ethernet Segment ID (ESI) is used when Customer Edge device is multi homed to different MPLS Edge

Switches as shown in Figure 2. It has new MPLS BGP Label Extended community which is used for split

horizon procedures in multi homing scenarios. As depicted in figure 2, host H1 has mac address of M1. It

sends the broadcast request to MES-1 and MES2. MES-1 and MES-2 identified that the request is coming

from Extended Segment ID-1, so before replicating the frames both MESs will append a split horizon

label on the frames. Once it will be done, frames get exchanged among the MESs. All MESs check the SH

label and if found the same ESI-1 is directly attached, the traffic is silently dropped because a frame

originated by a segment must not be received by the same segment. This technique helps to avoid

loops in multi homing scenarios.

MES-1

MES-2

MES-3

MES-4

Data Center

Cloud/Enterprise MP-iBGP

Full Mesh

ES

I-1

H1,M1

Data Center

Cloud/Enterprise

H2,M2

Step-1H1,M1 sends broadcast request as source mac M1 and destination as Broadcast

Step-2MES-1 will append split horizon(SH) label for multi destination and distributes over MP-iBGP. MES-2,MES-3 and MES-4will use SH label to perform split horizon filtering for frames destines to ESI-1.

Step-3MES-3 will install that route as nexy hop MES-1 and MES-2

Step-1H1,M1 sends broadcast request as source mac M1 and destination as Broadcast

Step-2MES-2 will append split horizon(SH) label for multi destination and distributes over MP-iBGP. MES-2,MES-3 and MES-4will use SH label to perform split horizon filtering for frames destines to ESI-1.

Step-3MES-4 will install that route as next hop MES-1 and MES-2

Figure 2

Shivlu Jain www.mplsvpn.info

Note:- Split horizon label is only used for unknown unicast, multicast and broadcast

Role of Designated Forwarder

As per figure 2, MES-3 and MES-4 will receive the multi destination frames via MP-iBGP for particular

segment. How will it be decided which MES has to forward the frames to downstream segment? Only

Designated Forwarder will forward the frames to particular segment and Designated forwarder election

is performed by each PE advertising the ESI in BGP route. All the non-Designated Forwarder MES will

block their respective port for that segment as shown in Figure 3.

MES-1

MES-2

MES-3

MES-4

Data Center

Cloud/Enterprise MP-iBGP

Full Mesh

ES

I-1

H1,M1

Data Center

Cloud/Enterprise

H2,M2

ES

I-2

X

Designated Forwarder Election

MES-3 is elected as Designated Forwarder(Highest IP Address) for ESI-2 segment.

MES-4 is elected as non-Designated Forwarder for ESI-2 segment. So MES-4 port towards ESI-2 Segment will remain in blocking state

Figure 3

Load Balancing

As per figure 3, MES-3 & MES-4 is receiving the update of host H1 with Mac M1 from MES-1 and MES-2

with Ethernet segment of ESI-1. So MES-3 and MES-4 install the two routes in the Forwarding

Information Base. Once the traffic of M1 destination is received both the routers will do the load

balancing during forwarding. The core will forward the traffic on the basics of next hop information for

M1 which is MES-1 and MES-2.

Shivlu Jain www.mplsvpn.info

Scaling by using Provider Backbone Bridge (PBB)

The EVPN scalability is achieved by using the existing technique of Provider Backbone Bridge aka PBB.

Below are the advantages while using PBB in EVPN:-

1. Subnetting of C-MAC addresses is not possible. But by using PBB, B-MAC addresses can be

subnetted easily which leads to mac address scalability.

2. In case of shifting of VM or local customer networks from one DC to another requires lot of mac

flushing. But by using B-MAC that C-MAC flushing will become transparent which leads to fast

convergence.

3. Per Site Policy Support by using B-MAC

4. Device MultiHoming

5. Network MultiHoming

6. C-MAC addresses need to be distributed in BGP but by using PBB-EVPN C-MAC advertisement

could be limited by assigning multiple C-MAC addresses to single B-MAC address.

References

EVPN requirement http://tools.ietf.org/html/draft-sajassi-raggarwa-l2vpn-evpn-req-00 BGP/MPLS IP VPN http://tools.ietf.org/html/rfc4364 PBB-EVPN http://tools.ietf.org/html/draft-ietf-l2vpn-pbb-evpn-03 VPLS http://tools.ietf.org/html/rfc4762 EVP http://tools.ietf.org/html/draft-ietf-l2vpn-evpn-00