arista networks vxlan overview - ohio networking user...
TRANSCRIPT
Confidential. Copyright © Arista 2016. All rights reserved.
Alternate text color: Hex color# 112346
ARISTA color Palette
Confidential. Copyright © Arista 2016. All rights reserved.
Arista Networks
Chris Kane, [email protected], CCIE #14430 @ccie14430 Columbus, OH (OH, MI, KY)
VXLAN – Overview
Agenda
2
What problem are we trying to solve? Data Center Interconnect VM Mobility Naked STP EoMPLS VPLS OTV VXLAN EVPN
Confidential. Copyright © Arista 2016. All rights reserved.
Industry Overview
Disclaimer
All opinions expressed in this presentation are mine (Chris Kane) and do not necessarily reflect those of my
current or previous employers
3
Confidential. Copyright © Arista 2016. All rights reserved.
Industry OverviewWhat Problem Are We Trying To Solve
4
One Data Center Redundancy Needed
Second Data Center (ex. Sungard)
Wasted Resources and Purchase Priority Rights
Own Second Data Center Wasted Resources if Active/DR and long recovery
Active/Active Mobility (ex. Re-IP for vMotion)
DCI (Data Center Interconnect) STP issues propagate quickly
I want Active/Active, no re-IP, no STP eruptions and it must be easy to manage
Confidential. Copyright © Arista 2016. All rights reserved.
Naked STP
• “We’ll just extend Layer 2 from Data Center 1 to Data Center 2” - STP Loop Prevention kicks in and blocks some of our paths ≫ Less bandwidth
≫ Longer convergence events (failure detection, MAC address table learning, etc…)
- But what about First Hop Redundancy Protocols? - Um….we had an STP issue in Data Center 1 that in a heartbeat just wiped out Data Center 2
5
Layer 2
Layer 2 Layer 2
Layer 2
Layer 2STP BPDUs
STP BPDUsLayer 2
STPBlocking
Loop Prevention
STPBlocking
Loop Prevention
STPBlocking
Loop Prevention
Confidential. Copyright © Arista 2016. All rights reserved.
EoMPLS
• AToM = Any Transport over MPLS
• RFC 4448 (April 2006) Updated by RFC 5462 (February 2009)
• To provide coverage for various use cases and leverage existing backbone MPLS; carriers offered both Layer 2 VPNs and Layer 3 VPNs
• Ethernet is another service over MPLS (Also included ATM and Frame Relay)
• A Pseudowire is a single point-to-point Layer 2 connection provided over an MPLS backbone
6
Confidential. Copyright © Arista 2016. All rights reserved.
EoMPLS
• Could buy Pseudowire service from a provider
• Or could roll your own Pseudowire if you have the appropriate equipment
- Vendors often required very specific hardware components and software versions
- Much like VRF in that only the most basic elements of an MPLS network are needed
• Two modes for Ethernet
- One option is to map Pseudowires on a per-VLAN basis (VLAN mode)
- Another associates the entire Ethernet interface to the Pseudowire (Port mode)
7
Confidential. Copyright © Arista 2016. All rights reserved.
EoMPLS – Example
8
PE/P
PE/P PE/P
PE/P
Layer 2
Layer 2 Layer 2
Layer 2
Layer 3
Layer 3
LoopbackCable
LoopbackCable
LoopbackCable
LoopbackCable
EoMPLS
EoMPLS
Confidential. Copyright © Arista 2016. All rights reserved.
EoMPLS – Example Configuration
• On the PE router (Port mode) config t interface Gig1/0 xconnect x.x.x.x 10 encapsulation mpls
- And on the P routers including the appropriate signaling protocol and label switch mapping of Tunnel label and inner VC label
9
Confidential. Copyright © Arista 2016. All rights reserved.
EoMPLS
10
• Pros - Layer 2 extension over Layer 3 - Service Provider managed
• Cons - Service Provider managed ≫ MRC ≫ Change Management
≫ Change Costs
- No switching behaviors incorporated
- If running your own; requires specific hardware options and software versions
- High CAPEX - High OPEX
Confidential. Copyright © Arista 2016. All rights reserved.
VPLS
• Virtual Private LAN Service
• Where EoMPLS is often point-to-point (Pseudowires), VPLS provides an Ethernet multipoint service
• Does more to take advantage of some switching features - Data Plane learning ≫ MAC address learning and aging out are part of the service
• While VPLS had been widely deployed by Service Providers, several short comings led to the development of other solutions (VXLAN -> EVPN) - Lack of multi-site Active/Active forwarding support - Limited Multicast support
11
Confidential. Copyright © Arista 2016. All rights reserved.
VPLS - Example
12
CE
CE
CE
PE
PE
PE
P
Data EthernetHeader Data Ethernet
HeaderAToMHeader Data Ethernet
Header
Data 802.1QHeader Data 802.1Q
HeaderAToMHeader Data 802.1Q
Header
CE
AccessPort
AccessPort
TrunkPort
TrunkPort
Confidential. Copyright © Arista 2016. All rights reserved.
VPLS – Configuration Example
PE example config t mpls label protocol ldp mpls ldp discovery targeted-hello accept mpls ldp router-id loopback0 force interface Gi1/0 ip address x.x.x.x/30 mpls ip I2 vfi <name> vpn id <oui:vpn-index> neighbor <remote-PE-loopback> encapsulation mpls interface vlan xxx xconnect vfi <vfi name>
13
Confidential. Copyright © Arista 2016. All rights reserved.
VPLS
• Pros - Service Provider managed - Multi-homing is supported
14
• Cons - Service Provider managed ≫ MRC ≫ Change Management
≫ Change Costs
- Provider to Provider interconnects, like the last mile, may conflict (read Battle Scar)
- If running your own; required specific hardware options and software versions
- High CAPEX - High OPEX - Multi-homing with all active
forwarding is not supported - Lacks Multicast support
Confidential. Copyright © Arista 2016. All rights reserved.
OTV
• Overlay Transport Virtualization
- OTV is a MAC-in-IP method of extending Layer 2 over a Layer 3 infrastructure
- Underlying routing protocol mechanism is IS-IS - Supports being deployed as either Multicast-based or Unicast-based
• Terminology - Join Interface – WAN Layer 3 interface - Overlay Interface – Virtual interface to perform encap/decap functions - Internal Interface – Where Layer 2 adjacency to internal data center LAN
occurs - Overhead = 42 byte header (needs Jumbo frames enabled on all possible
paths) - Sets the DF bit on all packets
15
16
OTV Example
Layer 3 WAN
Layer 3 WAN
AED for Odd VLANs AED for Odd VLANs
AED for Even VLANs AED for Even VLANs
OTV
OTV
Data Center 1 Data Center 2
AED = Authoritative Edge Device
OTVVDC
OTVVDC
OTVVDC
OTVVDC
17
OTV – Multicast Configuration Example
config t feature otv otv site-vlan 101 otv site-identifier 0000.0000.0001 interface overlay1 otv join-interface port-channel 10 otv control-group 239.1.1.1 otv data-group 232.1.1.0/24 otv extend-vlan 2-20 interface port-channel 10 mtu 9216 ip address x.x.x.x/30 ip igmp v3
18
OTV – Unicast Configuration Example
config t feature otv otv site-vlan 101 otv site-identifier 0000.0000.0001 interface overlay1 otv join-interface port-channel 10 otv adjacency-server unicast-only otv use-adjacency-server x.x.x.x unicast-only otv extend-vlan 2-20 interface port-channel 10 mtu 9216 ip address x.x.x.x/30 ip igmp v3
Confidential. Copyright © Arista 2016. All rights reserved.
OTV
• Pros - Layer 2 extension over Layer 3 - N7K or ASR1K - VLAN translation - Supports First Hop Redundancy
• Notables - Jumbo Frames on all possible
paths
19
• Cons - AED Odd and Even ownership - Specific hardware required - Proprietary - Not the go-forward strategy (VXLAN
and/or EVPN) - PIM requirement for Multicast
deployments - Join interface can’t have PIM enabled
because OTV sets that interface as an IGMP host
- ISSU not available on devices running OTV. OTV must be shutdown
Confidential. Copyright © Arista 2016. All rights reserved.
VXLAN
20
Confidential. Copyright © Arista 2016. All rights reserved.
VXLAN
• Virtual Extensible LAN
• RFC 7348 - Initial proposal August 2011, fully published in 2014
• Industry Standard
• UDP is used as a tunneling mechanism, wrapped around the MAC addresses
• Used for intra-data center Layer 2 over Layer 3
• Used for inter-data center Layer 2 over Layer 3 (most common in my patch)
21
Confidential. Copyright © Arista 2016. All rights reserved.
VXLAN
• Virtual Extensible LAN
• Scalable for large throughput numbers - Real world example; an IXC pushes 350 Gbps as an average run rate
• More in the reading and due diligence than in the configuration and operation
• Industry standard provides interoperability - Either Hardware VTEP to Hardware VTEP - Software VTEP to Software VTEP - Or Software VTEP to Hardware VTEP
22
Confidential. Copyright © Arista 2016. All rights reserved.
VXLAN - Header
23
24
VXLAN – Intra Data Center Example
Compute
Compute
LACP
LACP
Compute
Compute
LACP
LACP
Compute
Compute
LACP
LACP
MLAG
eBGP
Layer 3 Layer 3
VLANs 2 - 9 VLANs 10 - 19 VLANs 20 - 29
VTEP VTEP VTEP VTEP VTEP VTEP
Spine
Leaf
MLAGMLAG
10 GbE
40 GbE
1 GbE
100 GbE
Firewalls
Controllers
LACP
LACP
VTEP VTEP
Compute Leaf Compute Leaf Storage Leaf Services LeafVLANs 2 - 29
MLAG
VXLAN
25
VXLAN – Inter Data Center Example
Compute Compute
Compute
Layer 3 WAN
Layer 3 WAN
Compute
LACP
LACP
LACP
LACP
MLAG MLAG MLAGMLAG
MLAG MLAG
MLAG MLAG
VTEP
Compute
Compute
SVI
Jumbo Frames
Jumbo Frames
VXLAN
VXLAN
Compute
Compute
West Data Center East Data Center
Primary FW Secondary FW
10 GbE
40 GbE
1 GbE
Design Notes
One EOS image for all switch modelsLine rate forwarding on all interfacesActive/Active Forwarding for both Layer 2 and Layer 3Industry Standard VXLAN
Spine Spine
LeafLeaf
LeafLeaf
LeafLeaf
LeafLeaf
SpineSpine
VTEPVTEP
VTEPMLAG
MLAG
LACP
LACPLACP LACP
LACP
LACP
MLAG
MLAG
100 GbE
VXLAN
26
VXLAN – Inter Data Center Production Example
Confidential. Copyright © Arista 2016. All rights reserved.
VXLAN – Configuration Example
Manual Flood List interface Vxlan1 vxlan source-interface Loopback1 vxlan udp-port 4789 vxlan flood vtep 161.1.1.4 161.1.1.5 vxlan vlan 101 vni 20101 vxlan vlan 102 vni 20102 vxlan vlan 103 vni 20103 vxlan vlan 104 vni 20104 vxlan vlan 105 vni 20105
27
Dynamic Flood List (CVX) interface Vxlan1 vxlan source-interface Loopback1 vxlan controller-client vxlan udp-port 4789 vxlan vlan 101 vni 20101 vxlan vlan 102 vni 20102 vxlan vlan 103 vni 20103 vxlan vlan 104 vni 20104 vxlan vlan 105 vni 20105
Confidential. Copyright © Arista 2016. All rights reserved.
VXLAN – Example ‘show’ commands
28
Confidential. Copyright © Arista 2016. All rights reserved.
VXLAN – CVX with BYOC
29
Overlay Controller
Network Layer
Controller Layer
OVSDB
Topology/Device Dependent
Traditional Approach
Confidential. Copyright © Arista 2016. All rights reserved.
VXLAN – CVX with BYOC
30
OVSDB
Overlay Controller
Network Layer
Controller Layer
10x Improvement
SysDB State Sync
Topology/Device Abstraction
CloudVision Approach
Confidential. Copyright © Arista 2016. All rights reserved.
VXLAN – CVX with NSX
31
Confidential. Copyright © Arista 2016. All rights reserved.
VXLAN – Integration Example
32
Confidential. Copyright © Arista 2016. All rights reserved.
VXLAN
• Pros - Industry Standard - Scalability (from a traffic volume
perspective) - Maturity ≫ MCAST, UNICAST
- Widely deployed - Intra-data center - Inter-data center - VLAN translation - Supports First Hop Redundancy
Notables - Jumbo Frames required on all
possible paths
33
• Cons - Lack of Control Plane (in defined
standard) - Some interoperability
considerations ≫ MCAST v UNICAST