everyone in this room is a genius - amazon web … · 2013-2014 mpls, otv, lisp vxlan mpls, otv,...
TRANSCRIPT
![Page 1: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/1.jpg)
![Page 2: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/2.jpg)
Everyone in this room is a GENIUS
![Page 3: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/3.jpg)
![Page 4: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/4.jpg)
What are Best Practices ?
Learning from Others Mistakes
4
![Page 5: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/5.jpg)
Learning from your mistakes makes you
SMARTLearning from others mistakes makes you
GENIUS
![Page 6: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/6.jpg)
vPC Best Practices and Design on NXOS
Nazim Khan, CCIE#39502 (DC/SP)
Technical Marketing Engineer, Data Center Group
BRKDCT-2378
![Page 7: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/7.jpg)
Session Goals
• Best Practices and Designs for vPC – virtual port-channel
• Nexus 2000 (FEX) will only be addressed from vPC standpoint
• Fabricpath / vPC+ Overview
• vPC with FCOE
• vPC with VXLAN
• vPC with ACI
vPC : Get it Right the very First time
![Page 8: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/8.jpg)
Session Non-Goals
• vPC troubleshooting
• Details of vPC+
• Details of Fabricpath, FCoE, ACI and VXLAN
![Page 9: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/9.jpg)
Related Sessions at Cisco Live San Diego
Session Id Session Name
BRKDCT-2404 VXLAN deployment models - A practical perspective
BRKDCT-2081 Cisco FabricPath Technology and Design
BRKDCT-2458Nexus 9000/7000/6000/5000 Operations and
Maintenance Best Practices
BRKACI-2601 Real World ACI Deployment and Migration
![Page 10: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/10.jpg)
Agenda
• Feature Overview
• Configuration Best Practices
• Design Best Practices
• vPC Operations and Upgrade
• vPC with Fabric Technologies
• Scalability
• Reference Material
![Page 11: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/11.jpg)
Data Center Technology Evolution
FabricPath with vPC+
2010
2009
VPC
2008
STP2013-2014
MPLS, OTV,
LISP
VXLAN
MPLS, OTV,
LISP
2014-2015
ACI
2010
FEX with vPC
![Page 12: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/12.jpg)
Why vPC in 2015 ?
12
![Page 13: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/13.jpg)
vPC is Foundation
![Page 14: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/14.jpg)
Role of vPC in the Evolution of Data Center
• vPC launched in 2009
• Deployed by almost 95% of Nexus customers
• Used to redundantly connect network entities at the
edge of the Fabric
− Dual-homed servers (bare metal, blades, etc.)
− Network services (Firewalls, Load Balancers, etc.)
Unified Fabric
![Page 15: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/15.jpg)
Agenda
• Feature Overview− Concepts and Benefits− Terminology
![Page 16: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/16.jpg)
vPC Feature OverviewvPC Concept & Benefits
S1 S2
S3
• No Blocked Ports, More Usable Bandwidth, Load Sharing
• Fast Convergence
STP
S2S1
S3
vPC Logical Topology
S3
S1 S2
vPC Physical Topology
![Page 17: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/17.jpg)
Feature OverviewvPC Terminology
Layer 3 Cloud
vPC Member PortvPC
Orphan Device
Orphan Port
vPC Peer
CFS
vPC Domain
Peer-Link
vPC PeerKeepalive Link
S1
S3
S2
![Page 18: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/18.jpg)
SW3 SW4
vPC1 vPC2
vPC_PLink
vPC Peer-keepalive
vPC Failure Scenario vPC Peer-Keepalive Link up & vPC Peer-Link down
P S
Keepalive Heartbeat
Secondary vPCS
P Primary vPC
Suspend secondary
vPC Member Ports
vPC peer-link failure (link loss):
• vPC peer-keepalive up
• Status of other vPC peer known
• Both peers Active
• Secondary vPC peer disables all vPC’s
• Traffic from vPC primary.
• Orphan devices connected to secondary peer will be isolated
For YourReference
![Page 19: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/19.jpg)
vPC Failure Scenario – Dual ActivevPC Peer-Keepalive down followed by vPC Peer-Link down
P S
Secondary vPCS
P Primary vPC
P
SW3 SW4
vPC1 vPC2
vPC_PLink
vPC Peer-keepalive
Traffic Loss / Uncertain Traffic
Behavior
1. vPC peer-keepalive DOWN
2. vPC peer-link DOWN
3. DUAL-ACTIVE or SPLIT BRAIN
• vPC primary peer remains primary and secondary peer becomes operational primary role
• Result in traffic loss / uncertain traffic behavior
• When links are restored, the operational primary (former secondary) keeps the primary role & former primary becomes operational secondary
For YourReference
![Page 20: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/20.jpg)
Agenda
• vPC Configuration Best Practices− Building a vPC domain− Domain-ID− Peer-Link− Peer-Keepalive Link− Spanning-Tree− Peer-switch− Private VLAN (PVLAN)− Auto-recovery− Object tracking
![Page 21: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/21.jpg)
vPC Configuration Best PracticesBuilding a vPC domain – Configuration Steps
CFS
1. Define domains
2. Establish Peer Keepalive connectivity
3. Create a Peer link
4. Create vPCs
5. Make Sure Configurations are Consistent
(Order does Matter!)
S1 S2
S3
![Page 22: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/22.jpg)
vPC Configuration Best PracticesvPC Domain-ID
vPC Domain 10
vPC Domain 20
• The vPC peer devices use the vPC domain ID to
automatically assign a unique vPC system MAC
address
• You MUST use unique Domain id’s for all vPC
pairs defined in a contiguous layer 2 domain
! Configure the vPC Domain ID – It should be unique within the layer 2
domain
NX-1(config)# vpc domain 20
! Check the vPC system MAC address
NX-1# show vpc role
<snip>
vPC system-mac : 00:23:04:ee:be:14
S1 S2
S3 S4
S5
![Page 23: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/23.jpg)
vPC Configuration Best PracticesvPC Peer-Link
• vPC Peer-link should be a point-to-point connection
• Peer-Link member ports can be 10/40/100GE interfaces
• Peer-Link bandwidth should be designed as per the vPC
• vPC imposes the rule that peer-link should never be blocking
S1 S2
S3
S1 S2
S3
![Page 24: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/24.jpg)
vPC Configuration Best PracticesvPC Peer-Keepalive link
Preference Nexus 7X00 /
9500 series
Nexus 9300 /6000 /
5X00 / 3X00 series
1 Dedicated link(s)
(1GE/10GE LC)
mgmt0 interface
2 mgmt0 interface Dedicated link(s)
(1GE/10GE LC)
3 L3 infrastructure L3 infrastructure
Recommendations
(in order of
preference):
![Page 25: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/25.jpg)
vPC Configuration Best PracticesvPC Peer-Keepalive link – Dual Supervisors
Standby Management Interface
Active Management Interface
vPC1 vPC2
vPC_PL
Management Network
Management Switch
vPC_PKLvPC_PKL• When using dual supervisors and mgmt0 interfaces
to carry the vPC peer-keepalive, DO NOT connect
them back to back between the two switches
• Only one management port will be active a given point
in time and a supervisor switchover may break keep-
alive connectivity
• Use the management interface when you have an out-
of-band management network (management switch in
between)
For YourReference
![Page 26: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/26.jpg)
vPC Configuration Best PracticesSpanning Tree (STP)
• All switches in Layer 2 domain should run either Rapid-PVST+ or MST
• Do not disable spanning-tree protocol for any VLAN
• Always define the vPC domain as STP root for all VLAN in that domain
STP is running to manage
loops outside of vPC domain,
or before initial vPC
configuration !
S1 S2
S4S3
S5
![Page 27: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/27.jpg)
vPC Configuration Best PracticesvPC Peer-Gateway
• Allows a vPC switch to act as the active
gateway for packets addressed to the peer
router MAC
• Keeps forwarding of traffic local to the vPC
node and avoids use of the peer-link
• Allows Interoperability with features of some
NAS or load-balancer devices
N7k(config-vpc-domain)# peer-gateway
S1 S2
S4S3
![Page 28: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/28.jpg)
vPC Configuration Best PracticesvPC Peer-switch
Without Peer-switch
• STP for vPCs controlled by vPC primary.
• vPC primary send BPDU’s on STP designated ports
• vPC secondary device proxies BPDU’s to primary
With Peer-switch
• Peer-Switch makes the vPC peer devices to appear as a single STP root
• BPDUs processed by the logical STP root formed by the 2 vPC peer devices
N7k(config-vpc-domain)# peer-switch
Primary
vPC
Secondary
vPC
BPDUs
Primary
vPC
Secondary
vPC
![Page 29: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/29.jpg)
vPC Configuration Best PracticesPVLAN on vPC
• PVLAN configuration across both VPC switches should be identical
• PVLAN configuration not supported on Peer-Link
• Type-1 Compatibility Check
• Port mode is a type-1 check
• vPC leg brought down if PVLAN port mode different on vPC legs
• Type-2 Compatibility Check
• PVLAN will bring down mismatched tuple
S1 S2
vPC Primary vPC Secondary
P P
PVLAN-
PROMISC
(3500, 3501)
PVLAN-
PROMISC
(3500, 3501)
C
Community
VLAN
Note : This feature is currently not supported on N9X00
![Page 30: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/30.jpg)
Pvlan Isolated
trunk
vPC Configuration Best PracticesPVLAN VPC type 1 Consistency Check
S1 S2
vPC Primary vPC Secondary
P PS1 S2
vPC Primary vPC Secondary
I I
Pvlan
Promiscuous
trunk
Type 1
Consistency
Failure
S1 S2
vPC Primary vPC Secondary
TI
S3
S3S3
![Page 31: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/31.jpg)
vPC Configuration Best PracticesPVLAN VPC type 2 Consistency Check
S1 S2
vPC Primary vPC Secondary
P PS1 S2
vPC Primary vPC Secondary
I I
Type 2
Consistency
Failure
S1 S2
vPC Primary vPC Secondary
I
S3
S3S3
I
PVLAN-
PROMISC
(10, 201)
PVLAN-
PROMISC
(10, 201)
Secondary
Trunk (2,31)
(3,30), (4,100)
Secondary
Trunk (2,31)
(3,30), (4,100)
Secondary
Trunk (3,31)
(2,30), (4,100)
Secondary
Trunk (2,31)
(3,30), (4,100)
![Page 32: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/32.jpg)
vPC Configuration Best PracticesvPC auto-recovery
1. vPC peer-link down : S2 - secondary shuts all its vPC member ports
2. S1 down : vPC peer-keepalive link down : S2 receives no keepalives
3. After 3 keepalive timeouts, S2 changes role and brings up its vPCvPC Primary
vPC Secondary
P
S
P
Operational
Primary
S2S1
S3
P S
S1 S2
S3
S1 S2
S3
P S
![Page 33: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/33.jpg)
vPC Configuration Best PracticesvPC auto-recovery
Auto-recovery addresses two cases of single switch behavior
• Peer-link fails and after a while primary switch (or keepalive link) fails
• Both VPC peers are reloaded and only one comes back up
How it works
• If Peer-link is down on secondary switch, 3 consecutive missing peer-keepalives will
trigger auto-recovery
• After reload (role is ‘none established’) auto-recovery timer (240 sec) expires while
peer-link and peer-keepalive still down, autorecovery kicks in
• Switch assumes primary role
• VPCs are brought up bypassing consistency checks
For YourReference
Nexus(config)# vpc domain 1
Nexus(config-vpc-domain)# auto-recovery
![Page 34: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/34.jpg)
vPC Configuration Best PracticesWhy Object-Tracking ?
Primary Secondary
• Modules hosting peer-link and uplink fail on
the vPC primary
• Peer-Link is down and vPC Secondary shut all its vPC
• Auto-Recovery does not kick in as peer-keepalive link is active
• Traffic is black holed
S1 S2
S3
S5S4
![Page 35: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/35.jpg)
vPC Configuration Best PracticesObject-tracking
S1 S2
• Object Tracking triggered when the track object goes down
• vPC object tracking, tracks both peer-link and uplinks in a list of Boolean OR
• Traffic forwarded over the remaining vPC peer! Track the vpc peer link
track 1 interface port-channel11 line-protocol
! Track the uplinks
track 2 interface Ethernet1/1 line-protocol
track 3 interface Ethernet1/2 line-protocol
! Combine all tracked objects into one.
! “OR” means if ALL objects are down, this object will go down
track 10 list boolean OR
object 1
object 2
object 3
! If object 10 goes down on the primary vPC peer,
! system will switch over to other vPC peer and disable all local vPCs
vpc domain 1
track 10
• Suspends the vPCs on the impaired device
S4S5
S3
![Page 36: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/36.jpg)
Agenda
• vPC Design Best Practices
− Mixed Hardware across vPC Peers
− FHRP with vPC
− Hybrid topology (vPC and non-vPC)
− vPC and Network Services
− vPC Fex Supported Topologies
− Physical port vPC
− vPC as Data Center Interconnect (DCI)
− Dynamic Routing over VPC
− vPC and Multicast
![Page 37: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/37.jpg)
Design Best PracticesMixed Hardware across vPC Peers : Line Cards
Always use identical line cards on either sides of the peer link and VPC legs !
vPC Peer-link
S1 S2
vPC Primary vPC Secondary
M2M1
vPC
Examples
vPC Peer-link
S1
N7000
S2
N7700
vPC Primary vPC Secondary
F3
vPC
F3
F2E F2E
![Page 38: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/38.jpg)
Design Best Practices
X Y vPC
N9K-X9636PQ N9K-X9432PQ
N9K-X9564PX N9K-X9464PX
N9K-X9564TX N9K-X9464TX
N9K-X9536PQ N9K-X9736PQ
vPC Peer-link
S1
N9500S2
N9500
vPC Primary vPC Secondary
Y
vPC
X
X Y
Mixed Hardware across vPC Peers : Nexus 9500
![Page 39: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/39.jpg)
Design Best PracticesMixed Hardware across vPC Peers : Chassis & Supervisors
• N7000 and N7700 in same vPC Construct -Supported
• VDC type should match on both peer device
• vPC peers can have mixed SUP version* (SUP1, SUP2, SUP2E)
• N5500 and N5600 in same vPC Construct –Not Supported
*Recommended only for short period such as migration
N5500 N5600
vPC Primary vPC Secondary
S1
N7000
S2
N7700
vPC Primary vPC Secondary
S1 S2
![Page 40: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/40.jpg)
FHRP with vPCHSRP / VRRP/ GLBP Active/Active
FHRP
“Standby”:
Active for
shared L3 MAC
FHRP
“Active”:
Active for
shared L3 MAC
• FHRP in Active/Active mode with vPC
• No requirement for aggressive FHRP timers
• Best Practice : Use default FHRP timers
S1 S2
S3 S4
![Page 41: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/41.jpg)
P
L3L2
OSPF/EIGRP
Primary
vPC
Secondary
vPC
OSPF/EIGRP
VLAN 99
FHRP with vPCBackup Routing Path
• Point-to-point dynamic routing protocol adjacency between the vPC peers to establish a L3 backup path to the core through PL in case of uplinks failure
• Define SVIs associated with FHRP as routing passive-interfaces in order to avoid routing adjacencies over vPC peer-link
• A single point-to-point VLAN/SVI (aka transit vlan) will suffice to establish a L3 neighbor
• Alternatively, use an L3 point-to-point link between the vPC peers to establish a L3 backup path
Routing Protocol Peer
P
Use one transit vlan to establish L3 routing
backup path over the vPC peerlink in case L3
uplinks were to fail, all other SVIs can use
passive-interfaces
S2S1
S4S3
S5
P
P
P
![Page 42: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/42.jpg)
Hybrid topology (vPC and non-vPC)
S1 S2
S3 S4
vPC Primary vPC Secondary
vPC1
Bridge Priority
VLAN 1 4K
VLAN 2 8K
Bridge Priority
VLAN 1 8K
VLAN 2 4K
STP Root
VLAN 1STP Root
VLAN 2
STP Root
VLAN 1
VLAN 2
VLAN 1
(blocked)
VLAN 2
(blocked)
• Supports hybrid topology where vPC and non-vPC are connected to the same vPC domain
• Need additional configuration parameters : spanning-tree pseudo-information
• If previously configured global spanning tree parameters and subsequently configure spanning tree pseudo information parameters, then pseudo information parameters take precedence over the global parameters.
peer-switch
![Page 43: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/43.jpg)
Design Best PracticesASA Cluster
ASA Cluster Mode
• Use unique vPC for ASA Cluster Data Links to vPC domain
• Use vPC per ASA device for Cluster Control Link (CCL) to vPC domain
• Leverage peer-switch configuration
Cluster
Control Link
Cluster
Data Link
![Page 44: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/44.jpg)
Nexus 2000 (FEX) Straight-Through Deployment with VPC
• Port-channel connectivity from the server
• Two Nexus switches bundled into a vPC pair
• Suited for servers with Dual NIC and capable of running Port-Channel
Fex 100
Fabric Links
Fex 101
VPC
HIF HIF
S1 S2
![Page 45: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/45.jpg)
Nexus 2000 (FEX)Active-Active Deployment with VPC
Fabric Links
HIFHIF
Fex 100Fex 101
S1 S2
• Fabric Extender connected to two
Nexus 5X00 / 6000
• Suited for servers with Single NIC or
Dual NIC not having port-channel
capability.
• Scale implications of less FEX per
system and less VPC
Note : • This design is currently not supported on Nexus 9X00
• Nexus 7X00 will support this from release 7.2
![Page 46: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/46.jpg)
Nexus 2000 (FEX) - Enhanced VPC
Fabric Links
Fex 100 Fex 101
S1 S2• Port-channel connectivity to dual-
homed FEXs
• From the server perspective a single
access switch with port-channel
support – each line card supported by
redundant supervisors
• Ideal design for a combination of
single NIC and Dual NIC servers with
port-channel capability
• Scale implications of less FEX per
system and less VPC
Note :
This design is currently not supported on N7000 / N7700 and N9X00
HIFHIF
![Page 47: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/47.jpg)
Physical Port vPC
• vPC configuration on a physical Layer 2 port as opposed to a port-channel
• Front panel ports and FEX ports connected to F2/F2e/F3 only
• Improves scaling as separate PC interface not created for single-link VPC leg
• Key benefit: more than 1000 host facing VPCs with FEX
vPC domain
FEX101 FEX102
e101/1/1 e102/1/1Po1
Port-channel vPC
VPC1 VPC1Po1
vPC domain
FEX101 FEX102
e101/1/1 e102/1/1
Physical port vPC
VPC1 VPC1
Q2 CY15
NX-OS 7.2
(for FEX, scale and
F3 support)
interface e101/1/1
switchport
vpc 1
lacp mode active
![Page 48: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/48.jpg)
Long Distance
Dark Fiber
DC 1 DC 2C
OR
EA
GG
R
AC
CE
SS
Server Cluster
CO
RE
AG
GR
AC
CE
SS
Server Cluster
vPC domain 10 vPC domain 20
vPC domain 21vPC domain 11
vPC - Data Center Interconnect(DCI)
Rootguard
B
F
N
E
BPDUguard
BPDUfilter
Network port
Edge or portfast
- Normal port type
R
802.1AE (Optional)
E
- -
-
- -
E
E
E
E
F
F
F
F-
-
- -
-
--
B
N N
N
NN
N
R
R
-
RRRR
RR
NN
B
-
E
![Page 49: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/49.jpg)
Design Best PracticesvPC as Data Center Interconnect (DCI)
PROS
• vPC is easy to configure and it provides robust and resilient interconnect solution
CONS
• Maximum of only two Data Centers can be interconnected
• Layer 3 peering between Data Centers cannot be done through vPC and separate links are required
![Page 50: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/50.jpg)
Design Best PracticesvPC -Data Center Interconnect (DCI)
• vPC Domain id for vPC layers should be UNIQUE
• BPDU Filter on the edge devices to avoid BPDU propagation
• STP Edge Mode to provide fast Failover times
• No Loop must exist outside the vPC domain
• No L3 peering between Nexus 7000 devices (i.e. pure layer 2)
![Page 51: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/51.jpg)
Dynamic routing over vPC ?
![Page 52: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/52.jpg)
Dynamic routing over vPCUse Case 1 : Firewall at Aggregation layer
L3 Cloud
S1S2
FW-A FW-B
Dynamic Peering Relationship
• Peering Firewalls in routed mode over vPC
• Firewalls may be in active-standby mode
• Static routing / L3 P2P links NOT required
• External and internal traffic traverse same
port channel to firewall.
![Page 53: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/53.jpg)
Dynamic routing over vPCUse Case 2 : Remote Orphan Site Peering in DCI Deployment
• vPC as Data Center Interconnect (DCI)
• Each Switch has routing adjacency withboth vPC device in other DC
• Each DC connected to a remote site byorphan port
• Remote sites forms routing adjacencywith both peers of its directly connectedDC
Remote Site 1 Remote Site 2
S1 S2
S3 S4
![Page 54: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/54.jpg)
Dynamic Routing over vPCNew Supported Designs
![Page 55: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/55.jpg)
Dynamic routing over vPCSupported Designs
P
PP
PP
PP
Note : Supported only in Nexus 7X00 on F3 and F2E Line Cards starting from release 7.2.
Supported on Nexus 9X00 in ACI mode
Currently not supported on Nexus 5X00, Nexus 3X00, Nexus 9X00 (standalone mode), Nexus 7000 M-series Line card
Layer 3 services devices with vPC Layer 3 over DCI - vPC
![Page 56: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/56.jpg)
Dynamic routing over vPCSupported Designs
P
PP
P
PP
STP inter-connection using a vPC VLAN Orphan device with vPC peers over vPC VLAN
Note : Supported only in Nexus 7X00 on F3 and F2E Line Cards starting from release 7.2.
Supported on Nexus 9X00 in ACI mode
Currently not supported on Nexus 5X00, Nexus 3X00, Nexus 9X00 (standalone mode), Nexus 7000 M-series Line card
![Page 57: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/57.jpg)
B
SVI 1
IP Y
VIP A
SVI 1
IP Z
VIP A
S1 S2
A
Router
Dynamic Routing over vPC
• Don’t attach routers to VPC domain via L2 port-channel
• Common workarounds:
• Individual L3 links for routed traffic
• Static route to FHRP VIP
Router
L3 ECMP
Router
SVI 1
IP Y
VIP A
SVI 1
IP Z
VIP A
SVI 1
IP Y
VIP A
SVI 1
IP Z
VIP A
SVI 2
IP X
SVI 2
IP X
SVI 2
IP X
Static Route to VIP A
S1 S1 S2S2
Devices without L3 over vPC support
![Page 58: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/58.jpg)
Design Best PracticesvPC and Multicast
Source • vPC supports PIM-SM only
• vPC uses CFS to sync IGMP state
• Sources in vPC domain
− both vPC peers are forwarders
− Duplicates avoided via vPC loop-avoidance logic
• Sources in Layer 3 cloud
− Active forwarder elected on unicast metric
− vPC Primary elected active forwarder in case metric
are equal
Receivers
S2S1
Source
![Page 59: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/59.jpg)
vPC : Get it Right the very First time
![Page 60: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/60.jpg)
Agenda
• vPC Operations and Upgrade− vPC Self Isolation
− vPC Shutdown
− Graceful Insertion and Removal
− ISSU / ISSD with vPC
![Page 61: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/61.jpg)
VPC Self-isolation Automatically triggered isolation
Example Presented: All Line Cards Fail
Current Impact Self-isolation Behavior (7.2)
• When this failure
happens on primary,
peer-link is brought down.
• This causes the
secondary brings down
all legs.
• Traffic is completely
blocked.
When this failure happens:
• Physically bring down peer-link
• Physically bring down all vPC legs
• Send self-isolation through peer-keep-
alive
Peer switch:
• Receive self-isolation from the peer
through peer-keep-alive
• Change role to Primary
• Bring up all down vPC legs
PKA
Vlan 1-100
Vlan 1-100Vlan 1-100
Primary Secondary
Current
PKA
Vlan 1-100
PrimarySecondary
Self-isolation
NOTE: Available in NX-OS 7.2, 5k/6k/7k
![Page 62: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/62.jpg)
VPC Self-isolation Automatically triggered isolation
No up vlans on peer-link: This case address the issue that no vlansare up on the peer-link while the port channel is physically up (i.e., peer-link is up with no vlans). For example: vlan misconfiguration, hardware programming failure.
Current Behavior Self-isolation Behavior
• The up VLAN on vPC legs
are the results of configured
VLAN intersected with the
UP VLAN on peer-link (i.e.,
three-way intersection).
• All VLAN down on vPC
domain.
• This completely blocks the
traffic
Isolated switch:
• Physically bring down peer-link
• Physically bring down all vPC legs
• Send “self-isolation” through peer-keep-
alive
Peer switch:
• Receive self-isolation from the peer
through peer-keep-alive
• Change role to Primary
• Bring up all down vPC legs
PKA
No vlan
Vlan 1-100 Vlan 1-100
Current
PKA
Vlan 1-100
Self-
isolation
NOTE: Available in NX-OS 7.2, 5k/6k/7k
![Page 63: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/63.jpg)
vPC Shutdown
vPC Configuration Best Practices
• Isolates a switch from the vPC complex
• Isolated switch can be debugged, reloaded, or
even removed physically, without affecting the
vPC traffic going through the non-isolated switch
Primary Secondary
vPC
Note : This Feature is currently not supported on Nexus 3X00 and 9X00 series
S1 S2
S3
switch# configure terminal
switch(config)# vpc domain 100
switch(config-vpc)# shutdown
![Page 64: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/64.jpg)
vPC vPC
system mode maintenance
One command!Pre-change System Snapshot
Change window begins.
Graceful Insertion and Removal
![Page 65: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/65.jpg)
vPC vPC
One command!Pre/Post-change Snapshot Comparison
Change window complete.
system mode normal
Graceful Insertion and Removal
![Page 66: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/66.jpg)
Graceful Insertion and Removal
• Flexible framework providing a comprehensive, systemic method to isolate a node.
• Configuration profile foundation in NX-OS
• Initial support for:• vPC/vPC+• ISIS• OSPF• EIGRP• BGP• Interface
• Per VDC on Nexus 7x00
Platform Release
Nexus 5x00/6000 NX-OS 7.1
Nexus 7x00 NX-OS 7.2
Nexus 9000 NX-OS 7.X
![Page 67: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/67.jpg)
ISSU / ISSD with vPC
• ISSU is the recommended system upgrade in a multi-device vPC environment
• vPC system can be independently upgraded with no disruption to traffic
• Upgrade is serialized and must be run one peer at a time (config lock will prevent synchronous upgrades)
• Configuration is locked on “other” vPC peer during ISSU
• Similar process of downgrades (ISSD)
• Check ISSU / ISSD compatibility matrix & ensure ISSU is supported from current to target release
5.2(x) / 6.2(x)
![Page 68: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/68.jpg)
Agenda
• vPC with Fabric Technologies− vPC with Fabricpath (vPC+)
− vPC with FCOE
− vPC with VXLAN
− vPC with ACI
![Page 69: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/69.jpg)
FabricPath: an Ethernet FabricShipping on Nexus 7x00, Nexus 600x and Nexus 5x00
N7K(config)# interface ethernet 1/1
N7K(config-if)# switchport mode fabricpath
• Eliminates Spanning tree limitations
• High resiliency, fast network re-convergence
• Any VLAN, Anywhere in the Fabric
• Connect a group of switches using an arbitrary topology
• With a simple CLI, aggregate them into a Fabric
FabricPath
![Page 70: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/70.jpg)
• Physical architecture of vPC and vPC+ is the same from the access edge
• Functionality/Concepts of vPC and vPC+ are the same
• Key differences are addition of Virtual Switch ID and Peer Link is a FP Core Port
• vPC+ is not supported on Nexus 9X00 & Nexus 3X00 Series
Architecture of vPC and FabricPath with vPC+
VPC vs VPC+
FP VLAN’s
vPC+vPC
FP Port
FP
CE Port
CE VLAN’s
CE
![Page 71: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/71.jpg)
Dynamic Routing over vPC+
P Routing Protocol Peer
N55xx, N56xx, N6000
Router/ Firewall
Dynamic Peering Relationship
• Layer 3 devices can form routing adjacencies with both the vPC+ peers over vPC
• The peer link ports and VLAN are configured in FabricPath mode.
• N55xx, N56xx, N6000 support this design with IPv4/IPv6 unicast and PIM-SM multicast
• This design is not supported on N7X00
Fabricpath Link
vPC
P P
P
Fabricpath Core
![Page 72: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/72.jpg)
vPC with FCoEUnified Fabric Design
• vPC with FCoE is ONLY supported between hosts and N5K/N6K or N5K/N6K & N2232 pairs.
• Must follow specific rules:
• A ‘vfc’ interface can only be associated with a single-port port-channel.
• While the port-channel configurations are the same on both switches, the FCoE VLANs are different.
• FCoE VLANs are ‘not’ carried on the vPC peer-link (automatically pruned):
• FCoE and FIP ethertypes are ‘not’ forwarded over the vPC peer link.
• vPC carrying FCoE between two FCF’s is NOT supported.
• Best Practice: Use static port channel rather than LACP with vPC and boot from SAN.[If NX-OS is prior to 5.1(3)N1(1)]
VLAN 10,30
VLAN 10,20
STP Edge Trunk
VLAN 10 ONLY HERE!
Fabric A Fabric BLAN Fabric
Nexus 5000
FCF-A
Nexus 5000
FCF-B
vPC contains only 2 X 10GE
links – one to each Nexus 5X00
![Page 73: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/73.jpg)
Problems being addressed:
• VLAN scale – VXLAN extends the L2 segment ID field to 24-bits, potentially allowing for up to 16 million unique L2 segments over the same network
• Layer 2 segment elasticity over Layer 3 boundary – VXLAN encapsulates L2 frame in IP-UDP header
High Level Technology Overview:
• MAC-in-UDP encapsulation.
• Leverages multicast in the transport network to simulate flooding behavior for broadcast, unknown unicast and multicast in the same segment
• Leverage ECMP to achieve optimal path usage over the transport network
Why VXLAN ?
![Page 74: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/74.jpg)
VXLAN Packet Format
FCSOuter
Mac Header
Outer
IP HeaderUDP Header VXLAN
HeaderOriginal L2 Frame FCS
Dst.
MA
C A
ddr.
Src
.
MA
C A
ddr.
VLA
N T
ype
0x8100
VLA
N I
D
Ta
g
Eth
er
Type
0x0800
IP H
eader
Mis
cD
ata
Pro
tocol
0x11
Header
Checksum
Oute
r
Src
. IP
Oute
r
Dst.
IP
UD
P
Src
. P
ort
VX
LA
N P
ort
UD
P L
ength
Checksum
0x0000
VX
LA
N
RR
RR
1R
RR
Reserv
ed
VN
ID
Reserv
ed
14 Bytes
(4 bytes optional) 20 Bytes 8 Bytes 8 Bytes
48 48 16 16 16 72 8 16 32 32 16 16 16 16 8 24 24 8
• VXLAN is a Layer 2 overlay scheme over a Layer 3 network.
• VXLAN uses Ethernet in UDP encapsulation
• VXLAN uses a 24-bit VXLAN Segment ID (VNI) to identify Layer-2 segments
For YourReference
![Page 75: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/75.jpg)
VXLAN TerminologyVTEP – Virtual Tunnel End Point
• VXLAN terminates its tunnels on VTEPs (Virtual Tunnel End Point).
• VTEP has two interfaces :
1. Bridging functionality for local hosts
2. IP identification in the core network for VXLAN encapsulation / de-encapsulation.
Local LAN Segment
IP Interface
End SystemEnd System
VTEP
Transport IP Network
Local LAN Segment
IP Interface
End SystemEnd System
VTEP
![Page 76: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/76.jpg)
vPC VTEP
VXLAN
VLAN
vPC VTEP vPC VTEP
• When vPC is enabled an ‘anycast’ VTEP
address is programmed on both vPC
peers
• Multicast topology prevents BUM traffic
being sent to the same IP address across
the L3 network (prevents duplication of
flooded packets)
• vPC peer-gateway feature must be
enabled on both peers
• VXLAN header is ‘not’ carried on the vPC
Peer link
![Page 77: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/77.jpg)
VXLAN & VPCVPC Configuration
vtep
1
vtep
2
vtep
3
vtep
4
H110.10.10.10
VLAN 10(vpc)
H210.10.10.20
VLAN 10(vpc)
VTEP1
vlan 10
vn-segment 10000
interface loopback 0
ip address <VTEP individual IP – orphan)
ip address <VTEP anycast IP – per VPC domain> secondary
!
interface nve1
source-interface loopback0
member vni 10000 mcast-group 235.1.1.1
Map VNI to VLAN
VXLAN Tunnel Interface
Source Interfaceindividual IP is used for single attached Hostsanycast IP is used for VPC attached Hosts
VTEP2
vlan 10
vn-segment 10000
interface loopback 0
ip address <VTEP individual IP - orphan>
ip address <VTEP anycast IP – per VPC domain> secondary
!
interface nve1
source-interface loopback0
member vni 10000 mcast-group 235.1.1.1
For YourReference
![Page 78: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/78.jpg)
VXLAN & VPCVPC Configuration
vtep
1
vtep
2
vtep
3
vtep
4
H110.10.10.10
VLAN 10(vpc)
H210.10.10.20
VLAN 10(vpc)
VTEP1
vlan 10
vn-segment 10000
interface loopback 0
ip address 1.1.1.1/32
ip address 1.1.1.201/32 secondary
!
Interface nve1
source-interface loopback0
member vni 10000 mcast-group 235.1.1.1
VTEP3
vlan 10
vn-segment 10000
interface loopback 0
ip address 1.1.1.3/32
ip address 1.1.1.202/32 secondary
!
Interface nve1
source-interface loopback0
member vni 10000 mcast-group 235.1.1.1
VTEP2
vlan 10
vn-segment 10000
interface loopback 0
ip address 1.1.1.2/32
ip address 1.1.1.201/32 secondary
!
Interface nve1
source-interface loopback0
member vni 10000 mcast-group 235.1.1.1
VTEP4
vlan 10
vn-segment 10000
interface loopback 0
ip address 1.1.1.4/32
ip address 1.1.1.202/32 secondary
!
Interface nve1
source-interface loopback0
member vni 10000 mcast-group 235.1.1.1
For YourReference
![Page 79: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/79.jpg)
VXLAN & VPCDual attached Host to dual attached Host (Layer-2)
vtep
1
vtep
2
vtep
3
vtep
4
H110.10.10.10
VLAN 10(vpc)
H210.10.10.20
VLAN 10(vpc)
• Host 1 (H1) and Host 2 (H2) are dual
connected to a VPC domain
• As H1 is behind a VPC interface, the
anycast VTEP IP is the source for
the the VXLAN encapsulation
• As H2 is behind a VPC interface, the
anycast VTEP IP is the target
![Page 80: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/80.jpg)
Nexus 9000 + APIC = ACI
APICAPIC
APIC
![Page 81: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/81.jpg)
External
Network
App DBWeb
QoS
Filter
QoS
Service
QoS
Filter
ACI uses a policy based approach that focuses on the application.
![Page 82: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/82.jpg)
vPC and ACIACI fabric utilised for control-plane vPC
Domains
vPC peers
ACI
fabric
• No dedicated peer-link between vPC peers:
Fabric itself serves as the MCT
vPC vPC• CFS (Cisco Fabric Services) is replaced by Zero Message Queue (ZMQ)
vtep
1
vtep
2
vtep
3
• No out-of-band mechanism to detect peer liveliness:
Due to rich fabric-connectivity (leaf-spine), it is very unlikely that peers will have no active path between them
• As ACI fabric is VXLAN-based, an anycast VTEP is shared by both leaf switches in a vPC domain
![Page 83: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/83.jpg)
Agenda
• Scalability
![Page 84: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/84.jpg)
vPC Scalability
For Latest Scalability numbers please refer to the scalability limits pages for the platform
Nexus 7X00
http://www.cisco.com/en/US/docs/switches/datacenter/sw/verified_scalability/b_Cisco_Nexus_7000_Series_NX-OS_Verified_Scalability_Guide.html
Nexus 5X00http://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus5600/sw/verified_scalability/701N11/b_N5600_Verified_Scalability_701N11/b_N6000_Verified_
Scalability_700N11_chapter_01.html
Nexus 600X http://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus6000/sw/verified_scalability/602N21/b_N6000_Verified_Scalability_602N21/b_N6000_Verified_
Scalability_602N12_chapter_01.html
Nexus 9X00
http://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus9000/sw/7-x/scalability/guide_703I12/b_Cisco_Nexus_9000_Series_NX-
OS_Verified_Scalability_Guide_703I12.html
Nexus 3X00 http://www.cisco.com/en/US/docs/switches/datacenter/nexus3000/sw/configuration_limits/503_u5_1/b_Nexus3k_Verified_Scalability_503U51.html
84
![Page 85: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/85.jpg)
Agenda
• Reference Material
![Page 86: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/86.jpg)
Reference Material
• vPC Best Practices Design Guide: http://www.cisco.com/c/dam/en/us/td/docs/switches/datacenter/sw/design/vpc_design/vpc_best_practices_design_guid
e.pdf
• vPC design guides:
http://www.cisco.com/en/US/partner/products/ps9670/products_implementation_design_guides_list.html
• vPC and VSS Interoperability white Paper:
http://www.cisco.com/en/US/prod/collateral/switches/ps5718/ps708/white_paper_c11_589890.html
• VXLAN Overview :
http://www.cisco.com/c/en/us/products/collateral/switches/nexus-9000-series-switches/white-paper-c11-729383.html
• Fabrcipath whitepaper :
http://www.cisco.com/c/en/us/products/collateral/switches/nexus-7000-series-switches/white_paper_c11-687554.html
ACI Overview
http://www.cisco.com/c/en/us/products/collateral/cloud-systems-management/aci-fabric-controller/white-paper-c11-729587.html
For YourReference
![Page 87: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/87.jpg)
vPC in 2015
vPC Benefits
Fabricpath
VXLAN
Key Take-Aways
• No Blocked Ports
• High availability
• Fast Convergence
• Eliminates Spanning-Tree *
• High resiliency
• vPC+ for legacy switches,
servers, hosts
• L2 segment scalability
• VTEP redundancy with
vPC
VXLAN, ACI, Fabricpath
• Policy Based
• Fabric for vPC control
plane
ACI
• Unified Fabric for LAN &
SAN
FCoE
![Page 88: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/88.jpg)
Participate in the “My Favorite Speaker” Contest
• Promote your favorite speaker through Twitter and you could win $200 of Cisco Press products (@CiscoPress)
• Send a tweet and include
• Your favorite speaker’s Twitter handle
• Two hashtags: #CLUS #MyFavoriteSpeaker
• You can submit an entry for more than one of your “favorite” speakers
• Don’t forget to follow @CiscoLive and @CiscoPress
• View the official rules at http://bit.ly/CLUSwin
Promote Your Favorite Speaker and You Could Be a Winner
![Page 89: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/89.jpg)
Complete Your Online Session Evaluation
Don’t forget: Cisco Live sessions will be available for viewing on-demand after the event at CiscoLive.com/Online
• Give us your feedback to be entered into a Daily Survey Drawing. A daily winner will receive a $750 Amazon gift card.
• Complete your session surveys though the Cisco Live mobile app or your computer on Cisco Live Connect.
![Page 90: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/90.jpg)
Continue Your Education
• Demos in the Cisco campus
• Walk-in Self-Paced Labs
• Table Topics
• Meet the Engineer 1:1 meetings
• Related sessions
![Page 91: Everyone in this room is a GENIUS - Amazon Web … · 2013-2014 MPLS, OTV, LISP VXLAN MPLS, OTV, LISP 2014-2015 ACI 2010 ... After 3 keepalive timeouts, S2 changes role and brings](https://reader031.vdocuments.us/reader031/viewer/2022022613/5b9d5f2409d3f275078c3b90/html5/thumbnails/91.jpg)
Thank you