nexus 7000 series innovations: m3 module, dci, scale
TRANSCRIPT
Data Center Switching Product Management TeamDecember 2016
Cisco Nexus 7000 Series SwitchesDesigning Data Center Interconnect
2© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Agenda
Nexus 7000 Product Update
Nexus 7000 DCI Technologies
3© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Switching Infrastructure for Today’s Data Centers
Investment Protection
Architectural Flexibility
Operational Simplicity
Open and Programmable
Secure, Scalable, and Resilient
One Operating System Cisco NX-OS
Cisco Nexus 9000 Series
Cisco Nexus 7000 Series
Cisco Nexus 3000 Series
Cisco Nexus 2000 Series
Cisco Nexus 5000 Series
4© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Cisco Data Center Fabric Solutions
APPLICATION CENTRIC INFRASTRUCTURE PROGRAMMABLE FABRIC
Disaggregated approach based on Open standards
Support 3rd party SDN Controller
SDN across the entire Nexus Product line
Rich inter-domain support
Turnkey integrated solution with security, centralized management, compliance and scale
Automated application centric-policy model with embedded security
Broad and deep ecosystem
CONTROLLER
OPEN APIs
VXLAN with BGP EVPN
OPEN APIs
5© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Cisco Nexus 7700 Series Switch Family26
RU
14 R
U
9 R
U
Cisco Nexus® 770010-Slot
Cisco Nexus® 770018-Slot
Cisco Nexus® 77006-Slot
Environmental EFFICIENCY True front-to-back airflow
Smaller FOOTPRINTMore compact
Fabric BANDWIDTH1.32 Tbps
100G Density
40G Density
1G / 10G Density
192
384
768
96
192
384
48
96
192
12
24
48
3 R
U
Cisco Nexus® 77002-Slot
6© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
FabricPath
FCoEM1(2008)
80G/slot
Cisco Nexus 7000 Series Module Evolution
M2(2012)240G/slot
F1(2010)230G/slot
F2(2011)480G/slot
F3(2013)1.2T/slot
M3(2016)1.2T/slot
OTV
MPLS
LISP
EoMPLS/VPLS
Layer 3Layer 2
SampledNetFlow
FEX
Large Tables
Large Buffers
FullNetFlow
VXLANFSAOffload
L2-L2 GW
10G FSA 256-bitMACsec
40G / 100G
7© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
FabricPath
FCoE
Cisco Nexus 7000 Series Module Evolution
M2(2012)240G/slotF3(2013)
1.2T/slot
M3(2016)1.2T/slot
OTV
MPLS
LISP
EoMPLS/VPLS
Layer 3Layer 2
SampledNetFlow
FEX
Large Tables
Large Buffers
FullNetFlow
VXLANFSAOffload
L2-L2 GW
10G FSA 256-bitMACsec
40G / 100G
8© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
DC CORE | DC INTERCONNECT
24-Port 40GE
48-Port 1/10GE
Enhanced Scale | Enhanced Security | Deployment Flexibility | Investment Protection
Cisco Nexus 7000 M3 Series Modules
12-Port 100GE
48-Port 1/10GE
24-Port 40GE
9© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
256-bit AES MACsec#
48 1/10 GE Ports (SFP+) 24 40 GE Ports (QSFP) 12 100 GE Ports (QSFP28)
On all ports/speeds
Multi-Core Fabric Services Accelerator (FSA)Enhanced Performance for BFD, Netflow, and Other Distributed Fabric Services
New Cisco M3 ASIC VXLAN, OTV, LISP*, MPLS FabricPath*, Classic L2/L3 Cisco TrustSec – SGT, SXP, SGACLs
Larger Tables 2M* FIB Entries 384K* MAC Entries 128K ACL/QOS Entries
Advanced Parser Layer 2 to Layer 2 Gateway* GTP Hashing
Deeper Buffers 31.25MB per 10GE Port 125MB per 40GE Port 350MB per 100GE Port
* Hardware Capability. Software support may be available in the future. See NX-OS Software Release Notes.
Cisco Nexus 7000 M3 Series Modules
# MKA support may be available in a later release.
Compatible with Supervisor 2/2E and Fabric 2 Modules | VDC Interoperability with F3 or M2 I/O Modules
10© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
QSFP28 Optics for 100G connectivity
QSFP Optics for 40G connectivity
Approximately 6µsec cross-fabric latency
Nexus 7700 M3 12-Port 100G I/O Module
11© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Introducing the M3-Series into New or Existing Chassis
M3 and F3 or M2 modules interoperate at the Lowest Common Feature Set
(F3 + M3) VDC
Full Layer2 and Layer3 Interoperability
No L3 Proxy
M3
F3
(M3 only) VDC
Full Layer2 and Layer3 Interoperability
M3 – 48p 10G
M3 – 24p 40G
12© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
VDC Interface Allocation – M3-Series Modules
M3 40G6-port
port-group
VDC 1 VDC 2 VDC 3 VDC 4
M3 10G24-port
port-group
Port-group size varies depending on I/O module typeVDC Allocation on port-group boundaries – Aligns ASIC resources to VDCs
M3 100G2-port
port-group
13© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
4x10G Port Breakout Capability
Seamless 10G aggregation into dense 40G/100G portsBreakout per port not per line card | No need to reload
• Direct-attach active/passive copper breakout cables• Direct-attach active optical breakout cables• Fiber breakout cables (not included with optics transceivers)
Nexus 7000 24-Port 40GE
M3 Series I/O Module
Nexus 7000 12-Port 40GE
F3 Series I/O Module
Nexus 7700 24-Port 40GE
M3 Series I/O Module
Nexus 7700 24-Port 40GE
F3 Series I/O Module
14© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Nexus 7000 Series – Designed for DC & Campus Core
VPC BASED DESIGN
Classic STP Limitation 50% of all Links not utilized Complex to Harden
No STP Blocked Ports Full Links Utilization Faster Convergence
Simple to Configure Higher Fabric Bandwidth Consistent Latency
SpineScales to provide fabric bandwidth
LeafScales to provide access port density
Spine
Leaf
Horizontal Scale Out
FABRIC BASED DESIGNSTP BASED DESIGN
Workload Mobility | Application Communication | Port Density | Bandwidth
15© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
DCNM 10
DCNM 10 is a comprehensive toolbox for automated cloud-scale deployments
TOPOLOGY DISPLAYS OVERLAY, UNDERLAY and STORAGE NETWORKS – VXLAN ENABLED
OVERVIEWAUTOMATED LAN, SAN, and PROGRAMMABLE FABRIC MANAGER
POAP and AUTO-PROVISIONING FUNCTIONS FOR OVERLAY and UNDERLAY NETWORKS
MULTI-SITE, MULTI-TENANT, MULTI-FABRIC TURNKEY INFRASTRUCTURE
SUPPORTS ALL NEXUS and MDS SWITCHES
ENTERPRISE SCALE 500-1000 DEVICES
IMAGE, CONFIGURATION, and PATCH MANAGEMENT
INTEGRATES WITH VMWARE, OPENSTACK, REST
16© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Data Center Interconnect Technologies
17© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Nexus 7000 Series – Designed for DCI
WAN
L3 INTERCONNECT: IP VRF-LITE, MPLS, LISP
HITLESS ISSU, STATEFUL PROCESS RESTART, GRACEFUL INSERT & REMOVAL
L2 INTERCONNECT: FABRICPATH, VPC, OTV, LISP, VPLS, VXLAN
SOLUTION
BENEFITSLEVERAGE PROVEN & MATURE DCI TECHNOLOGIES AND IMPLEMENTATIONS
CONTROLLER
VXLAN with BGP EVPN
PROGRAMMABLE FABRIC
STP [OR] VPC BASED
STP, vPC, FabricPath
ACI
18© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Challenges in Traditional Layer 2 VPNsFlooding Behavior
- Unknown Unicast for MAC propagation- Unicast Flooding reaches all sites
Pseudo-wire Maintenance
- Full mesh of Pseudo-wire is complex- Head-End replication is a common problem
Multi-Homing
- Requires additional Protocols & extends STP- Malfunctions impacts multiple sites
19© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
EoMPLS
VPLSDark Fiber
Data Center Interconnect – Traditional Layer 2 Extensions
20© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
EoMPLS
VPLSDark Fiber
Data Center Interconnect – Traditional Layer 2 Extensions
• vPC or FabricPath• Applies easily for dual site interconnection• Over dark fiber or protected D-WDM• Easy crypto using end-to-end 802.1AE
• OTV – Overlay Transport Virtualization• MAC in IP
• EoMPLS & VPLS & A-VPLS & H-VPLS• PE style• Multi-tenants• Most deployed today
Ethernet
MPLS
IP
21© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Use Case: MACsec for Secure DCIsSingle Access dark Fiber Connectivity
Datacenter 1 Datacenter 2
Nexus 7000 Nexus 7000
Dual Access with dark Fiber ConnectivityDatacenter 1 Datacenter 2
Nexus 7000 Nexus 7000
Nexus 7000 Nexus 7000
VPC
VPC
Datacenter 1 Datacenter 2Nexus 7000 Nexus 7000
Nexus 7000 Nexus 7000
MPLS Core
Nexus 7000s as Bulk Encrypters for Self managed MPLS DCI Cores
22© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
OTV Enhancements
Loopback Address as Join Interface
50% more MAC Addresses per Site
50% more MAC Addresses across all Sites
* Hardware Capability. Software support may be available in the future. See NX-OS Software Release Notes.
23© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Interconnecting Fabrics with Nexus 7000 Switches and F3/M3 Series Cards
FabricPath VXLAN EVPN
EthernetFabricPath to VLAN
HandoffVLAN to VXLAN
Handoff
Interconnecting fabrics using Nexus 7000 Switches and F3/M3 cards requires two VDCs
24© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Loopback Address as OTV Join Interface - Benefits
Enables the use of multiple uplinks & ECMP in the core for better resiliency and traffic depolarization
DC EastDC West
Core
25© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Challenges in Traditional Layer 2 VPNsSolved by OTV
Flooding Behavior
- Unknown Unicast for MAC propagation- Unicast Flooding reaches all sites
Pseudo-wire Maintenance
- Full mesh of Pseudo-wire is complex- Head-End replication is a common problem
Multi-Homing
- Requires additional Protocols & extends STP- Malfunctions impacts multiple sites
✔ ✔ ✔Control-Plane Based
Learning Dynamic Encapsulation Native AutomatedMulti-Homing
26© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Yet Another Layer 2 Extension
Control-Plane
Multi-Homing
LoopPrevention
FaultContainment
TransportAgnostic
MulticastOptimization
Path Diversity
Multi-Site
GoodFabricPath ✖ ✔1 ✔✔ ✖ ✖ ✖ ✔ ✖VXLAN (Flood&Learn) ✖ ✔1 ✔2 ✖ ✔ ✔ ✔✔ ✖
BetterVXLAN BGP EVPN ✔ ✔1 ✔2 ✔✔ ✔✔ ✔ ✔✔ ✖
VPLS ✖ ✔1 ✔✔ ✖ ✖ ✖ ✔ ✔Best OTV ✔✔ ✔✔ ✔✔ ✔✔ ✔✔ ✔✔ ✔✔ ✔✔
1) Only with Multi-Chassis Link Aggregation (MC-LAG / VPC)
2) Limited Overlay Loop Prevention
27© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
MPLS
28© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
F3/M3 With MPLS L3 VPN• Highest density of 40G MPLS
• Available on M3 or M3 + F3 VDC
• VRF-Lite supported at FCS
L2L3 MPLS PE
MPLS P
MPLS Core
F3/M3 F3/M3
29© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
LISP*
* M3 Hardware Capability. Software support may be available in the future. See NX-OS Software Release Notes.
31© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
WAN/Campus
• Similar problem scale to DNS• Leverage demand based protocols
• A directory of hosts• Location as well as policy• Location != Routing
• Keep routing lean • Move all host state to LISP directory
• Minimize state on the routers and switches (cache on demand)
Handling host state at large scale with LISP
Branch/Closet
LISP XTR
DC 1 DC 2
LISP Host directory
32© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
WAN/Campus
• The Fabric can be based on any technology:• ACI, EVPN (PF), NSX
• LISP routers will take host routes received from the fabric and register them with the LISP directory
LISP Host Directory Services for any fabricBranch/Closet
LISP XTR
DC 1 DC 2
Local host routes
Local host routes
33© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
ACI WAN/DCI Handoff*
* M3 Hardware Capability. Software support may be available in the future. See NX-OS Software Release Notes.
34© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Nexus 7000
AUTO-PROVISION
OpFlex
TENANT SEGMENTATION
APIC
WAN/DCI OR
DC CORE
SOLUTION
SECURITY POLICY ENFORCEMENT AT ACI LEAF
PER-TENANT REACHABILITY WITH MP-BGP
GROUP POLICY AUTOMATION WITH OPFLEX
BENEFITSMULTI-DC WORKLOAD MOBILITY
LEVERAGE PROVEN/MATURE DCI TECHNOLOGIES AND IMPLEMENTATIONS
Nexus 7000 Series – ACI WAN/DCI Handoff
35© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
DCI Summary
* M3 Hardware Capability. Software support may be available in the future. See NX-OS Software Release Notes.
36© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Nexus 7000 Series – Designed for Interconnecting Fabrics
WAN
L3 INTERCONNECT: IP VRF-LITE, MPLS, LISP
HITLESS ISSU, STATEFUL PROCESS RESTART, GRACEFUL INSERT & REMOVAL
L2 INTERCONNECT: FABRICPATH, VPC, OTV, LISP, VPLS, VXLAN
SOLUTION
BENEFITSLEVERAGE PROVEN & MATURE DCI TECHNOLOGIES AND IMPLEMENTATIONS
CONTROLLER
VXLAN with BGP EVPN
PROGRAMMABLE FABRIC
STP [OR] VPC BASED
STP, vPC, FabricPath
ACI