vmworld 2015: vmware nsx deep dive
TRANSCRIPT
VMware NSX - Deep DiveJacob Rapp, VMware, Inc
NET5560
#NET5560
• This presentation may contain product features that are currently under development.
• This overview of new technology represents no commitment from VMware to deliver these features in any generally available product.
• Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.
• Technical feasibility and market demand will affect final delivery.
• Pricing and packaging for any new technologies or features discussed or presented have not been determined.
Disclaimer
CONFIDENTIAL 2
What You’ve Done with NSX
CONFIDENTIAL 3
NSX Customers700+
Production Deployments(adding 25-50 per quarter)
100+
Organizations invested US$1M+ in NSX
65+
What You’re Doing Next
EXPANDED SECURITY
New security partners, integrations, and projects and applications of NSX.
DEEPER INTEGRATION
New infrastructure and operations partners, integrations, and frameworks for IT organizations
√APPLICATION CONTINUITY
New functionality to scale deployments across vCenter instances, with the ability to:
• Pool resources from multiple data centers• Recover from disasters faster• Deploy a hybrid cloud architecture
• NSX 6.2 contains over 20 new features• Tested against over 1000 new scenarios
Session Objectives• Provide you with an in-depth understanding of the NSX architecture and components
• Understand how networking functions and services are implemented within the NSX platform
• Analyze key workflows for configuring virtual network & security services
• Provide pointers to reference design sessions and guides
CONFIDENTIAL 4
CONFIDENTIAL 5
ProvidesA Faithful Reproduction of Network & Security Services in Software
Management APIs, UI
Switching Routing
Firewalling
LoadBalancing
VPN
Connectivity to Physical Networks
Policies, Groups, Tags
Data Security Activity Monitoring
Physical Workloads
Security PoliciesSecurity Groups
Logical Switching, Routing, Firewall, Load Balancing
Web
App
Database
Web“Standard Web”
Firewall – allow inbound HTTP/S, allow outbound ANY IPS – prevent DOS attacks, enforce acceptable use
Database“Standard Database”
Firewall – allow inbound SQL Vulnerability Management –
Weekly Scan
App“Standard App”
Firewall – allow inbound TCP 8443, allow outbound SQL
VM VM
VM VM VM
VM
“Default” Firewall – Access shared services (DNS, AD) Anti-Virus – Scan Daily
Default
Creating Sophisticated Application Topologies
CONFIDENTIAL 6
Agenda
1 NSX Architecture and Components
2 Switching
3 Routing
4 Distributed Firewall & Micro-Segmentation
5 Services
6 Summary & Next Steps
CONFIDENTIAL 7
NSX Architecture and ComponentsCloud Consumption • Self Service Portal
• vCloud Automation Center, OpenStack, Custom
Data Plane
NSX Edge
ESXi Hypervisor Kernel Modules
Distributed Services
• High – Performance Data Plane• Scale-out Distributed Forwarding Model
Management PlaneNSX Manager
• Single configuration portal• REST API entry-point
Control Plane
NSX Controller• Manages Logical networks• Control-Plane Protocol• Separation of Control and Data Plane
FirewallDistributed Logical Router
LogicalSwitch
Logi
cal N
etw
ork
Phys
ical
Net
wor
k
…
…
HW VTEP
CONFIDENTIAL 8
NSX Data Plane ComponentsData Plane
Edge Clusters and HW VTEP (Physical-to-Virtual)
DFWVXLAN DLRSecurity
NSX Edge Service Gateways• VM form factor• Highly Available• Dynamic Routing:
• OSPF, IS-IS, BGP• L3-L7 Services:
• NAT, DHCP, Load Balancer, VPN, Firewall
• vSphere Distributed Switch • VMkernel Modules
• Logical Switching (VXLAN)• Distributed Logical Router• Distributed Firewall
ESXi
Hypervisor Kernel Modules (VIBs)
DistributedFirewall
Distributed Logical Router
LogicalSwitch
vSphere Components
DFWVXLAN DLRSecurity DFWVXLAN DLRSecurity
… …Compute Clusters
HW VTEP• ToR Switch
• Bandwidth and physical ports scale-out
• VLANs for Physical workloads local to a rack
CONFIDENTIAL 9
NSX Control Plane Components
• Properties– Virtual Form Factor (4 vCPU, 4GB RAM)– Data plane programming– Control plane Isolation
• Benefits– Scale Out – High Availability– VXLAN - no Multicast– ARP Suppression
NSX Controllers
vSphere Cluster vSphere HA DRS with Anti-affinity
VM ESXi VM VM
Host Agent
Data-Path Kernel Modules
10CONFIDENTIAL 10
Management Plane Components
• Runs as a Virtual Machine
• Provisioning and Management ofNetwork and Network services• VXLAN Preparation• Logical Network Consumption• Network Services Configuration
NSX Manager
NSX Manager1:1Management Plane
vRA/Openstack/Custom
vCenter
NSX REST APIsvSphere APIs
3rd Party Management Console
NSX Manager vSphere Plugin
Single Pane of Glass
Enabling Automation with NSX and vRANET5362
CONFIDENTIAL 11
OpenStack with NSX Deep DiveNET5836
12
NSX Component Interaction - Deployment and Configuration Deploy NSX Manager
12
53
Register with vCenter
Deploy NSX Controllers
4
Prepare HostsConfigure and deploy NSX
Edge Gateway(s) and network services
NSX Manager
vSphere Cluster 1 vSphere Cluster 2 vSphere Cluster N
NSX Controller
vCenter
NSX Edge Services GW
CONFIDENTIAL
CONFIDENTIAL 13
Management Plane ComponentsMulti-vCenter
Local VC Inventory Local VC Inventory Local VC Inventory
vCenter & NSX Manager A
Universal Object Configuration(NSX UI & API) Universal Configuration Synchronization
Universal Controller
Cluster
Primary Secondary
vCenter & NSX Manager B vCenter & NSX Manager H
Secondary
Universal LogicalSwitches
Universal Distributed Logical Router
UniversalDFW
Multi-VC Solutions with NSXNET5989
Deploying and Configuring VMware NSXDeploy VMware NSX
NSXEdge
NSXMgmt
Virtual Infrastructure
Deploy NSX Manager
Deploy NSX Controller Cluster
Component Deployment
Host Preparation
Logical Network Preparation
Preparation O
ne T
ime
ProgrammaticVirtual
Network Deployment
Logical Networks
+ + +Consumption
Rec
urrin
g
Deploy Logical Switches per tier
Create Bridged Network
Logical Network/Security Services
Deploy Distributed Logical Router or connect to existing
CONFIDENTIAL 14
Agenda
1 NSX Architecture and Components
2 Switching
3 Routing
4 Distributed Firewall & Micro-Segmentation
5 Services
6 Summary & Next Steps
CONFIDENTIAL 15
NSX Logical Switching
• Per Application/Multi-tenant segmentation • VM Mobility requires L2 everywhere• Large L2 Physical Network Sprawl – STP
Issues • HW Memory (MAC, FIB) Table Limits
• Scalable Multi-tenancy across data center• Enabling L2 over L3 Infrastructure • Overlay Based with VXLAN, etc.• Logical Switches span across Physical Hosts
and Network Switches
Challenges Benefits
VMw
are
NSX
Logical Switch 1 Logical Switch 2 Logical Switch 3
CONFIDENTIAL 16
Logical View: VMs in a Single Logical Switch
Web LS172.16.10.0/24
172.16.10.11 172.16.10.12 172.16.10.13
VM1 VM3VM2
172.16.20.12
VM5
172.16.20.11
VM4App LS172.16.20.0/24
CONFIDENTIAL 17
Physical View: VMs in a Single Logical Switch
VM1
vSphere Distributed Switch
VM2
Logical Switch 5001
VM3
Transport Subnet A 192.168.150.0/24
Physical Network
192.168.150.51 192.168.150.52 192.168.250.51
172.16.10.11 172.16.10.12 172.16.10.13
CONFIDENTIAL 18
19
IP Fabric
Host A Host B
vSphere Distributed Switch
Traffic Flow on a VXLAN Backed VDS
• In this setup, VM1 and VM2 are on different hosts but belong to the same logical switch
• When these VMs communicate, a VXLAN overlay is established between the two hosts
dvUplink-PG
Logical SW A
VM1
dvUplink-PG
dvPG-VTEP
VTEP
dvPG-VTEP
VTEP
VXLAN Overlay
Logical SW A
VM2
CONFIDENTIAL
Host BHost A
vSphere Distributed Switch
Traffic Flow on a VXLAN Backed VDS
• Assume VM1 sends some traffic to VM2:
dvUplink-PG
Logical SW A
VM1
dvUplink-PG
dvPG-VTEP
VTEP
dvPG-VTEP
VTEP
Logical SW A
VM2L2 frame L2 frame
IP FabricVXLAN Overlay
IP/UDP/VXLANL2 frame
VM1 sends L2 frame to local VTEP
1 VTEP adds VXLAN, UDP & IP headers2 Physical Transport
Network forwards as a regular IP packet
3 Destination Hypervisor VTEP de-encapsulates frame
4 L2 frame delivered to VM2
5
CONFIDENTIAL 20
NSX for vSphere VXLAN Replication Modes• NSX for vSphere provides three modes
of traffic replication (two which are Controller based, and onewhich is Data Plane based
• Unicast Mode– All replication occurs using unicast
• Hybrid Mode– Local replication offloaded to physical
network, while remote replication occurs via unicast
• Multicast Mode– Requires IGMP for a Layer 2 topology and
Multicast Routing for L3 topology
• All modes require an MTU of 1600 bytes
CONFIDENTIAL 21
Agenda
1 NSX Architecture and Components
2 Switching
3 Routing
4 Distributed Firewall & Micro-Segmentation
5 Services
6 Summary & Next Steps
CONFIDENTIAL 22
23
NSX Logical Routing Introduction
DLR Kernel Module
NSX Edge
ESXi
Hypervisor Kernel Modules (VIBs)
Distributed Logical Router
Distributed Logical RoutingOptimized for E-W Traffic Patterns
Centralized RoutingOptimized for N-S Routing
vSphere Host
LIF1 LIF2
Logical Routing Deep DiveNET5826
CONFIDENTIAL
NSX Routing: Distributed, Feature-Rich
• Physical Infrastructure Scale Challenges – Routing Scale
• VM Mobility is a challenge• Multi-Tenant Routing Complexity• Traffic hair-pins
Challenges
• Distributed Routing in Hypervisor• Dynamic, API based Configuration• Full featured – OSPF, BGP, IS-IS• Logical Router per Tenant• Routing Peering with Physical Switch
Benefits
SCALABLE ROUTING – Simplifying Multi-tenancy
L2
L2
Tenant A
Tenant B
L2
L2
L2 Tenant C
L2
L2
L2
CMP
CONFIDENTIAL 24
Logical View: VMs in a Single Logical Switch
VM1 VM3VM2
VM5VM4
Web LS172.16.10.0/24
172.16.10.11 172.16.10.12 172.16.10.13
172.16.20.12172.16.20.11App LS172.16.20.0/24
CONFIDENTIAL 25
Logical View: VMs with Distributed Routing
172.16.10.1
192.168.10.0/29192.168.10.1
Distributed Logical Router Service
VM1 VM3VM2
VM5VM4
Web LS172.16.10.0/24
172.16.10.11 172.16.10.12 172.16.10.13
172.16.20.12172.16.20.11App LS172.16.20.0/24
172.16.20.1
CONFIDENTIAL 26
Physical View: VMs in a Single Logical Switch
VM1
vSphere Distributed Switch
VM2
Logical Switch 5001
VM3
Physical Network
Transport Subnet A 192.168.150.0/24
192.168.150.51 192.168.150.52 192.168.250.51
172.16.10.11 172.16.10.12 172.16.10.13
CONFIDENTIAL 27
Physical View: Logical Routing
VM5VM1
vSphere Distributed Switch
VM2
Logical Switch 5001
VM3
Physical Network
VM4
Logical Switch 5002 Controller
Management Cluster
L3 Control Plane Programming
Data Plane
Transport Subnet A 192.168.150.0/24 Transport Subnet B 192.168.250.0/24
192.168.150.51 192.168.150.52 192.168.250.51
CONFIDENTIAL 28
29
NSX Logical Routing : Components Interaction
NSX Edge (Acting as next hop router)
172.16.10.0/24 172.16.20.0/24
DLR
192.168.10.1
192.168.10.2
External Network
192.168.10.3
DLR Control VM
DataPath
Control
Controller Cluster
Control
NSX Mgr
Dynamic routing protocol is configured on the logical router instance1
OSPF/BGP peering between the NSX Edge and logical router control VM3
Learnt routes from the NSX Edge are pushed to the Controller for distribution4
Controller sends the route updates to all ESXi hosts5
Routing kernel modules on the hosts handle the data path traffic6
1
34
5
6
Controller pushes new logical router Configuration including LIFs to ESXi hosts2
2
Peering
OSPF, BGP
Peering
OSPF, BGP
172.16.30.0/24
CONFIDENTIAL
Distributed East-West Routing Traffic FlowDifferent Hosts
30
vSphere Host
VM1
VDS
VXLAN Transport Network
VXLAN 5001
VM2
VXLAN 5002 1
4
vSphere HostLIF2 - ARP Table
DA: vMACSA: MAC1
DA: 20.20.20.20SA: 10.10.10.10
5002
MAC1 MAC25
172.16.10.10
2VM IP VM MAC
172.16.20.10 MAC2
PayloadL2 IP
DA: 172.16.20.10SA: 172.16.10.10
PayloadL2 IP
L2 IP UDP VXLAN PayloadL2 IP
172.16.20.10
LIF1LIF2 vMAC
LIF1LIF2 vMAC
Host 1 Host 2
3
10.10.10.10/24 20.20.20.20/24
3
DA: MAC2SA: vMAC
Example: Enterprise Routing Topology
VLAN 20
Core
Physical Routers
Web1 App1 DB1 Webn Appn DBnWeb2 App2 DB2
VXLAN 5020Uplink
Distributed Routing
Routing
Peering
Routing
Peering
E3 E8E1
Physical Routers
E2 …
Core
Routing Adjacencies
VXLAN
VLAN
Routing Adjacencies
CONFIDENTIAL 31
What Have We Seen Thus Far ..1. NSX architecture
2. An on-demand application deployment
3. Logical switching configuration
4. Understand logical networks
5. Logical routing and possible designs
CONFIDENTIAL 32
Agenda
1 NSX Architecture and Components
2 Switching
3 Routing
4 Distributed Firewall & Micro-Segmentation
5 Services
6 Summary & Next Steps
CONFIDENTIAL 33
NSX Distributed Firewalling
• Centralized Firewall Model• Static Configuration • IP Address based Rules• 40 Gbps per Appliance• Lack of visibility with encapsulated traffic
• Distributed at Hypervisor Level• Dynamic, API based Configuration• VM Name, VC Objects, Identity-based Rules • Line Rate ~20 Gbps per host• Full Visibility to encapsulated traffic
Challenges Benefits
PHYSICAL SECURITY MODEL DISTRIBUTED FIREWALLING
Firewall Mgmt
VMware NSX
API
CMP
NSX DFW Deep DiveSEC5589
CONFIDENTIAL 34
Distributed Firewall FeaturesVM5
VM1
vSphere Distributed Switch
Web-LS1
VM4App-LS1
Management Cluster192.168.150.51 192.168.150.52 192.168.250.51
VM2
• Firewall rules are enforced at VNIC Level• Policy independent of location (L2 or L3 adjacency)• State persistent across vMotion• Enforcement based on VM attributes like Tags, VM Names, Logical Switch, etc
Capabilities
CONFIDENTIAL 35
Distributed Firewall Rules VM5
VM1
vSphere Distributed Switch
Web-LS1
VM4App-LS1
Management Cluster192.168.150.51 192.168.150.52 192.168.250.51
VM2
Rules Based on VM NamesCONFIDENTIAL 36
Distributed Firewall Rules VM5
VM1
vSphere Distributed Switch
Web-LS1
VM4App-LS1
Management Cluster192.168.150.51 192.168.150.52 192.168.250.51
VM2
Rules Based on Logical Switches
CONFIDENTIAL 37
Example Building a Web DMZ
Web-Tier
App-Tier
External Network
Source Destination Service Policy
Any Web-Tier LS HTTPS Allow
Web-VM1 Web-VM2 Block
Any Web-Tier LS Block
Web-Tier LS App-Tier LS TCP 8443 Allow
Any App-Tier LS Block
STOP
Client to Web HTTPS Traffic
Web to App TCP/8443
CONFIDENTIAL 38
39External Network
VDS
Guest VM
Partner Services VM
vCenter Partner Console
DFW
Filtering Module
Slot 2
Slot 4Traffic RedirectionModule
NSX Distributed Firewall Packet WalkDFW, Filtering Module and Traffic Redirection Module
CONFIDENTIAL
Agenda
1 NSX Architecture and Components
2 Switching
3 Routing
4 Distributed Firewall & Micro-Segmentation
5 Services
6 Summary & Next Steps
CONFIDENTIAL 40
Features SummaryNSX Edge
Gateway Services
Rule configuration with IP, Port ranges, Grouping Objects, VC ContainersFirewall
Configuration of IP Pools, gateways, DNS servers and search domains.DHCP
IPSec site to site VPN between two Edges or other vendor VPN terminators.Site-to-Site VPN
Stretch your layer 2 across datacenters.L2VPN
Allow remote users to access the private networks behind Edge GSW.SSL VPN
Configure Virtual Servers and backend pools using IP addresses or VC ObjectsLoad Balancing
Source and Destination NAT capabilities.Network Address Translation
Active-Standby HA capability which works well with vSphere HA.High Availability
Static as well as Dynamic Routing protocols support (OSPF, BGP, ISIS)Routing
Allow configuring DNS relay and remote syslog servers.DNS/Syslog
NSX Edge Integrated Network Services
….
FirewallLoad BalancerVPNRouting/NAT
DHCP/DNS relayDDI
VM VM VM VM VM
Overview
• Integrated L3 – L7 services• Virtual appliance model to provide
rapid deployment and scale-out
Benefits
• Real time service instantiation• Support for dynamic service differentiation
per tenant/application• Uses x86 compute capacity
CONFIDENTIAL 42
NSX Load Balancing
• Application Mobility• Multi-tenancy• Configuration complexity – manual
deployment model
• On-demand load balancer service• Simplified deployment model for
applications – one-arm or inline • Layer 7, SSL, …
Challenges Benefits
LOAD BALANCER – Per Tenant Application Availability Model
Tenant A
VM1 VM2 VM1 VM2
Tenant B
NSX Load Balancing Deep DiveNET5612
CONFIDENTIAL 43
NSX L2VPN
Use Cases
• Brownfield NSX deployments (VLAN -> VXLAN)• Data Center Migrations (P2V, V2V)• Disaster Recovery & Testing• Cloud Bursting & Onboarding
Best Fit for L2 extensions with
• Long Distance / High Latency• Multiple management domains• NSX present only on a single site• Max 1500 byte MTU on WAN
Highlights
• SSL secured L2 extension over any IP network• Independent of vCenter Server boundaries• Can co-exist with existing default gateway • No specialized hardware required• Supports up to 750Mb/s per Edge• AES-NI supported if available
L2 VPN
Internet / WAN
Enterprise
L2 VPN
Internet / WAN
Hybrid Cloud
PublicCloud
Connecting Remote Sites with NSXNET5352
Agenda
1 NSX Architecture and Components
2 Switching
3 Routing
4 Distributed Firewall & Micro-Segmentation
5 Services
6 Summary & Next Steps
CONFIDENTIAL 45
46
VMware NSX – Summary and Takeaways • Faithful reproduction of L2 – L7 network & security services
• Services design for scale-out
• Central API for provisioning & monitoring
• All NSX components designed with resiliency
• Extensive 3rd party ecosystem for NSX platform
CONFIDENTIAL
NSX Ecosystem
CONFIDENTIAL 47
Service Insertion“Leverage full automation and
service insertion for NSX”
NSX aware“Leverage NSX API and
metadata to bring a solution”
Co-existence“Let’s meet in the network”
Works with any switching fabric
Works with routing ecosystem using
traditional protocols
Existing Physical firewall provide security sitting in front of NSX Edge at layer 3
Existing Physical/virtual ADC services can connect to NSX at layer 2 or layer 3
Network Virtualization Next Steps with VMware NSX
CONFIDENTIAL 48
virtualizeyournetwork.com
The online resource for the people, teams and organizations that are adopting network virtualization
communities.vmware.com
Connect and engage with network virtualization experts and fellow VMware NSX users
vmware.com/go/NVtraining
Build knowledge and expertise for the next step in your career
labs.hol.vmware.com
Test drive the capabilities of VMware NSX
VMware NSX - Deep DiveJacob Rapp, VMware, Inc
NET5560
#NET5560