achieving high performance with nfv data plane workloads
TRANSCRIPT
#vmworld
T5G1105BU
Achieving High Performance with NFV Data Plane Workloads: The Secrets
Jambi Ganbar, VMware, Inc.Dharma Rajan, VMware, Inc.
#T5G1105BU
VMworld 2019 Content: Not for publication or distribution
©2019 VMware, Inc.
Disclaimer
This presentation may contain product features or functionality that are currently under development.
This overview of new technology represents no commitment from VMware to deliver these features in any generally available product.
Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.
Technical feasibility and market demand will affect final delivery.
Pricing and packaging for any new features/functionality/technology discussed or presented, have not been determined.
2
The information in this presentation is for informational purposes only and may not be incorporated into any contract. There is no commitment or obligation to deliver any items presented herein. VMworld 2019 Content: Not for publication or distribution
©2019 VMware, Inc.
Agenda
3
Data Plane Performance and NSX Virtual Distributed Switch (N-VDS)
10G, 40G Performance Testing using vLAN and Overlay
VMware & Nokia Partnership
VMworld 2019 Content: Not for publication or distribution
4©2019 VMware, Inc.
Data Plane Performance and NSX Virtual Distributed Switch (N-VDS)
VMworld 2019 Content: Not for publication or distribution
©2019 VMware, Inc. 5
TELCO CLOUDS: CRITICAL FOR 5G SUCCESS
VMware: The Essential, Ubiquitous Digital Foundation across all 4 Clouds
VMware’s Vision - Any Cloud Strategy
ANY DEVICE
ANY APPLICATION Cloud Native SaaSTraditional
ANY CLOUD HYBRID PUBLIC
TELCO
EDGE
75
TELCOVMworld 2019 Content: Not for publication or distribution
©2019 VMware, Inc. 6
Use Cases Needing High Infrastructure Performance
NFV
vEPC
vIMS
vCPE
SD-WAN, uCPE
vRAN
MEC
Augmented Reality
Virtual Reality
Gaming
Video caching
Camera as a service
5G
eMBB, URLLC, mMTC
Connected cars
Real-time Telematics
AI / ML based systems
IoT
Factories of the Future
Narrow Band IoT
NFV - Network Function Virtualization
EPC – Evolved Packet Core
IMS – IP Multimedia Subsystem
CPE – Customer Premises Equipment
SD-WAN – Software Defined Wide Area Network
RAN – Radio Access Network
eMBB - enhanced Mobile Broadband
URLLC - Ultra Reliable Low Latency Communications
mMTC - massive Machine Type Communications
VMworld 2019 Content: Not for publication or distribution
©2019 VMware, Inc. 7
Data Plane Performance Requirement
NFV
64 - 1500 bytes packets
Packet loss < 0.001%
Latency < 50+msec
No Loss of virtualization function
MEC
High data volume
Real time decision
Low backhaul traffic
Near zero to 5 mslatency
Business continuity
5G
Data Rate 1 to 20Gb/s
Latency Data Plane 1-4 ms
Control Plane 20 ms
Density 1M device/sq km
Reliability – Six nines
IoT
%age of Devices up
Active users
Massive traffic increase
Significant IPV6 deployments
VMworld 2019 Content: Not for publication or distribution
©2019 VMware, Inc. 8
NFV Platform and N-VDS Secret
Data Plane VM
VMXNET3
Data Plane VM
VMXNET3
ESXi Host
N-VDS Enhanced Data Path mode
Dedicated CPU Cores for Networking
Dedicated Poll Mode Drivers
Service Provider Network
NSX-T Bare Metal Edge
VMworld 2019 Content: Not for publication or distribution
©2019 VMware, Inc. 9
virtual Evolved Packet Core Network
Data Plane Functions Connecting N-VDS
Management Plane Function
Data Plane Function
vSphere Distributed Switch
N-VDS SwitchEnhanced
Control Plane Function
N-VDS SwitchStandard
EMSMME
SGW-C
SGW-D
PGW-C
PGW-D
S5 (GTP-C)
S5 (GTP-U)
S11 (G
TP
-C)
S1 (GTP-U) UE
eNB
eNB
VMworld 2019 Content: Not for publication or distribution
©2019 VMware, Inc. 10
Effect of Packet Gap and Overlay Encapsulation HeaderHeader contributing to Performance
VLAN (802.1Q) RFC 3069 -- Max 4096 VLANS – Good for Non-Cloud DC
40GbE NIC
Original L2 Frame
VXLAN Header (8 bytes)
Outer UDP Header (8 bytes)
Outer Ethernet Header (14 bytes)
VXLAN Encapsulated Frame
Outer FCSOuter IP Header
(20 bytes)
Overlay Networks: VXLAN RFC 7348 -- Max 16 Million segments - Good for Cloud DC
40GbE NIC
Original L2 Frame
Geneve Header (8 bytes+ 8 byte option)
Outer UDP Header (8 bytes)
Outer Ethernet Header (14 bytes)
Geneve Encapsulated Frame
Outer FCSOuter IP Header
(20 bytes)
*https://datatracker.ietf.org/doc/draft-ietf-nvo3-geneve/
Overlay Networks: GENEVE IETF Draft* -- Variable length Options Good for Cross Cloud Architecture
VMworld 2019 Content: Not for publication or distribution
©2019 VMware, Inc. 11
Header contributing to Performance Gbps
Effect of Gap, Preamble and Header for 40G
VMworld 2019 Content: Not for publication or distribution
©2019 VMware, Inc. 12
Header contributing to Performance MPPS
Effect of Gap, Preamble and Header for 40G
VMworld 2019 Content: Not for publication or distribution
©2019 VMware, Inc.
vCloud NFV Verified Data Plane Performance KPIs
Paper Published in IEEE July 2019Achieving High Performance with Virtualized Data Plane Workloads for 5G Networks- D.Rajan
2019 Sixth International Conference on Software Defined Systems (SDS)https://ieeexplore.ieee.org/document/8768646
Packet Size bytes 64 128 256 512 IMIX 1024
Line rate Throughput ✓ ✓ ✓ ✓ ✓ ✓
Packet loss < 0.001% ✓ ✓ ✓ ✓ ✓ ✓
10 Gbps Scalability ✓ ✓ ✓ ✓ ✓ ✓
Logical cores/N-VDS (E) to saturate 10G link
4 3 2 1 1 1
Bi-Directional forwarding
✓ ✓ ✓ ✓ ✓ ✓
40 Gbps Scalability ✓ ✓ ✓ ✓ ✓ ✓
Logical cores/N-VDS (E)to saturate 40G link
4+ 4+ 4+ 4 3 3
Multiple links supported ✓ ✓ ✓ ✓ ✓ ✓
Sub second Latency ✓ ✓ ✓ ✓ ✓ ✓
Sub second Jitter ✓ ✓ ✓ ✓ ✓ ✓VMworld 2019 Content: Not for publication or distribution
14©2019 VMware, Inc.
NFV Performance Results and Configuration
VMworld 2019 Content: Not for publication or distribution
©2019 VMware, Inc. 15
A brief vCloud NFV performance tuning guide – Host Level
Secrets of High Data Plane Performance
• CPUs with a high count of cores per NUMA • Fast pNICs
Server Class
• BIOS tuned for performance - did you catch “Extreme Performance session” yesterday?
• Turbo Boost• Hyper Threads
Server BIOS
• Intel Fortville• Mellanox ConnectX-5
Network Interface Cards (examples)
VMworld 2019 Content: Not for publication or distribution
©2019 VMware, Inc. 16
NSX-T Configuration for Data PlaneHost-level configuration
Identify and install driver
Install PMD drivers and Firmware
VMworld 2019 Content: Not for publication or distribution
©2019 VMware, Inc. 17
NSX-T Configuration for Data PlaneNSX-T Manager-level configuration
Create a transport zone
Create a Logical Switch
Assign Cores to N-VDS (E)
Disable DFWVMworld 2019 Content: Not for publication or distribution
©2019 VMware, Inc. 18
Use recent DPDK version in the VM
Use VMXNET3
Use the latest VM hardware version
Tune your buffers
Align VM to NUMA
Set Latency Sensitivity (and pre-requisites)
The Secrets to Tuning Your VMs for PerformanceSimplified Configuration with vCloud NFV 3.x
VMworld 2019 Content: Not for publication or distribution
©2019 VMware, Inc. 19
Data Plane Acceleration with Overlay TrafficTest environment logical topology
Bidirectional Traffic
Acceptable packet loss of 0.001%
40GbE Intel XL710 NICs
PROX VMPROX VM
ESXi Host
N-VDS (E)
PROX VMPROX VM
ESXi Host
N-VDS (E)
Layer 2 Fabric
Spirent TestCenter
NSX BM Edge
Uplink
Downlink
VMworld 2019 Content: Not for publication or distribution
©2019 VMware, Inc. 20
If I use overlay, what performance can I expect from my host?
Single Host Overlay Performance
4 logical cores
3.1
3 4.5
5 6.8
6
7.9
5
8.9
4 11.4
8
11.7
3
0
2
4
6
8
10
12
14
1 2 3 4 5 6 7
Packet Size (Bytes)
Th
rou
gh
pu
t (M
PP
S)
GeneveLine Rate
GeneveLine Rate
GeneveLine Rate
VMworld 2019 Content: Not for publication or distribution
©2019 VMware, Inc. 21
3.1
6 4.5
3
5.2
8 6.8
6 8.4
7
14.9
7
18.1
2
18.1
4
0
2
4
6
8
10
12
14
16
18
20
1 2 3 4 5 6 7 8
Th
rou
gh
pu
t (M
PP
S)
Packet Size (Bytes)
BM Edge Scalability Test Results With Overlay Traffic – 40GbESmall NSX BM Edge – 12 cores
Genève Line RateGenève
Line Rate
Genève Line Rate
Genève Line Rate
Genève Line Rate
Genève Line Rate
VMworld 2019 Content: Not for publication or distribution
©2019 VMware, Inc. 22
3.1
6
4.5
3
5.2
8
6.8
6
8.4
7
14.9
7
24
.27
25
.10
0
5
10
15
20
25
30
1 2 3 4 5 6 7 8
Th
rou
gh
pu
t (M
PP
S)
Packet Size (Bytes)
BM Edge Scalability Test Results With Overlay Traffic – 40GbELarge NSX BM Edge – 20 cores
Genève Line RateGenève
Line Rate
Genève Line Rate
Genève Line Rate
Genève Line Rate
Genève Line Rate
Genève Line Rate
VMworld 2019 Content: Not for publication or distribution
©2019 VMware, Inc. 23
Lets SummarizeVMware vCloud NFV – Telco platform for all workloads
Use case and Requirements
vCloud NFV Architecture
Test Results and
Configuration
Fireside Chat
VMworld 2019 Content: Not for publication or distribution
24©2019 VMware, Inc.
VMware & Nokia PartnershipFireside Chat
Allwyn Sequiera SVP VMwareRon Heberman CTO Nokia Software
VMworld 2019 Content: Not for publication or distribution
©2019 VMware, Inc.
VMware & Nokia PartnershipCreating Value for Communication Service Providers
VMworld 2019 Content: Not for publication or distribution
VMworld 2019 Content: Not for publication or distribution
VMworld 2019 Content: Not for publication or distribution