nytro-xv_nwd_vm_performance_acceleration
TRANSCRIPT
LSI Data Center Solutions GroupDecember 17, 2013Khai Le
VMware Acceleration with Nytro WarpDrive PCI Flash and Proximal Data AutoCache
2Confidential
New Product Terminology and Solution Strategy
Nytro XD-V AutoCache
vAppAutoCache_1.1.21.ova
Nytro WarpDrive (Legacy)
LSI support for Nytro XD and XD-V EOL!
Third Party Solutions – Proximal AutoCache, EMC VF Cache, NetApp Flash Cache, SanDisk Express Cache and others
Nytro XP6209 and Nytro XP6210
3Confidential
Objectives:
Understand how AutoCache accelerates storage in virtualized environment
Understand how to test AutoCache performance scaling using synthetic benchmark
Understand how to tune for optimal AutoCache performance in virtualized environment
Understand how to monitor AutoCache IO workload and performance statistics in virtualized environment
4Confidential
Agenda
Why performance is important?
Customer satisfaction
Reduce solution and operating costs
Enable Nytro WarpDrive acceleration in virtualized environments
Product
AutoCache Overview Review Performance Improvement
VMware Storage Acceleration
Summary of results
Available Resources
5Confidential
Proximal Data AutoCache & OS SupportCache Device Auto Cache 1.1 Auto Cache 2.0
Intel® SSD 313 Series √ √
Intel® SSD 320 Series √ √
Intel® SSD 330 Series √ √
Intel® SSD 520 Series √ √
Intel® SSD 710 Series √ √
Intel® SSD 910 Series √ √
LSI Nytro WarpDrive WLP Series
√ √
LSI Nytro WarpDrive BLP Series
√ √
LSI Nytro WarpDrive SLP Series
√ √
Micron P400e SSD Series √ √
Micron P320h SSD Series √ √
More OS Support Auto Cache 1.1 Auto Cache 2.0
VMware ESXi 4.1, 5.0, 5.1
√ √
VMware ESXi 5.5 √
6Confidential
vCenter Diagram
7Confidential
Storage Acceleration for Server Virtualization
DASSANNAS
+
NytroXD-VCaching Software+
Nytro WarpDrive
AutoCacheSoftware
8Confidential
System Components impacting Max IOPS Nytro WarpDrive PCIe Flash Controller
AutoCache Flash Caching Software
Baseline (dataStorage - DAS /SAN /NAS)
Application I/O workload
Performance in Virtualized Environments
9Confidential
Single VM measure IOPS
Notes:• Baseline (R10-16HDD): Single 16-drive RAID-10 HDD datastore baseline• WarpDrive_100G_vmdk: Nytro WarpDrive 6203 as VM datastore, no caching• NXD-V 100G_Hot_Region: Nytro WarpDrive 6203 + AutoCache v1.1.21 caching software
10Confidential
Single VM measure Latency (milliseconds)
Notes:• Baseline (R10-16HDD): Single 16-drive RAID-10 HDD datastore baseline• WarpDrive_100G_vmdk: Nytro WarpDrive 6203 as VM datastore, no caching• NXD-V 100G_Hot_Region: Nytro WarpDrive 6203 + AutoCache v1.1.21 caching software
Use log scale so you can see the lower latency better?
11Confidential
VMware VisualESXTOP capture on AutoCacheMax IOPS & VM IOPS -Workload 8KB 100% RR
Virtual Disks (vmdk): 4 * 20K IOPS ~ 80K IOPS for Nytro WarpDrive 6203
12Confidential
Configuration
Vmware– ESXi version 5.0 Update 2– 64bit build 914586
AutoCache v1.1.21– AutoCache: Pdi-cache ver-500.1.1.21-1381159898– Cache Statistics Monitor: proximal ver-500.1.1.21-1381159898
Nytro WarpDrive 6203– 1.6TB (eMLC)
MegaRAID (TB) controller– VMware Data Storage (VD)
• 16 SAS 6G HDD RAID-10– WT, NRA, DIO, DCD, SS 64KB
VM information:– Windows 2008 R2 SP1– 4 vCPU @ 2.0 GHz– 4 GB of vRAM– 50 GB vmdk
13Confidential
Nytro WarpDrvie 6203 Information
Model: Nytro WarpDrive 6203 1.6TB DFF flash type: eMLC Over Provision: 28% Default Capacity: 1.46 TB after OP setting Firmware: 110.00.00.00 (Phase 10 components) Driver: mpt2sas 17.00.00.00.1vmw (Phase 10 components)
IOmeter ver-2008.06.18-RC2. - Highly patterned compressible data Workload: 8KB block Size, 100% Read 0% Write, 100% Random IO Queue Depth 64, 8KB Aligned I/O 1 worker 50G Hot Region Per VM
Workload
14Confidential
PowerCLI Script
Syntax for mulitple VMs kick off performance test at the same time– Ensures each VM receives balanced percentage of resources that
balances total available performance between VMS
$cred = Get-CredentialGc test.txt | % { Invoke-VMScript –RunAsync –VM $_ -ScriptText { C:\iom2008\DEMO_RUNME.bat } –GuestCredential $cred }
Test.txt vsiTarget001
vsiTarget002 vsiTarget003 vsiTarget004
DEMO_RUNME.bat C:\iom2008\iometer.exe /c iom2008\8kb100R0W.icf /r iom2008\vm4_results.csv
15Confidential
Run time270 intervals of 10 sec each = 45 mins
16Confidential
HDD Volume Performance Baseline4 VMs (vmdk) data Stores on same base volume
4 VMs * 500 IOPS/VM ~ 2,000 IOPS total
17Confidential
Accelerated Multiple (4) VMs Performance Scaling
AutoCache v1.1.21 Accelerate Data Storage (4 VMs -vmdk)Nytro WarpDrive 6203 PCIe Flash
Notes:• Baseline: DAS RAID-10 for datastore• NytroXD-V: Nytro WarpDrive 6203 + AutoCache v1.1.21 caching software
Hot data size = 50GB/VM
4 VMs * 2,000 IOPS/VM ~ 80,000 IOPS total
18Confidential
Nytro WarpDrive 6203 PCIe Flash – no AutoCache4 VMs (vmdk) on 1.6TB eMLC
4 VMs * 25,000 IOPS/VM ~ 100,000 IOPS totalShows AutoCache v1.1.21 Caching Overhead ~22.5us
19Confidential
Accelerated Multiple (4) VMs on SAN Storage
AutoCache v1.1.21 + Nytro WarpDrive 6203 PCIe FlashMix Workload: 8KB 100% Random, 70/30 (R/W)
Hot data size = 50GB/VM
4 VMs * 1750 IOPS/VM ~ 7,000 IOPS total
Notes:• SAN: backend connect to Snowmass,16-HDDs create RAID-10 for datastore• NXD-V: Nytro WarpDrive 6203 + AutoCache v1.1.21 caching software
20Confidential
Accelerated Multiple (4) VMs on SAN Storage
AutoCache v1.1.21 + Nytro WarpDrive 6203 PCIe FlashMix Workload: 8KB 100% Random, 70/30 (R/W)
Notes:• SAN: backend connect to Snowmass,16-HDDs create RAID-10 for datastore• NXD-V: Nytro WarpDrive 6203 + AutoCache v1.1.21 caching software
AutoCache reduces IO latency
21Confidential
AutoCache VMware Cache ManagementCapture for 4 VMs on SAN
Notes:• AutoCache Cache policy: Write Through mode
Mix workload 70/30 (R/W)
22Confidential
Performance Tip
Format Nytro WarpDrive 6203 PCIe Flash Controller Syntax:
./ddcli –c 1 –format ./ddmfgcli –c 1 –format
Precondition flash Cache device iometer.exe /c iom2008\8kb100%SW.icf /r iom2008\precondition_results.csv
Increase Nytro WarpDrive 6203 Driver Queue Depth 1. Syntax adjust SATA Q.Depth: esxcfg –module –s mpt2sas_SATA_queue_depth=128 mpt2sas
2. Syntax adjust max controller Q.Depth: esxcfg-module --set-options "max_queue_depth=620” mpt2sas
23Confidential
ESXi Host, Disk Queue Depth:VMware -> configuration -> Software ->
Advanced Settings -> Disk.SchedNumReqOutStanding=256
Create vmdk select option Independent disks
are not affect by snapshots
Continue Performance Tip
24Confidential
Performance Tuning:Driver Queue Depth and Disk Schedule Outstanding Request Count
Default Queue Depth
1VM Performance Shown
Notes:• IOmeter: 8KB 100% random read• NytroXD-V: Nytro WarpDrive 6203 + AutoCache v1.1.21 caching software
25Confidential
Esxtop follow cmd options:
-v disk vm-u device Queue Depth-d disk Adapter-c CPU utilization-m memory-n Network-p Power
VMware Real Time Performance Monitor
26Confidential
Cmd: esxtop v, dHDD Baseline Performance
500 IOPS x 4VM ~ 2000 IOPS
27Confidential
Cmd: esxtop v, dAccelerated AutoCache v1.1.21 Performance
28Confidential
How to save esxtop statistics:
esxtop -b -d 2 -n 100 > esxtopcapture.csv
esxtop -b -a -d 10 -n 100 | gzip -9c > esxtopoutput.csv.gz -
29Confidential
IO Analyzer under virtualization
Vmware Visual ESXtop– Connect to a live
Server vCenter, ESXi host Connect multiple windows– Chart metrics– Select metrics counter– Save results to a file– Replay data to chart or
graph
30Confidential
MySQL Database Workload Example
Commands/sec (total)
Reads/sec
Writes/sec
31Confidential
Additional Statistics Monitoring Options
Cmds/s -the number of cmd issued per second
DAVG/cmd -the average device latency(m.secs) per cmd
DAVG/rd -the average device latency per/read
DAVG/wd -the average device latency per/write
DQLEN -the storage device Q. Depth.
GAVG/cmd -the average Guest OS latency m.sec per cmd
KAVG/cmd -the average ESX Vmkernel latency per cmd
LOAD -ratio of ESX VMKernel active cmd +queue cmd
MBREAD/s –Mega bytes read per second
MBWRTN/s-Mega bytes write per second
QAVG/cmd –the average queue latency (ms)per cmd
QUED -# of cmd in ESX Vmkernel currently queued
READ/s –the number of read cmd issued per second
WRITE/s-the number of write cmd issued per second
More
32Confidential
AutoCache v1.1.21 Multiple NWD Performance Scaling
Second Nytro WarpDrive 6203 provides additionalcaching capacity but no performance improvement.
Notes:• IOmeter: 8KB 100% random read• NytroXD-V: Nytro WarpDrive 6203 + AutoCache v1.1.21 caching software
33Confidential
AutoCache v1.1.21 Average Response Time
Zoom in at 20 msec, so latency between 1 NXP and 2 NXPs about the same
1x Nytro WaprDrive 6203 Latency
2x Nytro WarpDrive 6203Latency
Notes:• IOmeter: 8KB 100% random read• NytroXD-V: Nytro WarpDrive 6203 + AutoCache v1.1.21 caching software
34Confidential
AutoCache 2.x Multiple NWD Performance Scaling
2x Nytro WarpDrive 6203 performance AutoCache v2.0 demonstrates Nytro WarpDrive 6203 PCIe flash performance scaling
1x Nytro WarpDrive 6203 performance
35Confidential
AutoCache 2.x Average Response Time
(milliseconds)
1x Nytro WarpDrive6203 Latency
2x Nytro WaprDrive 6203 Latency
36Confidential
AutoCache Summary
Nytro WarpDrive 6203 Scaling - 1 and 2 PCIe Flash Devices– AutoCache 1.1.21
• Significant performance improvement• Added flash caching capacity• Added performance for hot region size > 1st Nytro WarpDrive 6203 capacity
– AutoCache 2.0• Significant performance improvement• Added flash caching capacity• Added performance for any hot region size
Performance Ramp Up Time– Dependent on total hot region size (i.e., #VMs * Hot Region Size/VM)
Nytro WarpDrive 6203 Driver Performance Optimizations– Increasing Queue Depth and Disk Schedule Outstanding Request Count
• Significant acceleration improvement
37Confidential
Resources
Nytro WarpDrive and Nytro XP http://
www.lsi.com/products/flash-accelerators/pages/default.aspx#tab/product-family-tab-3
AutoCache Resources http://www.proximaldata.com/product/resources.phpv
Proximal Download Portal http://www.proximaldata.com/support/software_download_portal.php
Sphere Best Practices http://www.vmware.com/pdf/Perf_Best_Practices_vSphere5.0.pdf
esxtop Download http://www.yellow-bricks.com/esxtop/
VMware io Analyzer Download http://labs.vmware.com/flings/io-analyzer
Q/A?