vsphere performance best practices - vmware › vmworld › 2012 › top10 › vsp1800.pdf · $25...
TRANSCRIPT
![Page 1: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/1.jpg)
vSphere Performance Best Practices
Peter Boone, VMware, Inc.
INF-VSP1800
#vmworldinf
![Page 2: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/2.jpg)
2
Disclaimer
This session may contain product features that are currently under development.
This session/overview of the new technology represents no commitment from VMware to deliver these features in any generally available product.
Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.
Technical feasibility and market demand will affect final delivery.
Pricing and packaging for any new technologies or features discussed or presented have not been determined.
![Page 3: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/3.jpg)
3
Global Support Services and Customer Advocacy
Bangalore, India
Tokyo, Japan
Cork, Ireland Burlington, Canada
Palo Alto, CA Broomfield, CO
Support offices Local language support Spanish, Portuguese, French, German, Japanese, Chinese
Global Coverage 24x7, 365 days/year 6 Support Centers
1000+ Support Engineers
Follow-the-sun Support for
Severity 1 Issues
Support Relationships with 100% of the
Fortune 100; 99% of Fortune 500
![Page 4: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/4.jpg)
4
Customer Support Day Events
Coming to a location near you: sharing of VMware best practices! Support Days are a collaboration between VMware Support, Sales
and customers – you learn directly from the experts Topics are driven by
customer input, and typically include: • Best practices
• Tips/tricks
• Top issues
• Product roadmaps/demos
• Certification offerings
http://www.vmware.com/go/supportdays
![Page 5: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/5.jpg)
5
Overview
What a performance problem sounds like: • “My VM is running slow and I don’t know what to do!”
• “I tried adding more memory and CPUs but the problem got worse!”`
• “My VM is slow on one host but fast on another!”
What to look for? Where to start?
We will explore some of the most common performance-related
issues that our support centers receive cases for
![Page 6: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/6.jpg)
6
A word about performance….
Troubleshooting methodology must define: • How to find root cause
• How to fix the problem
Must answer these questions: 1. How do we know when we are done?
2. Where do we start looking for problems?
3. How do we know what to look for to identify a problem?
4. How do we find the root-cause of a problem we have identified?
5. What do we change to fix the root-cause?
6. Where do we look next if no problem is found?
![Page 7: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/7.jpg)
7
Agenda
Benchmarking & Tools Best Practices and Troubleshooting
The 4 “food groups”
• Memory
• CPU
• Storage
• Network
![Page 8: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/8.jpg)
© 2012 VMware Inc. All rights reserved
BENCHMARKING & TOOLS
![Page 9: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/9.jpg)
9
Benchmarking
Consistent and reproducible results Important to have base level of acceptable performance
• Expectation vs. Acceptable
Determine baseline of performance prior to deployment • Benchmark on a physical system if applicable
Avoid subjective metrics, stay quantitative • “The system seems slower”
• “This worked better last year”
![Page 10: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/10.jpg)
10
Benchmarking
Benchmarking should be done at the application layer • Use application-specific benchmarking tools and load generators
• Check with the application vendor
Isolate variables, benchmark optimum situation before introducing load
Understand dependencies • Human interaction
• Other “food groups”
• Compare apples-to-apples
![Page 11: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/11.jpg)
11
Aggregates thousands of metrics into Workload, Capacity, Health scores Self-learns “normal” conditions using patented analytics
Smart alerts of impending performance and capacity degradation
Identifies potential performance problems before they start
Slide 11
Tools – vCenter Operations
![Page 12: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/12.jpg)
12
Tools – vCenter Operations Slide 12
![Page 13: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/13.jpg)
13
Tools – esxtop
Valuable tool built in to vSphere hosts View or capture real-time data
• View or playback data later
• Import data in 3rd party tools
vSphere Client performance graphs get their data from esxtop data • Presentation/unit may be different (e.g. %RDY)
Little overhead impact on the host
![Page 14: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/14.jpg)
© 2012 VMware Inc. All rights reserved
MEMORY
![Page 15: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/15.jpg)
15
Memory – Allocation
A VM’s RAM is not necessarily physical RAM • vRAM + overhead = maximum physical RAM
Whether or not that memory is physical or virtual depends on…
• Host configuration
• Shares
• Limits
• Reservations
• Host load
• Idle/Active VMs
![Page 16: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/16.jpg)
16
Memory – Overhead
Source: vSphere 5.0 Resource Management Guide
![Page 17: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/17.jpg)
17
Memory – Host Memory Management
Occurs when memory is under contention Transparent Page Sharing
Ballooning
Compression
Swapping
![Page 18: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/18.jpg)
18
Memory – Transparent Page Sharing
![Page 19: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/19.jpg)
19
Memory – Ballooning
![Page 20: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/20.jpg)
20
Memory – Compression
![Page 21: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/21.jpg)
21
Memory – Swapping
![Page 22: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/22.jpg)
22
Memory – Swapping
![Page 23: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/23.jpg)
23
Memory – VM Resource Allocation
![Page 24: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/24.jpg)
24
Memory – Resource Pool Allocation
![Page 25: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/25.jpg)
25
Memory – Ballooning vs. Swapping
Ballooning is better than swapping Guest can surrender unused/free pages
Guest chooses what to swap, can avoid swapping “hot” pages
Idle memory tax uses ballooning
![Page 26: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/26.jpg)
26
Memory – Rightsizing
Generally, it is better to OVER-commit than UNDER-commit If the running VMs are consuming too much host/pool memory…
• Some VMs may not get physical memory
• Ballooning or host swapping
• Higher disk IO
• All VMs slow down
![Page 27: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/27.jpg)
27
Memory – Rightsizing
If a VM has too little vRAM… • Applications suffer from lack of RAM
• The guest OS swaps
• Increased disk traffic, thrashing
• SAN slow down as a result of increased disk traffic
If a VM has too much vRAM… • Higher overhead memory
• Possible decreased failover capacity
• Longer vMotion time
• Larger VSWP file
• Wasted resources
![Page 28: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/28.jpg)
28
Memory – Troubleshooting
Wrong resource allocation May not notice a limit, e.g. VM or template with a limit gets cloned
Custom share values
Ballooning or swapping at the host level • Ballooning is a warning sign, not a problem
• Swapping is a performance issue if seen over an extended period
Swapping/paging at the guest level • Under-provisioned guest memory
Missing balloon driver (Tools)
![Page 29: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/29.jpg)
29
Memory – Best Practices
Avoid high active host memory over-commitment • No host swapping occurs when total memory demand is less than the physical
memory (Assuming no limits)
Right-size guest memory • Avoid guest OS swapping
Ensure there is enough vRAM to cover demand peaks
Use a fully automated DRS cluster
• Test that vMotion works
• Use Resource Pools with High/Normal/Low shares
• Avoid using custom shares
![Page 30: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/30.jpg)
© 2012 VMware Inc. All rights reserved
CPU
![Page 31: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/31.jpg)
31
CPU – Overview
Raw processing power of a given host or VM • Hosts provide CPU resources
• VMs and Resource Pools consume CPU resources
CPU cores/threads need to be shared between VMs Fair scheduling vCPU time
• Hardware interrupts for a VM
• Parallel processing for SMP VMs
• I/O
![Page 32: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/32.jpg)
32
CPU – esxtop
![Page 33: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/33.jpg)
33
CPU – esxtop
Interpret the esxtop columns correctly %USED – Physical CPU usage %SYS – Percentage of time in the VMkernel %RUN – Percentage of total scheduled time to run %WAIT – Percentage of time in blocked or busy wait states %IDLE – %WAIT- %IDLE can be used to estimate I/O wait time
![Page 34: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/34.jpg)
34
CPU – Performance Overhead & Utilization
Different workloads have different overhead costs (%SYS) even for the same utilization (%USED) CPU virtualization adds varying amounts of system overhead
• Direct execution vs. privileged execution
• Non-paravirtual adapters vs. emulated adaptors
• Virtual hardware (Interrupts!)
• Network and storage I/O
![Page 35: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/35.jpg)
35
CPU – vSMP
Relaxed Co-Scheduling: vCPUs can run out-of-sync Idle vCPUs incur a scheduling penalty
• configure only as many vCPUs as needed
• Impose unnecessary scheduling constraints
Use Uniprocessor VMs for single-threaded applications
![Page 36: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/36.jpg)
36
CPU– Scheduling
Over committing physical CPUs
VMkernel CPU Scheduler
![Page 37: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/37.jpg)
37
CPU– Scheduling
Over committing physical CPUs
VMkernel CPU Scheduler
X X
![Page 38: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/38.jpg)
38
CPU– Scheduling
Over committing physical CPUs
VMkernel CPU Scheduler
X X X X
![Page 39: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/39.jpg)
39
CPU – Ready Time
The percentage of time that a vCPU is ready to execute, but waiting for physical CPU time
Does not necessarily indicate a problem • Indicates possible CPU contention or limits
![Page 40: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/40.jpg)
40
CPU – NUMA nodes
Non-Uniform Memory Access system architecture
Each node consists of CPU cores and memory A CPU core in one NUMA node can access memory in another
node, but at a small performance cost
NUMA node 1 NUMA node 2
![Page 41: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/41.jpg)
41
CPU – NUMA nodes
The VMkernel will try to keep a VM’s vCPUs local to its memory • Internal NUMA migrations can occur to balance load
Manual CPU affinity can affect performance
• vCPUs inadvertently spread across NUMA nodes
• Not possible with fully automated DRS
VMs with more vCPUs than cores available in a single NUMA node
may see decreased performance
![Page 42: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/42.jpg)
42
CPU – Troubleshooting
vCPU to pCPU over allocation • HyperThreading does not double CPU capacity!
Limits or too many reservations • can create artificial limits.
Expecting the same consolidation ratios with different workloads
• Virtualizing “easy” systems first, then expanding to heavier systems
• Compare Apples to Apples • Frequency, turbo, cache sizes, cache sharing, core count, instruction set…
![Page 43: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/43.jpg)
43
CPU – Best Practices
Right-size vSMP VMs Keep heavy-hitters separated
• Fully automated DRS should do this for you
• Use anti-affinity rules if necessary
Use a fully automated DRS cluster
• Test that vMotion works
• Use Resource Pools with High/Normal/Low shares
• Avoid using custom shares
![Page 44: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/44.jpg)
© 2012 VMware Inc. All rights reserved
STORAGE
![Page 45: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/45.jpg)
45
Storage – esxtop Counters
Different esxtop storage views • Adapter (d)
• VM (v)
• Disk Device (u)
Key Fields: • DAVG + KAVG = GAVG
• QUED/USD – Command Queue Depth
• CMDS/s – Commands Per Second
• MBREADS/s
• MBWRTN/s
![Page 46: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/46.jpg)
46
Storage – Troubleshooting with esxtop
High DAVG: issue beyond the adapter • bad/overloaded zoning, over utilized storage processors, too few platters in the
RAID set, etc.
High KAVG: issue in the kernel storage stack
• Driver issue
• Full queue
Aborts: GAVG exceeding 5000 ms
• Command will be repeated, storage delay for the VM
![Page 47: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/47.jpg)
47
Storage – Benchmarking with iometer
![Page 48: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/48.jpg)
48
Storage – Storage I/O Control
Allows the use of Shares per VMDK Throttling occurs when datastore reaches latency threshold
• Higher share VMDKs perform IO first
vCenter monitors latency across all hosts • Not effective if datastore shared with other vCenters
![Page 49: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/49.jpg)
49
Storage – Storage DRS
Datastore clusters • Maintenance mode
• Anti-affinity rules
vCenter monitors for latency and disk space
• Migrate VMDKs for better performance or utilization
Not effective with automated tiering SANs • Check HCL to confirm these features are compatible
![Page 50: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/50.jpg)
50
Storage – Troubleshooting
Snapshots Excessive traffic down one HBA / Switch / SP can cause latency
• Consider using Round Robin in conjunction with ALUA
• Always be paranoid when it comes to monitoring storage I/O
Consider your I/O patterns • Peak time for storage IO?
• Virus scans, database maintenance, user logins
Always consult with array vendor • They know the best practices for their array!
![Page 51: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/51.jpg)
51
Storage – Best Practices
Use different tiers of storage for different VM workloads • Slower storage for OS VMDKs
• Faster storage for databases or other high-IO applications
Use the Paravirtual SCSI adapter
• Reduced overhead, higher throughput
Use path balancing where possible, either through plugins
(Powerpath) / Round Robin and ALUA, if supported. Use Storage DRS with SIOC
• Balance for both free space and latency
• Simplified datastore management
![Page 52: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/52.jpg)
© 2012 VMware Inc. All rights reserved
NETWORK
![Page 53: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/53.jpg)
53
Network – Load Balancing
Load balancing defines which uplink is used • Route based on Port ID
• Route based on IP hash
• Route based on MAC hash
• Route based on NIC load
Probability of high-bandwidth VMs being on the same physical NIC
Traffic will stay on elected uplink until an event occurs
• NIC link state change, adding/removing NIC from a team, beacon probe timeout…
![Page 54: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/54.jpg)
54
Network – Troubleshooting
Check counters for NICs and VMs • Network load imbalance
• 10 Gbps NICs can incur a significant CPU load when running at 100%
Ensure hardware supports TSO • Use latest drivers and firmware for your NIC on the host
For multi-tier VM applications, use DRS affinity rules to keep VMs on same host • Same vSwitch / VLAN, rules out physical network
If using Jumbo Frames, ensure it is enabled end-to-end
![Page 55: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/55.jpg)
55
Network – Best Practices
Use the vmxnet3 virtual adapter • Less CPU overhead
• 10 Gbps connection to vSwitch
Use the latest driver/firmware for the NICs on the host Use network shares
• Requires Virtual Distributed Switch 4.1
Isolate vMotion and iSCSI traffic from regular VM traffic • Separate vSwitches with dedicated NIC(s)
• Most applicable with Gigabit NICs
![Page 56: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/56.jpg)
56
In conclusion…
![Page 57: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/57.jpg)
57
Key Takeaways – Performance Best Practices
Understand your environment • Hardware, storage, networking
• VMs & applications
Advanced configuration values do not need to be tweaked or
modified • In almost all situations
Use fully automated DRS
Use Paravirtual virtual hardware
![Page 58: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/58.jpg)
58
Important Links
![Page 59: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/59.jpg)
59
Important Links
![Page 60: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/60.jpg)
FILL OUT A SURVEY
EVERY COMPLETE SURVEY IS ENTERED INTO DRAWING FOR A
$25 VMWARE COMPANY STORE GIFT CERTIFICATE
![Page 61: vSphere Performance Best Practices - VMware › vmworld › 2012 › top10 › vsp1800.pdf · $25 VMWARE COMPANY STORE GIFT CERTIFICATE . vSphere Performance Best Practices . Peter](https://reader035.vdocuments.us/reader035/viewer/2022063002/5f250aea6481ef7ddb0f71f1/html5/thumbnails/61.jpg)
vSphere Performance Best Practices
Peter Boone, VMware, Inc.
INF-VSP1800
#vmworldinf