emc vplex virtual edition: use cases and performance planning · vplex virtual edition : ... the...

White Paper

Abstract

This white paper provides an overview of VPLEX/VE use cases and performance characteristics

EMC VPLEX VIRTUAL EDITION: USE CASES AND PERFORMANCE PLANNING

2 VPLEX VIRTUAL EDITION: USE CASES AND PERFORMANCE PLANNING

Copyright © 2014 EMC Corporation. All Rights Reserved.

EMC believes the information in this publication is accurate of its publication date. The information is subject to change without notice.

The information in this publication is provided “as is”. EMC Corporation makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose.

Use, copying, and distribution of any EMC software described in this publication requires an applicable software license.

For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com.

All other trademarks used herein are the property of their respective owners.

Part Number H13216.1

For more information: Explore and compare the latest VPLEX products in the EMC Store

https://store.emc.com/vplex


Table of Contents

Executive summary ............................................................................................................ 4

Document scope and limitations ........................................................................................ 4 Audience............................................................................................................................ 4

Introduction ...................................................................................................................... 5

VPLEX/VE Overview ........................................................................................................... 6

Architecture and Deployment ............................................................................................. 7 VPLEX Witness ............................................................................................................... 8 VPLEX/VE vApp Architecture .......................................................................................... 9 VPLEX/VE STORAGE CONCEPTS ...................................................................................... 9

VPLEX/VE Use Cases ........................................................................................................ 11 Block Storage High Availability .................................................................................... 11 Application HA ............................................................................................................. 11 Migration ..................................................................................................................... 12 Dynamic Workload Load Balancing across Sites........................................................... 12 Immediate Cross-Site vMotion ..................................................................................... 12

VPLEX/VE Performance Planning ...................................................................................... 13 Summary ..................................................................................................................... 13 Background ................................................................................................................. 13 Hardware Configuration ............................................................................................... 13

Key Performance Indicators (KPIs) .................................................................................... 14 Simulated Application Profiles ..................................................................................... 14 I/O Profile of Simulated Applications ........................................................................... 14 Notes on Selecting the Application Profile .................................................................... 15

Testing Methodology ....................................................................................................... 16 Type of Measurements ................................................................................................. 16 Data Charts .................................................................................................................. 16

Conclusion ...................................................................................................................... 19

VPLEX References ............................................................................................................ 20

Supplemental VMware References ................................................................................... 20


Executive summary EMC® VPLEX/VE™ delivers software-defined data mobility and availability within and across sites for VMware based infrastructures. VPLEX/VE is a unique virtual storage technology that enables mission critical applications to remain up and running during any of a variety of planned and unplanned downtime scenarios. VPLEX/VE permits fluid, non-disruptive data movement, taking technologies that were built assuming a single block storage instance and enabling them to function across disparate iSCSI arrays and across distance. EMC VPLEX/VE leverages the proven advantages of the EMC VPLEX family and puts that power in a customer installable, COTS hardware based configuration that is managed using standard VMware workflows and interfaces.

Document scope and limitations The use cases and performance overview discussed in this white paper are applicable to the VPLEX/VE 2.1 release. The use cases and performance details provided in this white paper are only applicable to environments consisting of the following elements:

• ESXi 5.1, 5.5 • VMware Enterprise+ Licensing • EMC VPLEX/VE 2.1 • 2+ EMC VNXe 3100 and 3200 series iSCSI arrays • 8-32 ESXi hosts [See the EMC VPLEX/VE Product Guide available at

http://support.emc.com for detailed host requirements] Please consult with EMC sales and support representatives if there is uncertainty as to the applicability of this information for specific VPLEX/VE environments.

Audience This white paper is intended for technology architects, storage administrators, and VMware system administrators who use or are planning to use EMC VPLEX/VE technology. It is assumed that the reader is familiar with VMware and iSCSI storage array technologies.

http://support.emc.com/


Introduction Data center infrastructure is undergoing a massive shift. Virtualization in the data center has had a profound impact on customer expectations of flexibility and agility. Especially as customers get to 70+% virtualized, they have the potential to realize tremendous operational savings by consolidating management in their virtualization framework. In this state, customers typically do not want to deploy physical appliances and want everything handled from their virtualization context. Similar changes in networking and storage have meant that the basic infrastructure is now completely in software running on generic hardware. This is the software defined data center. VPLEX has been no stranger to this conversation. Especially given the very strong affinity of VPLEX to VMware use-cases, customers have been asking us for a software only version of VPLEX. This is precisely what is being delivered with VPLEX/VE 2.1 software release.

This white paper reviews the following topics:

• VPLEX/VE Technology Overview • VPLEX/VE Architecture and Deployment • VPLEX/VE Use Cases • VPLEX/VE Performance Planning


VPLEX/VE Overview

EMC® VPLEX®/VE represents next-generation architecture for data mobility and information access. This architecture is based on EMC‘s 20+ years of expertise in designing; implementing and perfecting enterprise class intelligent cache and distributed data protection solutions.

VPLEX/VE for VMware® vSphere provides simplified block storage management and data mobility for VMware vSphere clusters. VPLEX/VE is exclusively managed using the VMware vSphere Web Client through a plug-in. The VPLEX/VE virtual appliance runs at each site or fault domain within an ESXi Cluster or VMware Metro Storage Cluster (vMSC). VPLEX/VE enables automatic data sharing and workload balancing along with HA and non-disruptive application mobility across sites. These benefits are achieved by providing a highly available shared block storage device to the ESX cluster. By default, the shared block storage device provided by VPLEX/VE will be formatted with VMFS and then consumed as a Distributed Datastore by the ESX cluster. By eliminating the need to move between datastores to vMotion from one ESXi host to another, VMotion and HA are now enabled across separate fault domains or sites with up to 10ms RTT latency between them.

Figure 1: VPLEX/VE within a 2 Site vSphere Cluster

VPLEX/VE is a two site (fault domain) solution. Comparing the hardware version of VPLEX Metro to VPLEX/VE we can see that the VPLEX directors have been converted into vDirectors. The VPLEX/VE vDirector configuration is called a ’4×4′ -- meaning deployment of four vDirectors at each site. From a configuration standpoint, that is analogous to two VPLEX engines on each site of a VPLEX Metro.

iSCSI iSCSI

VNXe 3150 / 3200

VPLEX/VE Site-1 vApp VPLEX/VE Site-2 vAppvSMSvDirector

VNXe 3150 / 3200

VPLEX/VE Distributed Datastore

Site-1 Site-2

ESXi

VMware Metro Storage Cluster

VMVM

vDirector

ESXivDirector

ESXivDirector

ESXi ESXi

VMVMVM

VMVMVM

vDirector

ESXivDirector

ESXivDirector

ESXivDirector

ESXi ESXi

VMVMVMVM

VMVMVMVM

VMVMVMVM

VMVMVM

IP SAN

VMVMVM

vSMS


Architecture and Deployment VPLEX/VE is deployed as vApp into an ESXi cluster managed by a single vSphere Server instance. VMware vSphere Web Client and Desktop Client do not have a concept of physical sites or fault domains within a cluster. One suggestion is to use host names that indicate which Site they are located in. It is incumbent on the VMware administrator to understand the physical location of each ESXi host in the cluster during VPLEX/VE deployment. The VPLEX/VE installation wizard will organize the vDirectors and vManagement Server as follows:

Figure 2: VPLEX/VE vDirector and vManagement Server Deployment

Within the ESXi cluster:

• ESXi hosts are organized into 2 Sites, each consisting of 4 or more hosts • 1 VPLEX/VE vApp is deployed per Site • Each VPLEX/VE Site consists of 4 Director virtual machines (vDirectors) • Each VPLEX/VE Site contains 1 Management Server virtual machine


Figure 3: VPLEX/VE within vSphere Web Client

VPLEX Witness

VPLEX/VE provides an intelligent quorum mechanism to allow VPLEX/VE to make a distinction between a site loss and a wan partition. This mechanism is known as VPLEX Witness. It consists of a single virtual machine that is deployed within a 3rd fault domain. VPLEX Witness communicates with Site 1 and Site 2 via independent IP networks.

Figure 4: VPLEX Witness Deployed in a 3rd Fault Domain

Technical Note: The VPLEX Witness virtual machine must reside on an ESXi host outside of the VMware cluster that contains the two VPLEX/VE Sites. The design goal is to run the VPLEX Witness VM in an isolated 3rd fault domain up to 1000ms (1 second) RTT from Site1 and Site 2.

vCenter ServerESXi Cluster

Site-1 ESXi Hosts

Site-2 ESXi Hosts

Site-1 Virtual Directors

Site-2 Virtual DirectorsSite-2 Virtual Mgmt Server

Site-1 Virtual Mgmt Server

Site-1 vApp

Site-2 vApp


VPLEX/VE allows the VMware administrator to select which Site will continue I/O operations during a dual wan-partition event. Setting the VM affinity to align with Site Bias for the virtual machines that comprise an application enables them to non-disruptively ride through dual wan partition events.

For Site failures, VPLEX/VE automatically guides the surviving/healthy site to continue I/O regardless of the Site Bias setting. No matter which site fails, IO carries on at the surviving site.

VPLEX/VE vApp Architecture

Each VPLEX/VE Site consists of 4 vDirectors running on independent ESXi hosts and 1 Management Server (vSMS). One vDirector per ESXi host ensures the VPLEX/VE vApp can survive at least 2 ESXi host failures before a Site is lost.

Figure 5: vDirector Deployment across ESXi Hosts

VPLEX/VE STORAGE CONCEPTS

Without VPLEX/VE ESXi hosts in a VMware cluster consume shared block storage devices from physical storage arrays that are within the same physical site. The ESXi hosts and the virtual machines at one site do not typically consume the iSCSI storage or datastores from another site. When attempting to vMotion a virtual machine across sites, the block storage device containing the datastore and corresponding files that make up the virtual machine would not be available to the ESXi host at the second site. This creates the need for Storage vMotion, to move the data across sites prior to using vMotion to move the virtual machine.

VPLEX/VE creates Distributed Datastores with backing physical block storage devices at both sites. VPLEX/VE distributed cache coherence feature ensures that the data is consistent and available in an active-active configuration at both sites. VPLEX/VE enables the creation of Distribute Datastores which are backed by distributed virtual volumes within VPLEX/VE. These Distributed Datastores are accessible to all ESXi hosts participating in the within the vSphere cluster. Virtual machines provisioned with

4 - vDirector VMs

ESXi-1 ESXi-2 ESXi-3 ESXi-4

Site-1VPLEX vApp 1

ESXi-5 ESXi-6 ESXi-7 ESXi-8

Site-2VPLEX vApp 2

VMware ESXi ClusterOne Logical Group of HostsvCenter Server

Two Physical Locations / Fault Domains

vSMS 4 - vDirector VMsvSMS

ESXi-n ESXi-m… …


Distributed Datastores enable vMotion across sites with no limitation due to local shared physical storage.

Figure 6: VPLEX/VE Storage I/O Path


VPLEX/VE Use Cases

VPLEX/VE allows VMware admins to combine ESXi infrastructure at disparate data centers or fault domains into a single pool of resources. Storage, CPU, Memory, and Network at two locations now becomes a single, larger, more resilient pool of IT infrastructure.

Block Storage High Availability

The AccessAnywhere feature of VPLEX/VE ensures cache-coherent active-active access to datastores across VPLEX/VE sites. The features of VPLEX/VE increase resiliency in the event of a site outage. The data is protected in the event of disasters or failure of components within data centers. With VPLEX/VE, the applications can withstand failures of storage arrays and site components. The VPLEX/VE components are not disrupted by a sequential failure of up to two vDirectors in a site. The failure of a VPLEX/VE site or a dual WAN partition event is tolerated to the extent that the site configured with site bias continues to access the storage infrastructure. This means that if a storage array is unavailable, another storage array configured under VPLEX/VE continues to serve the I/O.

Figure 7: VPLEX/VE Provides RAID-1 mirroring across Sites

Application HA

VPLEX/VE provides a shared block storage device that enables a single datastore to be accessible across sites. In doing so, the ESXi cluster can leverage all of the traditional functionality found within a single site across two sites separated by up to 10ms RTT latency. This means VMware HA can be applied to virtual machines within the ESXi cluster and provide automatic (hands-off) virtual machine restart in the event of a site loss or hardware failure.


Migration

VPLEX/VE simplifies data center block storage management and eliminates outages during data migrations between arrays or upgrading/maintaining array technology. With VPLEX/VE, a VMware Admin is able to:

• Perform non-disruptive array technology refresh tasks using the two-way data exchange between locations

• Create an active-active ESXi cluster configurations to achieve active use of resources at both sites • Provide instant access to data between data centers • Non-disruptively move between Datacenters up with up to 10ms RTT latency

Figure 8: Non-Disruptive Array Technology Refresh

Dynamic Workload Load Balancing across Sites • Dynamically move storage from busy arrays to idle arrays for better asset utilization • Use VMware DRS to balance virtual machine workloads within and across data centers

Figure 9: Cross-Site DRS and vMotion

Immediate Cross-Site vMotion

• Leverage vMotion without the need to use Storage vMotion when moving between sites.


VPLEX/VE Performance Planning

Summary

This section provides information about VPLEX/VE performance and some sample performance metrics. The information is straightforward, but fairly simplified. In other words, a number of simplifying assumptions have been made. Individual results WILL vary.

Background

Adding VPLEX/VE to a vSphere cluster provides increased block storage availability, flexibility, and capability. These benefits come at the expense of adding processing overhead / latencies associated with servicing I/O for the ESXi hosts running VPLEX/VE. Performance of the host system hardware and IP network infrastructure is, therefore, critical to the success of a VPLEX/VE deployment.

A representative vSphere cluster environment along with the corresponding performance data charts and tables are provided to illustrate what VPLEX/VE was able to achieve for various IO workloads. All numbers are based on measurements from the test environment with VPLEX/VE running against common IO profiles.

Hardware Configuration

The example vSphere cluster test configuration, which includes the VPLEX/VE Systems Under Test, or SUTs, has the following components:

• 4x4 VPLEX VE Metro configuration running on isolated ESX Server. The table below for performance information on the vDirectors.

• Cluster Storage: VNX iSCSI Array with LUNs evenly distributed across the two clusters.

• IP WAN network consisting of two iSCSI Ethernets.

Performance Attribute Description

CPU Cores 12 CPUs X 2.5 GHz

Processor Type Intel Xeon CPU ES-2640 at 2.5 GHz

Processor Sockets 2

Cores per Socket 6

Logical Processors 24

Hyper-Threading Active

Number of 1 GbE NICs 11

Figure 10: ESXi Test Host Specifications


Key Performance Indicators (KPIs) Five simulated application workloads were measured. These are the same application IO footprints that are part of the key performance indicators (KPIs) for our VPLEX appliance characterization.

Simulated Application Profiles

The following simulated application profiles were tested:

1. OLTP1 (mail application)

2. OLTP2 (small oracle application / database transactional)

3. OLTP2-HW (large oracle application / heavy-weight transactions)

4. DSS2 (database decision support)

5. DSS128K (multimedia streaming / large block read and writes)

These simulated application profiles are each a composite of five simple I/O profiles: Random Read Hits (rrh), Random Reads (Miss) (rr), Random Writes (rw), Sequential Reads (sr) and Sequential Writes (sw).

I/O Profile of Simulated Applications

The I/O size and proportion of each component I/O profile varies across the application profiles as detailed in the following tables.

OLTP1

I/O Profile

I/O Size (KB)

% of Total

rrh 4 40 rr 4 24 rw 4 16 sr 4 10 sw 4 10

OLTP2

I/O Profile

I/O Size (KB)

% of Total

rrh 8 20 rr 8 45 rw 8 15 sr 64 10 sw 64 10


OLTP2HW

I/O Profile

I/O Size (KB)

% of Total

rrh 8 10 rr 8 35 rw 8 35 sr 64 5 sw 64 15

DSS2

I/O Profile

I/O Size (KB)

% of Total

rrh 4 0 rr 4 15 rw 4 5 sr 64 70 sw 64 10

DSS128K

I/O Profile

I/O Size (KB)

% of Total

rrh 64 18 rr 64 18 rw 64 4 sr 128 48 sw 128 12

Notes on Selecting the Application Profile

In order to select the application profile to use, i.e., which is most like the application that will be used with VPLEX/VE; the following section will serve as a mini guide. The subtitles of the 5 apps are representative of their origin: OLTP1 is based on Exchange; OLTP2 on a typical Oracle OLTP app; OLTP2HW on the TPC-C benchmark (which emulates a heavy-weight OLTP application); DSS2 on a typical Decision Support app; and DSS128k on a multimedia distribution app.


While OLTP1 is the “lightest” application type, i.e., an OLTP1 transaction takes the fewest resources to execute, if the workload is specified in terms of KBps, then OLTP1 is also the “densest” application, i.e., it is able to saturate the CPU resources with the smallest bandwidth. That is because OLTP1 is made up of only small size (4 KB) I/Os. When comparing a given bandwidth’s worth of application demand, it should not be surprising when OLTP1 transactions saturate the system earlier than OLTP2HW transactions. The former will be doing many more IOps.

The correct application to select is the one that is predominately like the application workload that will be deployed onto VPLEX/VE storage. And the correct metric to size the workload (IOps or KBps) is the one for which the most environment specific data is available. Experience shows that the app estimates tend to be too conservative. Pick the average or (better) median case, not the “one-time-highest-peak-ever-seen-in-a-1-minute-interval.”

Testing Methodology

Type of Measurements

The types of performance measurements are presented in this document are called ‘Steady-State Performance’ or ‘Steady-State Latency’ Curves. Initial ‘peak’ workload is used to determine the workload range (‘peak’ = 100% duty cycle). A series of offered workloads at various duty cycles (10%, 30%, 50%, 70%, and 99% of ‘peak’) is run, attaining a steady-state measurement of each. This type of run is useful for creating more realistic workloads, or at least for measuring latencies by plotting the observed load against the observed latency. The latency measure at ‘peak’ is not steady-state, and, therefore not a “real” or useful latency.

Data Charts The charts provide an easy way to lookup the vApp vCPU utilization (%) and the latencies (Average Response Time) by picking either the I/O per second or KB per second on the X-axis and following up to the relevant curve and across to the relevant Y-axis. The default IP network MTU of 1500 is shown. Testing did not show a significant difference in performance when using an MTU of 9000. For example: 65,000 OLTP1 I/O per second would run at ~ 40% utilization (green line) and have about 2.3 ms response time (red line).

Technical Note: For the following charts, WAN RTT latency between Sites was set to 0 milliseconds. VPLEX/VE uses write-through caching, so the actual WAN RTT latency for the environment should be added each of the Average RT (ms) data points on the chart for the best overall estimates.


- 100 200 300 400 500

0

2

4

6

8

10

0%

20%

40%

60%

80%

100%

0 20,000 40,000 60,000 80,000 100,000 120,000 140,000

KB per second

Average Response Time (m

s) Pe

rcen

t Util

izatio

n

I/Os per second

OLTP1 IOps mtu:1500 avgRT mtu:1500

- 200 400 600 800 1,000 1,200 1,400

0

2

4

6

8

10

0%

20%

40%

60%

80%

100%

0 10,000 20,000 30,000 40,000 50,000 60,000 70,000

KB per second


s) Pe

rcen

t Util

izatio

n

I/Os per second

OLTP2 IOps mtu:1500 avgRT mtu:1500


- 200 400 600 800 1,000

05101520253035

0%

20%

40%

60%

80%

100%

0 10,000 20,000 30,000 40,000 50,000 60,000

KB per second


s

Perc

ent U

tiliza

tion

I/Os per second

OLTP2HW IOps mtu:1500 avgRT mtu:1500

- 5,000 10,000 15,000 20,000 25,000 30,000 35,000

0

2

4

6

8

10

0%

20%

40%

60%

80%

100%

0 500 1,000 1,500 2,000

I/Os per second


s) Pe

rcen

t Util

izatio

n

KBs per second

DSS2 KBps mtu:1500 avgRT mtu:1500


Conclusion Adding VPLEX/VE to a new or existing vSphere cluster provides increased storage resiliency, flexibility, and functionality, but it does come with a cost. Though minor, the expense of added I/O processing overhead / latencies when servicing IO from must be accounted for during the initial planning process. Performance of the underlying host hardware system is the key to getting the desired result. This document reviewed VPLEX/VE architecture, use cases, and performance characteristics. Data charts and a test methodology were provided that can be used to estimate VPLEX/VE ability to support a given IO workload. The estimates are based on measurements from a representative host hardware and IP network configuration running various well known IO profiles. A number of environmental and applications assumptions were made to simplify the overall discussion. For this reason, the results indicated are only a guide and may not represent the actual results obtained in specific VPLEX/VE environments.


VPLEX References The following reference documents are available at http://Support.EMC.com:

• EMC VPLEX/VE Site Preparation Guide • EMC VPLEX/VE Release Notes • EMC VPLEX/VE Security Configuration Guide • EMC VPLEX/VE Configuration Worksheet • EMC VPLEX/VE CLI Guide • EMC VPLEX/VE Product Guide • EMC VMware ESXi Host Connectivity Guide

Supplemental VMware References

• The CPU Scheduler in VMware vSphere 5.1:

http://www.vmware.com/files/pdf/techpaper/VMware-vSphere-CPU-Sched-Perf.pdf

• Performance Best Practices for VMware vSphere 5.1: http://www.vmware.com/pdf/Perf_Best_Practices_vSphere5.1.pdf

• Best Practices for Running VMware vSphere on iSCSI: http://www.vmware.com/files/pdf/iSCSI_design_deploy.pdf

• Best Practices for Performance Tuning of Latency-Sensitive Workloads in vSphere VMs: http://www.vmware.com/files/pdf/techpaper/VMW-Tuning-Latency-Sensitive-Workloads.pdf

• Deploying Extremely Latency-Sensitive Applications in VMware vSphere 5.5 - Performance Study: http://www.vmware.com/files/pdf/techpaper/latency-sensitive-perf-vsphere55.pdf

For more information: Explore and compare the latest VPLEX products in the EMC Store

http://support.emc.com/



http://www.vmware.com/pdf/Perf_Best_Practices_vSphere5.1.pdf

http://www.vmware.com/files/pdf/iSCSI_design_deploy.pdf

http://www.vmware.com/files/pdf/techpaper/VMW-Tuning-Latency-Sensitive-Workloads.pdf

http://www.vmware.com/files/pdf/techpaper/VMW-Tuning-Latency-Sensitive-Workloads.pdf

http://www.vmware.com/files/pdf/techpaper/latency-sensitive-perf-vsphere55.pdf

http://www.vmware.com/files/pdf/techpaper/latency-sensitive-perf-vsphere55.pdf

https://store.emc.com/Product-Family/EMC-Store-Products/c/EMCStoreProducts?q=%3Arelevance%3AProductFamily%3AVPLEX+Products&facetselected=true

emc vplex virtual edition: use cases and performance planning · vplex virtual edition : ... the...

Documents