vmworld 2015: explaining advanced virtual volumes configurations
TRANSCRIPT
Explaining Advanced Virtual Volumes ConfigurationsKen Werneburg, VMware, Inc
Andy Banta, SolidFire
STO5074
#STO5074
• This presentation may contain product features that are currently under development.
• This overview of new technology represents no commitment from VMware to deliver these features in any generally available product.
• Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.
• Technical feasibility and market demand will affect final delivery.
• Pricing and packaging for any new technologies or features discussed or presented have not been determined.
Disclaimer
CONFIDENTIAL 2
3
vSphere Virtual VolumesManagement & Integration Framework for External Storage
vSphere Virtual
Volumes
The Basics
• Virtualizes SAN and NAS devices
• Virtual disks are natively represented on arrays
• Enables VM granular storage operations using array-based data services
• Storage Policy-Based Management enables automated consumption at scale
• Supports existing storage I/O protocols (FC, iSCSI, NFS)
• Industry-wide initiative supported by major storage vendors
• Included with vSphere
CONFIDENTIAL
CONFIDENTIAL 4
vSphereVirtual Volumes
SAN / NAS
Vendor Provider (VASA)
Control Path
Control Path
Storage Policies
Access
Capacity
Published Capabilities
Snapshot
Replication
Deduplication
QoS
Virtual Datastore
Storage Admin
vSphere Admin
VVOLs
DataPathProtocol Endpoint PE
vSphere Virtual Volumes Architecture
CONFIDENTIAL 5
VVols are Just Storage “Stuff”
CONFIDENTIAL 6
• Virtual Volumes – Virtual machine objects stored natively on the
array– No file system on-disk formatting required
• There are five different types of recognized Virtual Volumes:– CONFIG – vmx, logs, NVRAM, log files, etc
– DATA – VMDKs
– MEM – Snapshots
– SWAP – Swap files
– Other – Vendor solution specific
vSphere Web Client View
vvol
vSphere Virtual Volumes
CONFIDENTIAL 7
Storage Containers are Where “Stuff” Lives
VM
VM
VM
VM
CONFIDENTIAL 8
Protocol Endpoints are How You Get the “Stuff”
iSCSI Target
CONFIDENTIAL 9
Bindings are How the “Stuff” Finds Its Way to You
Truck A
Truck B
Truck C
VASA Provider
10
CONFIDENTIAL 11
Overview• VVols motivations
– Separation of VMware and storage management– Better use of storage capacity and network bandwidth– The upshot: better scalability and control
• Configuration examples– Memory overprovisioning– Multi-tenant use of VVols– Disaster Recovery configurations
CONFIDENTIAL 12
Motivations• Ease of management
• Built on top of existing protocols
• Datastore scalability– Less wasted storage space– No file system contention– Logical Storage Container concept as a replacement
• Scalability through less SAN bandwidth– No vSphere data mover for normal operations– Less Storage vMotion– Less need for SDRS and SIOC
CONFIDENTIAL 13
Different Management Perspectives
• What do the Admins see?
• How are the storage containers set up?
• What does the vSphere Admins see?
• Why are datastores still required?
vSphere Web Client
Storage Management UI
Datastore
Storage Container
CONFIDENTIAL 14
Ease of Management
• What do the Admins see?
• How are the storage containers set up?
• What does the vSphere Admins see?
• Why are datastores still required?
Storage policies
vSphere Web Client
Storage Management UI
Datastore
Storage Container
Storage Capabilities
virtual volumes
virtual machines
CONFIDENTIAL 15
Virtual Volumes
VM objects view from a storage container on an arrayVM objects view from a datastore
vSphere Web Client Storage Management UIvSphere Admin View Storage Admin View
VVol Storage UI
CONFIDENTIAL 16
Multi-Protocol SupportProtocol Endpoints• Access points that enable communication
between ESXi hosts and storage array systems– Part of the physical storage fabric– Created by Storage administrators
Scope of Protocol Endpoints• Compatible with all SAN and NAS Protocols:
- iSCSI- NFS v3 - FC- FCoE
• A Protocol Endpoint can support any one of the protocols at a given time
• Existing multi-path policies and NFS topology requirements can be applied to the PE
Why Protocol Endpoints?
SAN / NAS
Virtual Datastore
DataPathProtocol Endpoint PE
vSphereVirtual Volumes
Storage Container
CONFIDENTIAL 17
Scalability by Separating Datastores and Protocol Endpoints• Today, there are different types of logical
management constructs to store VMDKs/objects:– NFS mount points– IP or block based datastores
• Datastores serve two purposes today:– Endpoints – receive SCSI or NFS reads,
write commands– Storage Container – for large number of VMs
metadata and data files• Differences between Protocol Endpoints
and Datastores:– PEs no longer stores VMDKs but it only becomes
the access point – Now you wont need as many datastores or mount
point as before• Certain offloading operations will be done via
VASA and other will be done using the standard protocol commands
vSphere
storage fabric
PEprotocol endpointSCSI: proxy LUNNFS: mount-point
datastore = protocol endpoint + storage container
storage system
1 VVol (storage container)Per VMDK
One entity on the fabric
CONFIDENTIAL 18
SAN Bandwidth Used Without VVols
VASA API Compatible Array
2
vSphere
VMFS VVOLs
1
vSphere Admins
Migrate VM from VMFS toVVOL datastore
software data mover implementation
CONFIDENTIAL 19
Operations Offloaded with VVols, Using Less SAN Resources
vSphere
VMFS VVOLs
vSphere Admins
offload to array
offload to array
Virtual Machine Operation Offloaded• Virtual Machine provisioning
• Virtual Machine deletes
• Virtual Machine full clones
• Virtual Machine Linked Clones
• Virtual Machine Snapshots
• Storage vMotion
CONFIDENTIAL 20
Advantages All Day Long with VVols
Device-centric Silos
✖ Static classes of service
✖ Rigid provisioning
✖ Lack of granular control
✖ Frequent data migrations
✖ Time consuming processes
✖ Lack of automation
✖ Slow reaction to request
Complex Processes
VI Admin
Storage Admin
App Admin
✖ Not commodity
✖ Low utilization
✖ Overprovisioning
Specialized Costly HW
CONFIDENTIAL 21
Real Life Examples
Memory overprovisioning
Multi-tenant configuration
Disaster Recovery
Snapshot management
CONFIDENTIAL 22
Using Storage Capabilities and VM Storage Policies• Storage Capabilities – are array based features
and data services specifications that capture storage requirements that can be satisfied by a storage arrays advertised as capabilities
• Storage capabilities define what an array can offer to storage containers as opposed to what the VM requires
• Arrays Storage Capabilities are advertises to vSphere through the Vendor Provider and VASA APIs
• In vSphere Storage Capabilities are consumed via VM Storage Policy constructs
• VM Storage Policies is a component of the vSphere Storage Policy-based management framework (SPBM)
SPBM
object manager
virtual disk
Datastore ProfileVM Storage Policy
vSphere VM Storage Policy Management Framework
Storage Capabilities for Storage Array
Access
Capacity
Published CapabilitiesSnapshot
Replication
Deduplication
QoS
Virtual Datastore
CONFIDENTIAL 23
Special Handling of Swap VVols• If you’re running a memory over-provisioned system, prioritize swap VVols
• Swap volumes need the lowest latency and highest throughput priorities– If your VM can’t use its memory, it can’t do much of anything useful
• If the swap volumes are being used heavily, it might be time to reconfigure vSphere
CONFIDENTIAL 24
Multi-tenant use of VVols• Use Storage Containers as logical separation
• Use individual policies and capabilities as VMs require– Different policies for different VMs– Different policies for different requirements in the VM
• Desktop• Database• Log
Storage Containers
Access
Capacity
Published CapabilitiesSnapshot
Replication
Deduplication
QoS
CONFIDENTIAL 25
Disaster Recovery• Replication policies
• Failover techniques
Published Capabilities
Snapshot
Replication
Deduplication
Encryption
vSphereStorage Policy-Based Mgmt.
Virtual Volumes
Storage PolicyCapacity
Availability
Performance
Data Protection
Security
PEVASA Provider
PE
CONFIDENTIAL 26
Snapshots• Snapshots are a point-in-time, copy-on-write
image of a Virtual Volume with a different ID from the original
• Virtual Volumes snapshots are useful when creating:– A quiesced copy for backup or archival purposes,
creating a test and rollback environment for applications, instantly provisioning application images, and so on
• Two type of snapshots supported:– Managed Snapshot – Managed by ESX
• A maximum of 32 snapshot are supported for fast clones
– Unmanaged Snapshot – Managed by the storage array• Maximum snapshot dictated by the storage array
Managed Snapshot - vSphere
Unmanaged Snapshot - Array
CONFIDENTIAL 27
vSphere Virtual Volumes Supported Features Supported vSphere Features
• SPBM
• Thin Provisioning
• Linked Clones
• Native Snapshots
• Protocols: NFS3, iSCSI, FC, FCoE
• View Storage Accelerator (CBRC)
• vMotion
• SvMotion
• DRS
• XvMotion
• vSphere SDK (VC APIs)
• VDPA/VDP
• View
• vRealize Operations
• vRealize Automation
• Stateless / Host Profiles
Published Capabilities
Snapshot
Replication
Deduplication
Encryption
vSphereStorage Policy-Based Mgmt.
Virtual Volumes
Storage Policy
Capacity
Availability
Performance
Data Protection
Security
PEVASA Provider
PE
CONFIDENTIAL 28
The Benefits of vSphere Virtual VolumesA More Efficient Operational Model For External Storage
Improves Resource Utilization
• Increase capacity utilization• Eliminate overprovisioning• Reduce management overhead
• Eliminate inefficient handoffs between VI and Storage Admin
• Faster storage provisioning through automation
• Simplified change management through flexible consumption
• Self-service provisioning via cloud automation tools
Simplifies Storage Operations
• Leverage native array-based capabilities
• Fine control at the VM level• Dynamic configuration on the fly• Ensure compliance through policy
enforcement using automation
Simplifies Delivery of Service Levels
Wrap upVvols was motivated by manageability, transparency and scalability.
If you don’t care about those reasons, they can help you manage configurations that are otherwise difficult to manage
29
30
31
Explaining Advanced Virtual Volumes ConfigurationsKen Werneburg, VMware, Inc
Andy Banta, SolidFire
STO5074
#STO5074