xenserver - understanding snapshots (v1.1)
TRANSCRIPT
WHITE PAPER | Citrix XenServer: Understanding Snapshots
www.citrix.com
Citrix XenServer
Understanding Snapshots
Page 2
Contents
Introduction ........................................................................................................................................................ 3
Snapshot Behavior by Storage Repository Type ........................................................................................... 4
1. Volume-based Storage Repositories ...................................................................................................................................... 4
2. File-based Storage Repositories ............................................................................................................................................ 10
3. StorageLink Gateway Storage Repositories ........................................................................................................................ 16
Advanced Coalescing Scenarios ..................................................................................................................... 20
Best Practices .................................................................................................................................................... 22
Appendix A: Behavior in volume-based storage repositories upgraded from 5.0 to 5.5. ..................... 23
Page 3
Introduction
Snapshots are a very important feature of XenServer, as they provide a “point in time” disk state
that is useful for virtual machine backup as well as in test & development environments. Snapshot
behavior and storage consumption is determined by the type of storage repository (SR) used. As it
pertains to the behavior of snapshots, there are three categories of SR:
- Volume-based: Local disks, iSCSI and Hardware HBA (typically Fibre channel)
- File-based: NFS
- StorageLink: Provided in Citrix Essentials for XenServer
Firstly, it’s important to understand how storage space is consumed and reported by XenServer
virtual machines. When virtual machines are created in XenServer, the term virtual disk image (VDI)
is used to refer to the disks. For example, a Linux VM with a 10 GB disk has one XenServer VDI.
The storage space consumed and reported will differ based on the SR containing the VDI.
In the subsequent sections, VDI storage and snapshot behavior will be discussed in detail for each
type of storage repository.
Snapshot Behavior by Storage Repository Type
1. Volume-based Storage Repositories
This section describes the behavior for volume-based (LVHD) storage repositories. Volume-based storage repositories include iSCSI,
Hardware HBA (typically Fibre Channel), and local disks.
Note: this section describes the behavior of storage repositories created in XenServer 5.5. The behavior of SR’s upgraded from earlier
versions of XenServer is described in Appendix A.
We will use an example of an empty iSCSI storage repository (SR) with just under 20 GB of space. After the SR is created in XenCenter, the
screen shot below shows what is reported in XenServer. The only space that is consumed (4 MB) is by SR formatting information.
Page 5
We’ll create a Linux VM with one 5 GB disk to illustrate how space is consumed.
After the VM and its disks are created, the
“Storage” tab for the SR will appear as shown.
Here is how the SR space utilization will be
reported in XenCenter. With the iSCSI SR
the entire 5 GB of space required by the VDI
is consumed at the time it is provisioned. This
is because iSCSI (and Hardware HBA) SRs are
“thick provisioned.” (Note: a future release of
XenServer will also enable thin provisioning,
similar to NFS SRs today).
Here is the detailed VDI information shown
from the CLI. Note that with the iSCSI SR,
the physical utilization is slightly greater than
the virtual size. This is because the physical
utilization calculation includes the VHD
header and metadata information as well (not
just the raw disk data).
Page 6
After the snapshot is performed, here is what is shown in the consoles.
Snapshot
SR disks
SR level space consumption
Page 7
At this point it is instructive to examine the underlying VDIs, using the xe vdi-list command. To explain the creation of these additional VDIs, we’ll refer to them as “A”, “B”, and “C.” The snapshot performed the following operations:
VDI A (now provided the name label “base copy”) became the “parent node” and received a new UUID. The physical utilization of this VDI will be equivalent the actual utilization by the VM guest operating system. In this case, the Linux OS is only using about 14% of the 5 GB disk. Because of this XenServer will deflate the VDI to conserve storage space (Note: deflation occurs as a background process and is not immediately reflected in the physical consumption of the VDIs). If the guest OS were using the full 5 GB of the allocated disk, the physical utilization of the VDI would also be 5 GB. It’s important to recognize that a VM snapshot could require as much as 100% of the virtual size (i.e. allocated size) of the base VDI(s) when using volume-based storage repositories.
A new VDI B is created to contain subsequent writes, and is provided the UUID formerly assigned to VDI A (the one starting with 52d434b0-). This process is called “VDI remapping.” In this manner, the VM continues to reference the same UUID as before the snapshot was taken.
A new VDI C is created as a child node of VDI A to support potential writes to the snapshot.
CLI commands executed: 1. “xe sr-list” to get sr-uuid 2. xe vdi-list sr-uuid=<sr-uuid from step 1> params=uuid,name-label,name-
description,virtual-size,physical-utilisation,is-a-snapshot,sr-name-label
Page 8
After the snapshot is deleted, here is what is shown in the consoles.
SR disks.
This is the SR level space consumption screen. Note that, by default, the storage utilization (now at 5.7GB) on the SR doesn’t return all the way to 5 GB, as it was prior to the snapshot being created. In cases where the guest OS uses the full amount of disk space allocated (5 GB), the amount of space consumed on the SR at this point could be as high as 10 GB (2 x 5 GB). Understanding this variance is important for storage planning considerations.
This screen shows the VDI details obtained from the xe vdi-list command. At this point the VDI representing the snapshot has been deleted. Due to the UUID remapping process performed during the snapshot operation, there are two VDI’s associated with the VM. The VDI with the original UUID (52d434b0-) is still referenced by the VM when writes are performed to the disk. The second VDI (with the label “base copy”) is not visible within XenCenter. Note: with XenServer 5.5 Update 1 and later, the “base copy” VDI can be manually coalesced with the active VDI to reclaim storage space. Please reference the following knowledge center article for more information: CTX123400.
Page 9
Alternatively, the result of these steps can be illustrated below:
Here is a key to the illustrations below
After VM creation, the VM has one VDI of 5 GB. The VDI is thick provisioned and is assigned a UUID.
After snapshot is created, the following steps occur:
VDI A contains data up to the point of the snapshot. This VDI is deflated to conserve space.
UUID remapping from A to B occurs; VDI B becomes the active node
VDI C is empty (except for VHD header information)
After the snapshot is deleted, the following steps occur:
VDI C is deleted.
VDI A does not change, and continues to have data up to the point of the snapshot (even though the snapshot was technically deleted)
VDI B continues to be used for writes, and will grow over time.
Page 10
2. File-based Storage Repositories
This section describes the behavior for file-based storage repositories, typically NFS.
Note: This section also applies to local disks configured for ext3 according to the steps outlined in CTX120867
We will use an example of an empty NFS storage repository (SR) with just under 20 GB of space. After the SR is created in XenCenter, the
screen shot below shows what is reported in XenServer. The only space that is consumed (576KB) is by SR formatting information.
Page 11
We’ll create a Linux VM with one 5 GB disk to illustrate how space is consumed on the SR.
After the VM and its disks are created, the
“Storage” tab for the SR will appear as follows,
with the 5 GB disk for the VM.
Here is how the SR space utilization will be
reported in XenCenter. As shown here, 644 MB
of space is actually consumed even though 5 GB
is allocated. This is why the VDI is reported as
“14% on disk” in the previous screen shot. The
difference between actual and allocated storage
utilization is explained by the XenServer NFS SR’s
“thin provisioning” capability. This means that
space not currently required by the VDI is only
allocated, but not actually consumed until it is
required by the VM.
From the XenServer CLI, we can access deeper
information about the VDI using the “xe vdi-list”
command. Here is the result of the command run
against the VDI for the Linux VM.
CLI commands executed: 1. “xe sr-list” to get sr-uuid 2. xe vdi-list sr-uuid=<sr-uuid from step 1> params=uuid,name-label,name-description,virtual-
size,physical-utilisation,is-a-snapshot,sr-name-label
Page 12
After taking a snapshot of the VM, the table below shows the information that appears in the consoles:
The size reported in the VM snapshot tab is 14.5 KB. (Note that this value should be referenced once background coalescing processes have completed)
Here are the disks shown in the SR storage tag, and the overall storage reported. Note that the “base copy” virtual disk (explained later) is not shown here.
The space used on the SR is now reported as 644.1 MB, slightly larger than before due to the space consumed after performing the snapshot.
Page 13
At this point it is instructive to examine the underlying VDIs, using the xe vdi-list command. To explain the creation of these additional VDIs, we’ll refer to them as “A”, “B”, and “C.” The snapshot performed the following operations:
VDI A (now provided the name label “base copy”) became the “parent node” and received a new UUID. The physical utilization for VDI A is identical to what the VM was consuming prior to the snapshot.
A new VDI B is created to contain subsequent writes, and is provided the UUID formerly assigned to VDI A (the one starting with 67f45eaf-). This process is called “VDI remapping.” In this manner, the VM continues to reference the same UUID as before the snapshot was taken. The physical utilization by VDI B is small; only the VHD header information consumes space on the NFS storage repository.
A new VDI C is created as a child node of VDI A and is the snapshot container (is-a-snapshot=true)
CLI commands executed: 1. “xe sr-list” to get sr-uuid 2. xe vdi-list sr-uuid=<sr-uuid from step 1> params=uuid,name-label,name-description,virtual-size,physical-
utilisation,is-a-snapshot,sr-name-label
Page 14
Next, it’s important to understand the default behavior when a snapshot is deleted. Here is what is shown in the consoles:
Show disks in XenCenter. Note that the disk shown here is different than what appeared after the VM creation. This screen now references the “remapped” VDI which is used for writes.
The space used on the SR is now reported as 644 MB, back to what it was before the snapshot was taken.
This screen shows the VDI details obtained from the xe vdi-list command. At this point the VDI representing the snapshot has been deleted. Due to the UUID remapping process performed during the snapshot operation, there are still two VDI’s associated with the VM. The VDI with the original UUID (67f56eaf-) is still referenced by the VM when writes are performed to the disk. The second VDI (with the label “base copy”) actually contains the bulk of the data on the VM disk, hence the much higher physical utilization than the first VDI. This “base copy” VDI is not visible within XenCenter. Even though the virtual size of these two VDIs is 5 GB, it is important to recognize is that the physical utilization is essentially the same as it was prior to the snapshot operation.
CLI commands executed: 1. “xe sr-list” to get sr-uuid 2. xe vdi-list sr-uuid=<sr-uuid from step 1> params=uuid,name-label,name-
description,virtual-size,physical-utilisation,is-a-snapshot,sr-name-label
Page 15
Alternatively, the result of these steps can be illustrated below:
Here is a key to the illustrations below
After VM creation, the VM has one VDI. The VDI is thin provisioned and is assigned a UUID.
After snapshot is created, the following steps occur:
VDI A contains data up to the point of the snapshot
VDI B and C are empty (except for VHD header information)
UUID remapping from A to B occurs.
After the snapshot is deleted, the following steps occur:
VDI C is deleted.
VDI A does not change, and continues to have data up to the point of the snapshot (even though the snapshot was technically deleted)
VDI B continues to be used for writes, and will grow over time.
Page 16
3. StorageLink Gateway Storage Repositories
The StorageLink Gateway is a service that acts as a broker between XenServer and a block-based storage system. In this example, we will
illustrate the behavior of snapshots performed against a NetApp system, brokered by StorageLink Gateway via iSCSI. One of the benefits of
StorageLink Gateway is that the native SAN functionality for snapshots is leveraged, resulting in optimized performance and storage
utilization.
Page 17
1. After creating Linux VM with a single 5GB disk, here is what is reported in the consoles.
This is the storage repository
general information shown in
XenCenter.
This is the storage repository disk
information shown in XenCenter.
This screen shows the VDI details
obtained from the command line.
This screen shows the NetApp
FilerView LUN information. We
can see that a 5 GB LUN has been
automatically provisioned for the
VM.
Page 18
2. After a VM snapshot is performed
This is the storage repository disk
information shown in XenCenter. Note
that a new disk (VDI) is created with the
label “Backup (snapshot)”
This screen shows the VDI details
obtained from the command line. A new
VDI is listed with “is-a-snapshot”
property set to “true.” The VDI behavior
of the snapshot is notably different from
NFS or iSCSI/HBA SRs since only one
additional VDI is created. This is because
in this case, the native NetApp snapshot
facility is used.
This screen shows the NetApp FilerView
LUN information. This view shows the
LUN created for the snapshot with
“snap00” appended to the label.
Page 19
3. After snapshot is deleted, here is what is shown in the consoles
This is the storage repository disk
information shown in XenCenter. The
snapshot disk no longer appears.
This screen shows the VDI details
obtained from the command line. Note
that only the base VDI remains and the
VDI representing the snapshot has been
deleted.
This screen shows the NetApp FilerView
LUN information. The base LUN
remains and the snapshot LUN has been
deleted. In this manner, StorageLink
provides the easiest and most efficient
snapshot mechanism from a storage
utilization standpoint.
Page 20
Advanced Coalescing Scenarios
XenServer includes the ability to collapse VDI chains to eliminate redundant nodes that result from creation and deletion of snapshots. This
process is known as coalescing, and occurs as a background process. In this section, we will describe the coalescing process in detail.
Example 1:
Here is a key to the illustrations below
We’ll take an example of a VM with two
snapshots, VDI C and D.
When snapshot C is deleted, parent B is no
longer required and will be coalesced into A. In
the end, three VDIs will be retained for the
VM.
Page 21
If the remaining snapshot E is deleted, two
VDIs will remain
When snapshot G is subsequently taken, parent
D is coalesced into A+B. Three VDIs will exist
for the VM at this point.
Page 22
Best Practices
Use of StorageLink is preferred when snapshots will be used extensively in a XenServer virtualization deployment. This assumes that the array
in question is supported by StorageLink and the necessary array snapshot capabilities are available. Because of the tight integration with the
specific array functionality, StorageLink provides the best overall performance and the most efficient storage utilization for snapshots.
Volume-based approaches such as iSCSI and Hardware HBA (Fibre Channel) offer excellent performance and broad support by storage
vendors. Because these XenServer storage repositories use thick provisioning today, they will require more space for snapshots than with file-
based storage repositories. This space consumption will need to be planned for accordingly. Volume-based storage repositories upgraded
from 5.0 to 5.5 will require the greatest amount of space because the deflation capability is only available on native 5.5 storage repositories (see
Appendix A for a full explanation). Storage space can be conserved by migrating virtual machines from 5.0 volume-based storage repositories
to native 5.5 volume-based storage repositories.
As it pertains to storage utilization by snapshots, file-based approaches like NFS are optimal because of XenServer’s thin provisioning
implementation. If the best possible performance is critical, it would be advisable to use optimized NFS systems such as those offered by
vendors such as NetApp.
Page 23
Appendix A: Behavior in volume-based storage repositories upgraded from 5.0 to 5.5.
Here is a key to the illustrations below
After VM creation, the VM has one VDI of 5 GB. The VDI is thick provisioned and is assigned a UUID.
After snapshot is created, the following steps occur:
VDI A contains data up to the point of the snapshot. Because the storage repository was upgraded from 5.0 to 5.5, it remains in raw LVM format and cannot be deflated. As a result, this VDI will require 5 GB of space.
UUID remapping from A to B occurs; VDI B becomes the active node
VDI C is empty (except for VHD header information)
After the snapshot is deleted, the following steps occur:
VDI C is deleted.
VDI A does not change, and continues to have data up to the point of the snapshot (even though the snapshot was technically deleted)
VDI B continues to be used for writes, and will grow over time.
Page 24
About Citrix
Citrix Systems, Inc. (NASDAQ:CTXS) is the leading provider of virtualization, networking and software as a service technologies for more than 230,000 organizations
worldwide. Its Citrix Delivery Center, Citrix Cloud Center (C3) and Citrix Online Services product families radically simplify computing for millions of users, delivering
applications as an on-demand service to any user, in any location on any device. Citrix customers include the world’s largest Internet companies, 99 percent of
Fortune Global 500 enterprises, and hundreds of thousands of small businesses and prosumers worldwide. Citrix partners with over 10,000 companies worldwide
in more than 100 countries. Founded in 1989, annual revenue in 2008 was $1.6 billion.
©2009 Citrix Systems, Inc. All rights reserved. Citrix®, Access Gateway™, Branch Repeater™, Citrix Repeater™, HDX™, XenServer™, XenApp™, XenDesktop™ and
Citrix Delivery Center™ are trademarks of Citrix Systems, Inc. and/or one or more of its subsidiaries, and may be registered in the United States Patent and
Trademark Office and in other countries. All other trademarks and registered trademarks are property of their respective owners.