ibm smartcloud enterprise storage demo

© 2011 IBM Corporation

IBM SmartCloud EnterpriseStorage Availability Areas and Dynamic Disk Attach / Detach Demowww.ibm.com/cloud/enterprise/

Alex Amies, Solution Architect, IBM

May 2012

© 2011 IBM Corporation2

Agenda

• Concepts and Scenarios

• Demos• UI• Command line• Java API


Agenda




Concepts

The demo illustrates the use of storage availability areas and dynamic attach / detach features in an end-to-end data back-up and verification scenario.

A storage volume in SmartCloud Enterprise is a virtualized block storage unit. It can be mounted to a virtual machine as a storage device and formatted with a file system to store data. Other terms sometimes used to refer to the same concept in SCE are storage unit or just volume. Cloud users can use storage availability areas to store data in different areas within the same data center for the back-up purposes.

When creating a storage volume a storage availability area can optionally be chosen for the volume to be provisioned to. If no storage availability is chosen then one will be assigned by the system. When cloning a storage volume it may be cloned to a different data center.

A storage volume may be attached to a virtual machine instance after creation using dynamic attach. This is in contrast to static attach where a storage volume is attached to a disk during the virtual machine creation time. The storage volume may be detached without deleting the virtual machine instance using dynamic detach.


Scenarios for Data Backup

Whole volume backup

1.Create a storage volume – choose a storage area or accept the default

2.Create a virtual machine

3.Attach the volume to the virtual machine (separate step, i.e. dynamically)

4.Add data to the volume

5.Detach the volume

6.Backup the data by cloning the volume

7.Re-attach the volume and continue using it

8.Note: you must dynamically attach and detach for this scenario to work

Data backup

1.Create two volumes each on different availability areas

2.Create a virtual machine attaching both volumes (single step or two steps, i.e. either statically or dynamically)

3.Use one volume for active data storage and retrieval

4.Back-up data by copying to the other volume

5.Note: need to manage your own data with mounted volumes in this case


Scenarios for Image Import / Copy

Image Import

1.Create a storage volume


3.Attach the volume to the virtual machine (separate step, i.e. dynamically)

4.Upload a virtual machine image OVF bundle to the volume using the native tools in the virtual machine

5.Detach the volume

6.Import the image to the cloud using the image import API or command line

Image Copy

1.Create a storage volume

2.Copy a virtual machine image OVF bundle to the volume from the cloud image repository using the SCE image copy command or API


4.Attach the volume to the virtual machine (i.e. dynamically)

5.Copy the image OVF bundle from the volume to a location outside the cloud using the native tools of the virtual machine


Agenda




User Interface in web portal

The storage availability area options appear in the storage volume provisioning wizard, in the storage volume table, and in the clone storage volume dialog.

Dynamic attach and detach are not available in the user interface. However, the changes in state for triggered by their use via the command line tool and API are apparent in the user interface.


Create Storage VolumeStorage availability areas are displayed for the supported data centers in the storage volume provisioning wizard.


Detailed View of Storage VolumeAfter a storage volume is provisioned then the availability area that the volume is provisioned to will be displayed.


Clone Storage VolumeWhen cloning a storage volume you will be presented the choice of availability area for supported data centers.


Agenda




Demo – backup storage volume to a different availability area

1.Create a volume in a known storage availability area

a)Query to discover existing availability areas

b)Find storage volume offering ID, available sizes, and supported formats

c) Create storage volume on a selected availability area

d)Check status, wait until provisioning is complete

2.Use the volume to store data

a)Find supported sizes and check support for dynamic attach / detach for selected virtual machine image

b)Determine SSH key to use for new virtual machine instance

c) Create an instance with selected image and supported size

d)Check state of instance - wait for instance to be created

e)Dynamically attach storage volume to instance

f) SSH to instance, partition, format, and mount the volume. Now that it is ready to use, save data to it.

3.Back the volume up

a)Detach the volume

b) Clone the volume to another availability area to back up the data

c) Dynamically attach the cloned storage volume to the instance to verify that the data has been backed up successfully


Demo with Command line toolUse Windows for demo, Linux is similarStep 1a: create password file and list storage availability areas

# Create a password file

> ic-create-password.cmd -u [email protected] -p *** -w unlock -g mykey.ext

# Find storage availability areas with the describe locations command

> ic-describe-locations.cmd -u [email protected] -w unlock -g mykey.ext

ID: 41

Location: RTP

Name: Raleigh, U.S.A

State: ONLINE

Description: This data center is located in Raleigh, North Carolina, U.S.A. The

services provided are: Guest Instances, Image Capture, Persistent Storage, Reser

ved IP, Private VLAN/VPN.

Available Storage Areas :

Storage Area Id: 2

Storage Area Name: Area2

...............

Storage Area Id: 1


. . .


Demo with Command line toolStep 1b: Find storage volume offering informationUse data found in the create volume command, step 1c

>ic-describe-volume-offerings.cmd -u [email protected] -w unlock -g mykey.ext

. . .

ID: 20035200 ## Note for create volume command

Name: Storage

Label: Storage

Location: 41 ## Data is for RTP, other locations will be listed but output is not shown here

Supported sizes in Gib: 60,256,512,1024,2048,4112,8224,10240 ## Use offering ID with list of numeric sizes

Price: $0/GHR

CurrencyCode: USD

CountryCode: SPT

PricePerQuantity: 1

UnitOfMeasure: GHR

Supported Format :

~~~~~~

Format : EXT3

Format : RAW ## In upper case, note for create volume command

. . .


Demo with Command line toolStep 1c: Create storage volume on a given availability areaUse the input parameters from steps 1a and 1b, note the volume ID for step 2e

>ic-create-volume.cmd -u [email protected] -w unlock -g mykey.ext -n"Storage Demo" -s 60 -f RAW -L 41 -O 20035200 -A 1

Executing action: CreateVolume ...

The request has been submitted successfully.

ID: 31854 ## Used later

Name: Storage Demo

Owner: [email protected]

Size: 60

Format : raw

InstanceId:

OfferingID: 20035200

Storage Area Id: 1


CreatedTime: May 18, 2012 2:55:30 PM CST

Executing CreateVolume finished


Demo with Command line toolStep 1d: Check status of new volume, wait for provisioning to completeIf the status is NEW or PROVISIONING wait for a few minutes and try again

>ic-describe-volume.cmd -u [email protected] -w unlock -g mykey.ext -v 31854

Executing action: DescribeVolume ...

ID: 31854

Name: Storage Demo


Size: 60 Gib

Format : raw

InstanceId:

State: UNMOUNTED

Location: 41

Storage Area Id: 1


I/O operation price: $0.11/million


Platform Version: 2

Executing DescribeVolume finished


Demo with Command line toolStep 2a: Find supported sizes and check support for dynamic attach / detach for selected virtual machine image>ic-describe-image.cmd -u [email protected] -w unlock -g mykey.ext -k 20025206

Executing action: DescribeImage ...

ID: 20025206

Name: Red Hat Enterprise Linux 6.2 (64-bit)(RTP)

Visibility: PUBLIC

State: AVAILABLE

Owner: SYSTEM

Platform: Red Hat Enterprise Linux/6

Platform Version: 1.2 ## OK

Location: 41

Image Size: 5.781006 Gib

----------------------------------

InstanceType ID: COP64.2/4096/60 ## Needed in the ic-create-instance command

Label: Copper 64 bit

Detail: Copper - 64 bit (vCPU: 2, RAM: 4 GiB, Disk: 60 GiB)

Price: $0.3/UHR

CurrencyCode: USD

CountryCode: SPT

PricePerQuantity: 1

UnitOfMeasure: UHR

. . .


Demo with Command line toolStep 2b: Determine SSH key to use for new virtual machine instanceUse for create instance, step 2c

>ic-describe-keypairs.cmd -u [email protected] -w unlock -g mykey.ext

Executing action: DescribeKeyPairs ...

10 keys

----------------------------------

Name: july26 ## Needed in the ic-create-instance command

Default Key: true

Material: ***

LastModifiedTime: August 28, 2011 9:39:05 PM CST

1 instances related.

Instance ID: 210671

. . .


Demo with Command line toolStep 2c: Create an instance with selected image and supported sizeThe primary purpose of the instance is to access data on the storage volume

>ic-create-instance.cmd -u [email protected] -w unlock -g mykey.ext -t "COP64.2/4096/60" -n "Storage Demo VM" -k 20025206 -c july26 -L 41

Executing action: CreateInstance ...


1 instances!

----------------------------------

ID: 234165 ## Needed later in multiple commands

Name: Storage Demo VM

Hostname:

InstanceType: COP64.2/4096/60

IP:

KeyName: july26


RequestID: 234465

RequestName: Storage Demo VM

Status: NEW ## Need to wait for the status to change to ACTIVE

Volume IDs:

----------------------------------

Executing CreateInstance finished


Demo with Command line toolStep 2d: Check state of instance - wait for instance to be createdWe are looking for an ACTIVE state to move to the next step

>ic-describe-instance.cmd -u [email protected] -w unlock -g mykey.ext -l 234165

Executing action: DescribeInstance ...

ID: 234165

Name: Storage Demo VM

Image ID: 20025206

Hostname: vhost1772.site1.compute.ihost.com

InstanceType: COP64.2/4096/60

IP: 170.224.166.242

KeyName: july26


RequestID: 234465

RequestName: Storage Demo VM

Status: ACTIVE

Location: 41

Volume IDs:

Disk Size: 60

Minimal local disk: false

Expiration Time: May 18, 2014 5:36:30 PM CST

Executing DescribeInstance finished


Demo with Command line toolStep 2e: Dynamically attach storage volume to instanceGet the volume and instance ID's from previous steps 1c and 2c

>ic-attach-volume.cmd -u [email protected] -w unlock -g mykey.ext -l 234165 -v 31854

Executing action: AttachVolume ...


Executing AttachVolume finished

## Virtual machine status will go ATTACHING. Check status as in previous step to wait for state to return to ACTIVE.


. . .

Status: ATTACHING

. . .


. . .

Status: ACTIVE

. . .


Demo with Command line toolStep 2f: Log onto the instance and use the attached volume to store dataCheck presence of volume (vdc) and partition it (Bash commands)Log onto instance with SSH and change to root

$ sudo /bin/bash

# /sbin/fdisk -l ## List partitions

Disk /dev/vda: 64.4 GB, 64425558016 bytes

. . .

Disk /dev/vdb: 4294 MB, 4294967296 bytes

. . .

Disk /dev/vdc: 64.4 GB, 64424509440 bytes ## Newly attached partition

. . .

Disk /dev/vdc doesn't contain a valid partition table ## Not partitioned

# /sbin/fdisk /dev/vdc ## Partition the new volume, follow the menu

. . .

# mkfs -t ext3 /dev/vdc1 ## Format the partition to ext3

. . .

# mkdir /volume1 ## Create mount point

# /bin/mount /dev/vdc1 /volume1 ## Mount the file system

# cd /volume1 ## Write data to the volume

# vi my_data.txt

. . .

# umount /volume1 ## unmount file system

# exit ## Log out of instance


Demo with Command line toolStep 3a: Detach the volumeThe volume must be detached to clone it

>ic-detach-volume.cmd -u [email protected] -w unlock -g mykey.ext -l 234165 -v 31854

Executing action: DetachVolume ...


Executing DetachVolume finished

## Volume status will go ATTACHING. Check status as in previous step to wait for state to return to UNMOUNTED.


. . .

State: DETACHING

. . .


. . .

State: UNMOUNTED


Demo with Command line toolStep 3b: Clone the volume to another availability area to back up the dataGet the source volume ID from step 1c

>ic-clone-volume.cmd -u [email protected] -w unlock -g mykey.ext -n Backup -v 31854 -A 2

Executing action: CloneVolume ...


ID: 31872

Name: Backup


Size: 60

Format : raw

InstanceId:

OfferingID: 20035200

Storage Area Id: 2


Data Center Location: 41


Executing CloneVolume finished


Demo with Command line toolStep 3c: Verifying Backup

1.Attach the backup volume to the virtual machine instance

2.Mount the volume to the file system

3.List the files on the volume to make sure that they are the same as the source volume

# /bin/mount /dev/vdc1 /volume1

# ls -l /volume1

total 20

drwx------. 2 root root 16384 May 16 09:41 lost+found

-rw-r--r--. 1 root root 13 May 16 09:43 my_data.txt

# umount /volume1

# exit


Agenda




Demo with Java APIStep 1a: Query to discover existing availability areasGet sample code StorageDemo.java in RAM asset

All the examples that follow are methods that are added to the main program on the next page.

Classes and packages to import with the Java API examples

import com.ibm.cloud.api.rest.client.DeveloperCloud;

import com.ibm.cloud.api.rest.client.DeveloperCloudClient;

import com.ibm.cloud.api.rest.client.bean.*;


import java.util.List;

import com.ibm.cloud.api.rest.client.*;

import com.ibm.cloud.api.rest.client.bean.*;

// Class to list storage areas in a location

public class StorageDemo {

public static final String USERNAME = "[email protected]";

public static final String PASSWORD = "***";

private DeveloperCloudClient client = DeveloperCloud.getClient();

public StorageDemo() {

client.setRemoteCredentials(USERNAME, PASSWORD);

}

// Method to list storage areas

private void describeStorageAreas(String locationID) throws UnknownErrorException, IOException {

List<Location> locations = client.describeLocations();

for (Location l: locations) {

if (locationID.equals(l.getId())) {

List<StorageArea> storageAreas = l.getStorageAreas();

for (StorageArea storageArea : storageAreas) {

System.out.println("Found storage area id " + storageArea.getId() + ", " +

storageArea.getName() + ", " + storageArea.getDescription());

}

}

}

}

public static void main(String[] args) throws Exception {

StorageDemo demo = new StorageDemo();

demo.describeStorageAreas("41");

}

}


Java API DemoList availability areas - program output

Found storage area id 1, Area1, gpfs1 of SONAS1

Found storage area id 2, Area2, gpfs2 of SONAS1


Java API DemoStep 1b: Find storage volume offering ID, available sizes, and supported formatsThis method is part of the class in step 1a

private void describeStorageOfferings(String locationID) throws UnknownErrorException, IOException, UnauthorizedUserException {

List<VolumeOffering> volumeOfferings = client.describeVolumeOfferings();

for (VolumeOffering volumeOffering: volumeOfferings) {

if (locationID.equals(volumeOffering.getLocation())) {

System.out.println("Found offering for target location: " + volumeOffering.getId() +

", " + volumeOffering.getName() + ", " + volumeOffering.getLabel());

if ((volumeOffering.getSupportedSizes() != null) && (volumeOffering.getSupportedSizes().size() > 1)) {

List<Long> sizes = volumeOffering.getSupportedSizes();

System.out.print("Sizes: ");

for (Long size : sizes) {

System.out.print(" " + size);

}

System.out.println();

}

}

}

}


Java API DemoList offering ID's and supported sizes - program output

Found offering for target location: 20001208, Small, 256 GiB

Found offering for target location: 20001209, Medium, 512 GiB

Found offering for target location: 20001210, Large, 2048 GiB

Found offering for target location: 20035200, Storage, Storage

Sizes: 60 256 512 1024 2048 4112 8224 10240


Java API DemoStep 1c: Method to create a volume in a specific storage availability areaThis method is also part of the class in step 1a

private void createVolume(String name, String locationID, String offeringId, String format,

String size, String areaID) throws InsufficientResourcesException,

InvalidConfigurationException, PaymentRequiredException, UnauthorizedUserException,

UnknownErrorException, IOException {

Volume volume = client.createVolume(name, locationID, offeringId, format, size, areaID);

System.out.println("Volume ID: " + volume.getID());

}

. . .

// Invoke the method

demo.createVolume("My Disk", "41", "20035200", "RAW", "60", "1");


Java API DemoStep 1d: Check status, wait until provisioning is completeThis method is a simple wrapper for the DeveloperCloudClient.describeVolumes methodthat prints out some information about the volumes

private void describeVolumes() throws InsufficientResourcesException,


UnknownErrorException, IOException, UnknownVolumeException {

List<Volume> volumes = client.describeVolumes();

for (Volume volume : volumes) {

System.out.println("Volume ID: " + volume.getID());

System.out.println("Name: " + volume.getName());

System.out.println("State: " + volume.getState());

System.out.println("Area: " + volume.getStorageArea().getId() + ", " +

volume.getStorageArea().getName());

}

}

…

demo.describeVolumes();


Java API DemoMethod to list storage availability area for individual volumes - program output

Volume ID: 31598

Name: My Disk

State: UNMOUNTED

Area: 1, Area1


Java API DemoStep 2a: Find supported sizes and check support for dynamic attach / detach for selected virtual machine imageGiven by the platform version property, which must be at least 1.2

private Image getImage(String imageID) throws UnauthorizedUserException, UnknownImageException,

UnknownErrorException, IOException {

Image image = client.describeImage(imageID);

System.out.println("Image ID: " + image.getID());

System.out.println("Image name: " + image.getName());

System.out.println("Image Platform Version: " + image.getPlatformVersion());

List<InstanceType> supportedSizes = image.getSupportedInstanceTypes();

for (InstanceType supportedSize : supportedSizes) {

System.out.println("Size: " + supportedSize.getId() + ", " + supportedSize.getLabel());

}

return image;

}

. . .

Image image = demo.getImage("20025206");


Java API DemoStep 2b: Determine if the image supports dynamic attach and detach - program output

Image ID: 20025206

Image name: Red Hat Enterprise Linux 6.2 (64-bit)(RTP)

Image Platform Version: 1.2

Size: COP64.2/4096/60, Copper 64 bit

Size: BRZ64.2/4096/60*500*350, Bronze 64 bit

Size: SLV64.4/8192/60*500*500, Silver 64 bit

Size: GLD64.8/16384/60*500*500, Gold 64 bit

Size: PLT64.16/16384/60*500*500*500*500, Platinum 64 bit

Size: PM264.16/32768/60*500*500*500*500, Platinum-M2 64 bit


Java API DemoStep 2c: Determine SSH key to use for virtual machine instance

private String getDefaultKey() throws UnauthorizedUserException, UnknownErrorException, IOException {

String defaultKey = null;

List<Key> keys = client.describeKeys();

for (Key key: keys) {

System.out.println("Found key with name " + key.getName());

if (key.isDefaultKey()) {

defaultKey = key.getName();

}

}

System.out.println("Found default key " + defaultKey);

return defaultKey;

}


Java API DemoStep 2d: Create an instance with selected image and supported size

private void createInstance(String instanceName, Image image, String instanceTypeIdStr,

String keyName) throws UnauthorizedUserException, UnknownErrorException,

IOException, InsufficientResourcesException, InvalidConfigurationException, PaymentRequiredException {

System.out.println("Provisioning a new instance");

String location = image.getLocation();

String imageID = image.getID();

Address address = null;

Volume volume = null;

// Create instance

List<Instance> instances = client.createInstance(instanceName, location, imageID,

instanceTypeIdStr, keyName, address, volume, new HashMap<String,Object>());

// Check the status of the provisioning test

for (Instance instance: instances) {

System.out.println("New instance with id: " + instance.getID() + " status: " + instance.getStatus());

}

}

. . .

Image image = demo.getImage("20025206");

String keyName = demo.getDefaultKey();

demo.createInstance("Java API Test Instance", image, "COP64.2/4096/60", keyName);


Java API DemoMethod to create virtual machine instance - program output

Provisioning a new instance

New instance with id: 232906 status: NEW


Java API DemoStep 2e: Check the state of instance - wait for instance to be createdThis method is a simple wrapper for DeveloperCloudClient.describeInstance

private void describeInstance(String id) throws InsufficientResourcesException,


UnknownErrorException, IOException, UnknownInstanceException {

Instance instances = client.describeInstance(id);

System.out.println("Name: " + instances.getName());

System.out.println("Status: " + instances.getStatus());

List<String> volumeIds = instances.getVolumeIds();

System.out.println("Number of attached volumes: " + volumeIds.size());

for (String volumeId : volumeIds) {

System.out.println("VolumeId: " + volumeId);

}

}

. . .

demo.describeInstance("232906");


Java API DemoMethod to check status of virtual machine instance - program output

Name: Java API Test Instance

Status: ACTIVE

Number of attached volumes: 0


Java API DemoStep 2f: Method to dynamically attach a volumeCall it from the main() method

private void attachVolume(String instanceID, String volumeID) throws UnauthorizedUserException, InvalidConfigurationException,

InvalidStateException, UnknownErrorException, UnSupportFeatureException, IOException {

client.attachStorage(instanceID, volumeID);

}

. . .

demo.attachVolume("232906", "31598");

// Check status as above or check in web portal as shown on the next page


Java API DemoStatus of virtual machine instance during attaching operation – web portalThe status changes to ATTACHING for a short period


Java API DemoStep 3a: Log onto the instance and use the attached volume to store data

Same as step 10 of the command line tool demo, which lists the Bash commands


Java API DemoStep 3b: Detach volumeCall it from the main method

private void dettachVolume(String instanceID, String volumeID) throws UnauthorizedUserException, InvalidConfigurationException,

InvalidStateException, UnknownErrorException, UnSupportFeatureException, IOException {

client.detachStorage(instanceID, volumeID);

}

. . .

demo.dettachVolume("232906", "31598");

// Check status as above or check in web portal as shown on the next page


Java API DemoStatus of virtual machine instance during detaching operation – web portalSelect the volume in the web portal to see the notifications for it


Java API DemoStep 3b: Status of volume during cloning operation – Java methodClone volume with ID 31598, named My Disk, from availability area 1 to area 2

private void cloneVolume(String name, String volumeID, String targetAreaID) throws

InsufficientResourcesException, InvalidConfigurationException, PaymentRequiredException,

UnauthorizedUserException, UnknownErrorException, IOException {

client.cloneVolume(name, volumeID, targetAreaID);

}

. . .

demo.cloneVolume("Backup", "31598", "2");


Java API DemoStatus of volume during cloning operation – web portal


Java API DemoStep 3c: Verifying Backup

Same as step 13 of the command line tool demo


Resources

Infrastructure as a service cloud development Blog

https://www.ibm.com/developerworks/mydeveloperworks/blogs/iaas_cloud/?lang=en

SCE Command Line Toolkit and Reference

https://www-147.ibm.com/cloud/enterprise/ram/assetDetail/generalDetails.faces?guid={F1466F46-A4AB-3879-D883-1A26A43BF046}

SCE Java API and JavaDoc

https://www-147.ibm.com/cloud/enterprise/ram/search/_rlvid.jsp.faces?_rap=!assetDetails&_rvip=/search/index.jsp&guid={555C7964-F531-AECF-F8E9-F5F2D37B64D8}

SCE REST User Guide

https://www-147.ibm.com/cloud/enterprise/ram/artifact/%7B62ACBF96-CEF0-0B67-8529-144388066543%7D/*/GA_APIUserGuide.pdf

Red Hat Enterprise Linux Storage Administration Guide

http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/6/html/Storage_Administration_Guide/index.html


Trademarks and notes

©IBM Corporation 2012

IBM, the IBM logo, ibm.com, Cognos, DB2, Informix, Lotus, Rational, SmartCloud, System x, Tivoli and WebSphere are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with the appropriate symbol (® or ™), these symbols indicate US registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the web at “Copyright and trademark information” at www.ibm.com/legal/copytrade.shtml

Intel is a trademark of Intel Corporation or its subsidiaries in the United States and other countries.

Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both.

Microsoft and Windows are trademarks of Microsoft Corporation in the United States, other countries, or both.

Other company, product and service names may be trademarks or service marks of others.

References in this publication to IBM products or services do not imply that IBM intends to make them available in all countries in which IBM operates.

ibm smartcloud enterprise storage demo

Technology

cloned storage volume

storage area

volume clone

clone storage volume

demo backup storage

volume upadetach

storage volume11

storage volume3