grid based vm provisioning - a presentation by arunabh das
TRANSCRIPT
8/8/2019 Grid Based VM Provisioning - A Presentation by Arunabh Das
http://slidepdf.com/reader/full/grid-based-vm-provisioning-a-presentation-by-arunabh-das 1/36
Grid-based VM-provisioning UsingSTAR / Nimbus
Date : 2 Apr, 2009Presented by : Arunabh DasSchool of Computer ScienceSources:http://workspace.globus.orghttp://www.gridvm.org
Other Sources :Enabling Cost-Effective Resource Leases with Virtual Machines, Sotomayor, B., K. Keahey, I. Foster, T. Freeman. HPDC 2007 Hot Topics session,
Monterey Bay, CA. June 2007 (pdf)
Virtual Workspaces for Scientific Applications, Keahey, K., T. Freeman, J. Lauret, D. Olson. SciDAC 2007 Conference, Boston, MA. June 2007 (pdf)
8/8/2019 Grid Based VM Provisioning - A Presentation by Arunabh Das
http://slidepdf.com/reader/full/grid-based-vm-provisioning-a-presentation-by-arunabh-das 2/36
8/8/2019 Grid Based VM Provisioning - A Presentation by Arunabh Das
http://slidepdf.com/reader/full/grid-based-vm-provisioning-a-presentation-by-arunabh-das 3/36
What do we seek?What are we looking to find?
Ginormous amounts of compute powerAvailable 24x7x365Humongous amounts of storage
Also available 24x7x365The ability to access the above from the cupboards
that professors and post-doc fellows and the millionsof starving graduate students in North America,Europe, Asia and Africa live in
8/8/2019 Grid Based VM Provisioning - A Presentation by Arunabh Das
http://slidepdf.com/reader/full/grid-based-vm-provisioning-a-presentation-by-arunabh-das 4/36
8/8/2019 Grid Based VM Provisioning - A Presentation by Arunabh Das
http://slidepdf.com/reader/full/grid-based-vm-provisioning-a-presentation-by-arunabh-das 5/36
So we still haven't found what we'relooking for?
Kate Keahey is a scientist at Argonne NationalLaboratory and a Computation Institute fellow atUniversity of Chicago
She created and leads the Nimbus ProjectShe calls it Infrastructure as a Service (IaaS)Which makes perfect sense!!
8/8/2019 Grid Based VM Provisioning - A Presentation by Arunabh Das
http://slidepdf.com/reader/full/grid-based-vm-provisioning-a-presentation-by-arunabh-das 6/36
A Brief History of Nimbus
Research onagreement-based
services
Xen released
First WSRFWorkspace Service
releaseEC2 gatewayavailable Support for EC2 interfaces
2003 20092006
EC2 goes online
First STARproductionrun on EC2
Nimbus Cloudcomes online
Context Broker releaseSource -
http://colab.cim3.net/file/work/Expedition_Workshop/2009_02_09_LeveragingSOA_Cy
berinfrastructure/magic-Keahey.cloudcomputing.ppt
8/8/2019 Grid Based VM Provisioning - A Presentation by Arunabh Das
http://slidepdf.com/reader/full/grid-based-vm-provisioning-a-presentation-by-arunabh-das 7/36
Grid Technologies : A brief overview
Infrastructure (”middleware”) for establishing,managing and evolving multi-organizationfederations
Secure, coordinated sharingDynamic, autonmous, domain independentOn-demand, ubiquitous access to computing, data
and services
Globus Toolkit : An implementation of the most basiccapabilitiesA de facto implementation standard
8/8/2019 Grid Based VM Provisioning - A Presentation by Arunabh Das
http://slidepdf.com/reader/full/grid-based-vm-provisioning-a-presentation-by-arunabh-das 8/36
A typical grid use-case
2. Finds available resources
3. Starts a remote computation
4. Transfers data from a remote location
Grid Security Infrastructure (GSI)Monitoring and Discovery Service (MDS)
Grid Resource and Allocation Manager (GRAM)Data Transfer (GridFTP)
gridmapfile
1. User logs into the Grid (single sign-on):grid-proxy-init
8/8/2019 Grid Based VM Provisioning - A Presentation by Arunabh Das
http://slidepdf.com/reader/full/grid-based-vm-provisioning-a-presentation-by-arunabh-das 9/36
The case for VM on grid
Most grid applications and grid infrastructure just needs to be able to handle heavycomputation and heavy lifting of data
However, certain applications (Ex – Nuclear Physics STAR Experiment) rely heavilyon dynamically loading external libraries depending on the task to be performed
Configuring an environment for such an application is complex
Deployment on non-dedicated platform = effort consumingEven when the application compiles on a new platform, validating it is a controlledprocess subject to quality assurance and regression testing to ensure Physicsreproducibility and result uniformity
Heavy reliance of an application (ex-Physics Engine) on dependencies deeplyembedded in the environment
=> Porting application would be easiest if we could take the full software stack fromthe operating system up, and simply install that environment on remote resources
8/8/2019 Grid Based VM Provisioning - A Presentation by Arunabh Das
http://slidepdf.com/reader/full/grid-based-vm-provisioning-a-presentation-by-arunabh-das 10/36
The case for VM on grid (contd)
Virtual machine provides a software-basedvirtualization of a physical host machine
Dedicated
Configured with a full software stack Once configured, deploys on a remote resource in a
matter of millisecondsResource provisioning via Vms is attractive
8/8/2019 Grid Based VM Provisioning - A Presentation by Arunabh Das
http://slidepdf.com/reader/full/grid-based-vm-provisioning-a-presentation-by-arunabh-das 11/36
More benefits of VM on grid
A scientist can develop his or her application within afamiliar environment
Can port this environment between local and remote
resources as the need arisesThis facilitates provisioning resources for an
applicationThe virtual machine can be run as easily on local
resources as on remote resources or resourcesoutsourced commercially
8/8/2019 Grid Based VM Provisioning - A Presentation by Arunabh Das
http://slidepdf.com/reader/full/grid-based-vm-provisioning-a-presentation-by-arunabh-das 12/36
A quick look at Virtual Workspaces(STAR)
STAR is the predecessor of Nimbus and was developed by Kate Keahey and TimFreeman at ANL
The Solenoidal Tracker at RHIC (STAR) is a detector which specializes in tracking thethousands of particles produced by each ion collision at RHIC. (Relativistic HeavyIon Collider)
STAR is a massive detector.It is used to search for signatures of the form of matter that RHIC was designed tocreate: the quark-gluon plasma.
It is also used to investigate the behavior of matter at high energy densities by makingmeasurements over a large area.
It is a proof-of-concept strategy developed for the High Energy and Nuclear Physics
(HENP) group
8/8/2019 Grid Based VM Provisioning - A Presentation by Arunabh Das
http://slidepdf.com/reader/full/grid-based-vm-provisioning-a-presentation-by-arunabh-das 13/36
A Brief Look at VMA VM is a virtualization abstraction of a physical machine (hardware resources
+ software infrastructure)Software running on a host supporting VM deployment, typically called a
VMM (Virtual Machine Monitor) or Hypervisor is responsible forsupporting this abstraction by intercepting and emulating instructions issuedby the guest machine
Hypervisor provides an interface allowing client to start, pause, serialize, andshut down multiple guests
VM image is composed of a full image of a VM RAM, disk images andconfiguration files
Thus, VM can be paused, its state serialized and later resumed at a different
time and in a different location=> Decouples image preparation from deployment => Easy migration
Sources :Enabling Cost-Effective Resource Leases with Virtual Machines, Sotomayor, B., K. Keahey, I. Foster, T. Freeman. HPDC 2007 Hot Topics session,
Monterey Bay, CA. June 2007 (pdf)Virtual Workspaces for Scientific Applications, Keahey, K., T. Freeman, J. Lauret, D. Olson. SciDAC 2007 Conference, Boston, MA. June 2007 (pdf)
8/8/2019 Grid Based VM Provisioning - A Presentation by Arunabh Das
http://slidepdf.com/reader/full/grid-based-vm-provisioning-a-presentation-by-arunabh-das 14/36
ParavirtualizationVirtualization technique that presents a software interface to virtual
machines that is similar but not identical to that of the underlyinghardware.
Example of paravirtualization =The virtual monitor can present the host operating system with an
intelligent NIC with support for DMA-based sending of packets, even
though the NIC on the real system lacks this capability.Sending packets is then done entirely by the virtual monitor and NIC
interrupts may be processed by the monitor tooSince delivering interrupts to the host operating system is expensive,
performance can improve.Who'd a thunk it?Paravirtualization actually helps performance!!
8/8/2019 Grid Based VM Provisioning - A Presentation by Arunabh Das
http://slidepdf.com/reader/full/grid-based-vm-provisioning-a-presentation-by-arunabh-das 15/36
Virtual Workspace Features
Workspace provides interfaces based on the WSRFAllows an authorized Grid client to deploy, shutdown,
pause and reactivate VMS
8/8/2019 Grid Based VM Provisioning - A Presentation by Arunabh Das
http://slidepdf.com/reader/full/grid-based-vm-provisioning-a-presentation-by-arunabh-das 16/36
Worker node deployment workflow
Worker node deployment is requested on-demand by an authorizedoff-site grid client
Resource allocation request asks for 2 GB memory and the full useof a CPU for each virtual node
On deployment, each node reports to Condor headnode and joinsthe Condor pool
A web application displays current virtual cluster node informationbased on Condor pool properties
A client can then start jobs on the deployed VM using GRAM2deployed on the static CE (compute element)
8/8/2019 Grid Based VM Provisioning - A Presentation by Arunabh Das
http://slidepdf.com/reader/full/grid-based-vm-provisioning-a-presentation-by-arunabh-das 17/36
Schematic for provisionig of STAR nodes
TeraPort node TeraPort node TeraPort node TeraPort node TeraPort node
WorkspaceService
OSGCEGRAM
Star node
Star node
Star node
Star node
Star node
Star node
Star nodeStar node
ProvisioningSTAR nodes
Execute newSTAR instance
STAR
8/8/2019 Grid Based VM Provisioning - A Presentation by Arunabh Das
http://slidepdf.com/reader/full/grid-based-vm-provisioning-a-presentation-by-arunabh-das 18/36
Current Cloud providers
Go GridAmazon Web ServicesGoogle App Engine
MossoSlice HostMedia TempleFlexiscaleJoyent
Although they provide webservices and compute-power-on-demand, they are not VirtualMachines on grid
8/8/2019 Grid Based VM Provisioning - A Presentation by Arunabh Das
http://slidepdf.com/reader/full/grid-based-vm-provisioning-a-presentation-by-arunabh-das 19/36
Cloud Computing – Everything as a Service
Elastic computing,Pay-as-you-go,
Capital expense
operational expense
Source : Cloud Computing with Nimbus, FNAL, January 2009Kate Keahey, University of Chicago, ANL
8/8/2019 Grid Based VM Provisioning - A Presentation by Arunabh Das
http://slidepdf.com/reader/full/grid-based-vm-provisioning-a-presentation-by-arunabh-das 20/36
Cloud Computing – Everything-as-a-service
IaaS
PaaS
SaaS
Source : Cloud Computing with Nimbus, FNAL, January 2009
Kate Keahey, University of Chicago, ANL
Software as a Service
Platform as a Service
Infrastructure as a Service
The anology to the real world is that it used to be that if you wanted to go to the airport, you couldcall a cab and pay the cab-driver.Then, they said, you know what – if you pay us, we can let you rent the car and you can have thecar, but you can't be setting the car on fireNow – you can lease a car and have the car for as long as you want and do whatever you want tothe car. Ofcourse, you are going to be able to do a lot more than just drive to the airport with thecar.
8/8/2019 Grid Based VM Provisioning - A Presentation by Arunabh Das
http://slidepdf.com/reader/full/grid-based-vm-provisioning-a-presentation-by-arunabh-das 21/36
Main problems we're trying to solve
Code complexityResource control
Source - The Nimbus Toolkit : http://workspace.globus.org
8/8/2019 Grid Based VM Provisioning - A Presentation by Arunabh Das
http://slidepdf.com/reader/full/grid-based-vm-provisioning-a-presentation-by-arunabh-das 22/36
The concept of 'workspaces'
Dynamicall provisioned environmentEnvironment controlResource control
Hardware implementations vs. virtualization
Source : Cloud Computing with Nimbus, FNAL, January 2009
Kate Keahey, University of Chicago, ANL
8/8/2019 Grid Based VM Provisioning - A Presentation by Arunabh Das
http://slidepdf.com/reader/full/grid-based-vm-provisioning-a-presentation-by-arunabh-das 23/36
Nimbus Overview
Goal: open source, extensible, IaaS implementation and toolsSpecifically targeting scientific communityA platform for experimentation with features for scientific needsSet up private clouds (privacy, expense considerations)
Tools
IaaS layer (Workspace Service)Orchestration layer (Context Broker, gateway)http://workspace.globus.org/
Source : Cloud Computing with Nimbus, FNAL, January 2009
Kate Keahey, University of Chicago, ANL
8/8/2019 Grid Based VM Provisioning - A Presentation by Arunabh Das
http://slidepdf.com/reader/full/grid-based-vm-provisioning-a-presentation-by-arunabh-das 24/36
Workspace Pilot and the concept of resource leases
Resource leases – Allow users to request direct access to resourcesrather than ask for a job to be run on those resource
ExamplesA static long-term agreement with a hosting companyOn-demand provisioning of a physical cluster partition with a
specified configuration (Cluster-on-demand)Dynamically deploying a virtual machine for an hour on
resource provied by Amazon's EC2 service
Ad f l i
8/8/2019 Grid Based VM Provisioning - A Presentation by Arunabh Das
http://slidepdf.com/reader/full/grid-based-vm-provisioning-a-presentation-by-arunabh-das 25/36
Advantages of 'Flying Low'(Workspace Pilot)
A user can adapt resource to his needsUse it to support an interactive sessionRun computations requiring an application-specific
schedulerSupport portability tests across a variety of
environmentsExemplified by 'pilot job' approaches that use batch
scheduler installations on sites to deliver a leaserather than submit a job to that scheduler
Source : Flying Low : Simple Leases with Workspace Pilot
Tim Freeman Kate Keahey, University of Chicago, ANL
I l i f VWS
8/8/2019 Grid Based VM Provisioning - A Presentation by Arunabh Das
http://slidepdf.com/reader/full/grid-based-vm-provisioning-a-presentation-by-arunabh-das 26/36
Implementation of VWS(Virtual Workspace Service)
Poolnode
Poolnode
Poolnode
Poolnode
Poolnode
Poolnode
Poolnode
Poolnode
Poolnode
Poolnode
Poolnode
Poolnode
VWSService
Source : Cloud Computing with Nimbus, FNAL, January 2009Kate Keahey, University of Chicago, ANL
8/8/2019 Grid Based VM Provisioning - A Presentation by Arunabh Das
http://slidepdf.com/reader/full/grid-based-vm-provisioning-a-presentation-by-arunabh-das 27/36
The Workspace Service
Poolnode
Trusted Computing Base (TCB)
Poolnode
Poolnode
Pool
node
Pool
node
Pool
node
Poolnode
Poolnode
Poolnode
Poolnode
Poolnode
Poolnode
The workspace service publishes
information on each workspaceas standard WSRF ResourceProperties.
Users can interactdirectly with their
workspaces the sameway the would with a
physical machine.
VWSService
Source : Cloud Computing with Nimbus, FNAL, January 2009Kate Keahey, University of Chicago, ANL
8/8/2019 Grid Based VM Provisioning - A Presentation by Arunabh Das
http://slidepdf.com/reader/full/grid-based-vm-provisioning-a-presentation-by-arunabh-das 28/36
Workspace Service Interfaces and Clients
Web Services basedWeb Service Resource Framework (WSRF)
GT-basedElastic Computing Cloud (EC2)
Supported: ec2-describe-images, ec2-run-instances, ec2-describe-instances, ec2-terminate-instances, ec2-reboot-instances, ec2-add-keypair, ec2-delete-keypair
Unsupported: availability zones, security groups, elastic IP assignment,REST
Used alongside WSRF interfacesE.g., the University of Chicago cloud allows you to connect via the cloud
client or via the EC2 client
8/8/2019 Grid Based VM Provisioning - A Presentation by Arunabh Das
http://slidepdf.com/reader/full/grid-based-vm-provisioning-a-presentation-by-arunabh-das 29/36
Nimbus Schematic
workspacecontrol
workspaceresourcemanager
workspace
pilot
workspaceservice
workspaceclient
cloudclient
IaaSgateway
contex
t broker
contextclient
EC2potentially other providers
storageservice
E C 2
W S R F
Source : Cloud Computing with Nimbus, FNAL, January 2009Kate Keahey, University of Chicago, ANL
8/8/2019 Grid Based VM Provisioning - A Presentation by Arunabh Das
http://slidepdf.com/reader/full/grid-based-vm-provisioning-a-presentation-by-arunabh-das 30/36
Science Clouds
q Make it easy for scientific projects to experiment with cloud computingx Can cloud computing be used for science?
q Evolve software in response to the needs of scientific projectsx Start with EC2-like functionality and evolve to serve scientific projects: virtual
clusters, diverse resource leasesx Federating clouds: moving between cloud resources in academic and commercial
space
Source : Cloud Computing with Nimbus, FNAL, January 2009Kate Keahey, University of Chicago, ANL
8/8/2019 Grid Based VM Provisioning - A Presentation by Arunabh Das
http://slidepdf.com/reader/full/grid-based-vm-provisioning-a-presentation-by-arunabh-das 31/36
Science Cloud Resources
q University of Chicago (Nimbus):x first cloud, online since March 4th 2008x 16 nodes of UC TeraPort cluster, public IPs
q University of Floridax Online since 05/08x 16-32 nodes, access via VPN
q Other Science Cloudsx Masaryk University, Brno, Czech Republic (08/08), Purdue (09/08)x Installations in progress: IU, Grid5K, others
q Using EC2 for overflowq Minimal governance modelq http://workspace.globus.org/clouds
8/8/2019 Grid Based VM Provisioning - A Presentation by Arunabh Das
http://slidepdf.com/reader/full/grid-based-vm-provisioning-a-presentation-by-arunabh-das 32/36
Nimbus Walkthrough
8/8/2019 Grid Based VM Provisioning - A Presentation by Arunabh Das
http://slidepdf.com/reader/full/grid-based-vm-provisioning-a-presentation-by-arunabh-das 33/36
Open Source IaaS Implementations
q Eucalyptusx Open source implementation of EC2x UCSB, R. Wolski & team, 06/2008
q OpenNebulax Open source datacenter implementationx University of Madrid, I. Llorente & team, 03/2008
q Cloud-enabled Nimrod-Gx Monash University, MeSsAGE Lab, 01/2009
q Industry effortsx openQRM, Enomalism
8/8/2019 Grid Based VM Provisioning - A Presentation by Arunabh Das
http://slidepdf.com/reader/full/grid-based-vm-provisioning-a-presentation-by-arunabh-das 34/36
The Nimbus Community
q Committers: Kate Keahey & Tim Freeman (ANL/UC), Ian Gable (UVIC)q A lot of help from the community, see:
http://workspace.globus.org/people.htmlq Collaborations:
x Cumulus: S3 implementation (Globus team)x
EBS implementation with IUx Appliance management: rPath and Bcfg2 projectx Virtual network overlays: University of Floridax Security: Vienna University of Technology
8/8/2019 Grid Based VM Provisioning - A Presentation by Arunabh Das
http://slidepdf.com/reader/full/grid-based-vm-provisioning-a-presentation-by-arunabh-das 35/36
Future
q Increasing Importance of Appliance Providersq Cloud computing toolsq Increased interest in cloud interoperability
x Standards: “rough consensus & working code”x Image formats, contextualization capabilities, cloud interfaces, etc.
q
Cloud markets
8/8/2019 Grid Based VM Provisioning - A Presentation by Arunabh Das
http://slidepdf.com/reader/full/grid-based-vm-provisioning-a-presentation-by-arunabh-das 36/36