balticgrid-ii project bg 2nd ahm, god, 12.05.2009, riga1 balticcloud: cloud technologies for...

BalticGrid-II ProjectBalticGrid-II Project

BG 2nd AHM, GOD, 12.05.2009, Riga 1

BalticCloud: Cloud Technologies for Industry and Academia

BalticCloud: Cloud Technologies for Industry and Academia

IlIljja Livenson, NICPBa Livenson, NICPB, Estonia, EstoniaTechnical coordinator Technical coordinator of BalticCloudof BalticCloud

[email protected]@kbfi.ee

2

OutlineOutline

Quick introduction to Cloud computingQuick introduction to Cloud computing

BalticCloudBalticCloud What services will we offer Technological solution

““New” application classesNew” application classes

Collaboration with SMEsCollaboration with SMEs

BG 2nd AHM, GOD, 12.05.2009, Riga

Cloud computing Cloud computing

““Cloud Computing refers to both the Cloud Computing refers to both the applications applications delivered as services over the Internet delivered as services over the Internet and the and the hardware and systems software in the hardware and systems software in the datacenters that provide those servicesdatacenters that provide those services… …

The datacenter hardware and software is what The datacenter hardware and software is what we will call a Cloudwe will call a Cloud……

Cloud computing has the following characteristicsCloud computing has the following characteristics

1. The illusion of infinite computing resources..

2. The elimination of an up-front commitment by Cloud users..

3. The ability to pay for use … as needed…” UC Berkeley RAD Labs

3BG 2nd AHM, GOD, 12.05.2009, Riga

Spectrum of CloudsSpectrum of Clouds

Instruction Set VM (Amazon EC2)Instruction Set VM (Amazon EC2)

Bytecode VM (Microsoft Azure)Bytecode VM (Microsoft Azure)

Framework VM (Google App Engine)Framework VM (Google App Engine)

EC2 Azure AppEngine

Lower-level,Less management

Higher-level,More management

Automatic scalability and failover4BG 2nd AHM, GOD, 12.05.2009, Riga

Cloud model and BCCloud model and BC

5

… as a Service

BalticCloud

Users

BG 2nd AHM, GOD, 12.05.2009, Riga

Building brick: Datacenter as IaaSBuilding brick: Datacenter as IaaS

Virtualize computational resourcesVirtualize computational resources There are drawbacks!

– Virtualization overhead (CPU, IO), “noisy neighbours”, state preservation, etc

On-demand resourcesOn-demand resources Creating virtual machine with a specified set of

resources is possible Applications consume as much resources as they

actually need, overprovisioning is still there, but to a lesser extent

““Cloud of clouds” approachCloud of clouds” approach Open question


Virtualization toolkitVirtualization toolkit

There are several projects out there offering cloud There are several projects out there offering cloud solutions (IaaS)solutions (IaaS) OpenSource: Eucalyptus, Nimbus, AbiCloud, … Commercial: VMWare, Citrix, …

Our choice: EucalyptusOur choice: Eucalyptus Great team! Integration with RightScale Latest version: 1.5.1

– EC2, S3, EBS


Eucalyptus servicesEucalyptus services

EC2 virtual machine provisioningEC2 virtual machine provisioning

IImplementation of Amazon S3 interfacesmplementation of Amazon S3 interfaces - Walrus - Walrus Very simple bucket based filesystem with ACLs

And Amazon's Elastic Block StoreAnd Amazon's Elastic Block Store analogue analogue Persistent volumes

Doesn’t include scalability/reliability mechanismsDoesn’t include scalability/reliability mechanisms But...


Security (access)Security (access)

Eucalyptus uses X.509 security infrastructureEucalyptus uses X.509 security infrastructure “Light” version No VOMS extensions, no OCSP, no CRLs, no SAML

assertions, etc

Can we use BalticGrid/EGEE infrastructure?Can we use BalticGrid/EGEE infrastructure? Well, yes, but… One of priorities for us


Security (process)Security (process)

Job/Application isolation levelJob/Application isolation level Low (same users within VO)

– what if if size(VO) > 1000? Semi-low (different user groups) Medium (e.g. jail/chroot, shared kernel) High (separate VM for each job)

Lower level = lower security level, less Lower level = lower security level, less customization options, (more efficient)customization options, (more efficient)Network securityNetwork security Not every switch supports VLANs, or does that well Need that if we want to give local root to the users

– Or just encrypt everything critical


Use case: InteroperabilityUse case: Interoperability

Complicated problemComplicated problem

OGF has published a number of standardsOGF has published a number of standards E.g. OGSA-BES for execution or GLUE for information

system

Middleware of interest for our region:Middleware of interest for our region: gLite, ARC and UNICORE Implementation of standards support is slow

Mid-term solutionMid-term solution Provide core components for every system as virtualized

images On-demand WN creation


“New” application classes“New” application classes

Grid was built for batch processingGrid was built for batch processing There are some workarounds

– Pull-mode execution– “VO-box” component of gLite

But no solution!

OLAP/OLTPOLAP/OLTP Databases Application servers, Web servers BI Load balancing

Hadoop frameworkHadoop frameworkAppScale and other PaaS solutionsAppScale and other PaaS solutions


Missing functionalityMissing functionality

MissingMissing Monitoring Accounting Reasonable OS image management Billing

Some issues couldSome issues could be solved by integration with be solved by integration with RightScaleRightScale Vendor lock-in Not open-source

We are thinking of reinventing some wheelsWe are thinking of reinventing some wheels


Collaboration with SMEs (1)Collaboration with SMEs (1)

SME = Small and Medium EnterprisesSME = Small and Medium EnterprisesSo far so badSo far so bad It has been very painful with gLite based solution It hasn’t been to easy with “pbs cluster access”

ReasonsReasons Shared WN (typical case for multicore) is not acceptable Interfaces are too complicated and components are too

unstable Not enough control: “What do you mean by SL3/4? What do

you mean by CLI? What do you mean by queuing? We already have J2EE application, just give us the cluster to deploy it on!”

Accounting Interactive applications


Collaboration with SMEs (2)Collaboration with SMEs (2)

We hope to make it better with cloud approachWe hope to make it better with cloud approach Industry driven, not HEP driven Reasonable economical models Existing success stories

Courses for SMEs on best practices using cloudsCourses for SMEs on best practices using clouds In all countries involved Building a network of adopters

Free access to resources for academic Free access to resources for academic startups/spinoffsstartups/spinoffsHybrid cloudsHybrid clouds Selling unused cycles to SMEs


SummarySummary

Cloud computing is not a silver bulletCloud computing is not a silver bullet Too much hype

But…But… It does improve on many aspects

– Security, resource usage, interactive applications Economically motivated

– Startups and prototyping, one-off tasks, research at scale– Scaling solutions

BalticCloud will strive to provide cloud services to BalticCloud will strive to provide cloud services to academia and industryacademia and industry


CreditsCredits

Many thanks to people involvedMany thanks to people involved Aake Edlund - BalticGrid Project Director (Sweden) Mario Kadastik (Estonia) Janis Kulins (Latvia) Dalius Mazeika (Lithuania) Eduardas Kutka (Lithuania) Yuri Ziamtsou (Belarus) (your name here)

http://cloud.balticgrid.eu


Happy endHappy end

Questions?Questions?

Many thanks to the Many thanks to the Grid Open Day Grid Open Day organizers!organizers!


balticgrid-ii project bg 2nd ahm, god, 12.05.2009, riga1 balticcloud: cloud technologies for...

Documents