new roles in the cloud

©2010 HP

NEW ROLES IN THE CLOUD

Steve Loughran

HP Laboratories

March 2010

©2010 HP 2

Cloud computing will change• how applications are delivered• how applications are designed• how teams work

©2010 HP 3

CLASSIC SERVER-SIDE APPLICATIONS

– Architect designs the application

– Developers code and test on local machines

– Operations buy and configure production machines

– Developers get blame for things not working

– Operations get blame for security & availability problems

– Getting predictions of demand wrong can kill your project

©2010 HP 4

CLOUD COMPUTING ELIMINATES

– Buying hardware based on predicted load

– 2+ week lead time on new hardware, storage

– High Availability

– Homogeneity

– Static machine names, addresses and capabilities

– Stable machines

– A fast private network

– Someone in the datacentre who cares about you

©2010 HP 5

IMPLICATIONS FOR HARDWARE VENDORS

(servers, routers, storage, ... )

– No. of major accounts drops to? 5? 10? Less?

– Power consumption becomes a key concern

– Network: power, agile routing, bandwidth, throttling

– Servers: power, VM-hosting, uniformity, no-extra features

©2010 HP6

AGILE APPLICATIONS

©2010 HP 7

WHAT HAS GONE?

– High Availability through hardware

– Static hostnames, network addresses,

– rDNS, multicast IP

– Maybe: SQL databases

©2010 HP 8

APPLICATIONS MUST BE AGILE

– Directory, database or CM service to configure

– Applications to rebind on loss of server connectivity

– Use dynamic DNS services; don’t cache IPAddrs

– Don’t expect HDD content to last on a single disk

– Restart VMs on any app failure

Nothing is static. Nothing lasts.

©2010 HP 9

HADOOP’S ASSUMPTIONS

– Master nodes don’t move

– Workers can spin for them

– Failed workers get blacklisted

– Single, static hostnames

– Cache all DNS entries

– Disks don't move between hosts

Different strategies are needed

©2010 HP 10

AGILE TEAMS

©2010 HP11

CLASSIC TEAM ROLES

Business Development

Architecture

OperationsDevelopment

©2010 HP 12

Business Development

Architecture

Operations

Development

THE OLD PROCESS

Design

Code TestStaging Live

©2010 HP 13

Architecture

DevelopmentBusiness

Development

Operations

CLOUD-HOSTING BLURS THE ROLES

Design

Code Test

LiveStaging

©2010 HP 14

CLOUD ARCHITECT

1. Agile Application

2. Scale up under load

3. Scale down when quiet

4. No known SPOF

5. Deploys with VMs

Design an agile, HA application from cloud services and VMs

©2010 HP 15

CLOUD OPERATIONS

– Instrumentation

– Reconfiguration

– Monitoring

– Reporting

– Datamining

Create the VMs, manage and monitor staging and production

©2010 HP16

ROLES IN CLOUD APIS

Different team roles need different rights and different service interfaces

©2010 HP17

ROLES IN THE CLOUD

Role Task Tooling

Cloud Architect Design App structure Text editor (?))Powerpoint (?

Cloud Operations Build VMs, set parametersManage production

Text under SCMWeb & Command line

Developers Request test VMs Web, IDE, build tools

Biz Dev Worry about costs and revenue

Web & Spreadsheet

©2010 HP18

CLOUDFARMER

©2010 HP19

Role Specification

machineRole extends HadoopVMRole { description "A basic machine with SmartFrog installed"; vmPrefix "vm"; bootAutoVol "vol-0-1-47"; min 0; recommendedMin 0; links extends Compound { "HTTP port 80" ["http", 80, "/"]; "HTTP port 8080" ["http", 8080, "/"]; }}

©2010 HP20

WEBAPP LISTS AVAILABLE ROLES

©2010 HP21

HADOOP CLUSTER: MASTER + WORKER

©2010 HP22

REQUEST COMPLETED

©2010 HP23

LIST HOSTS IN A ROLE

Actions: Add a worker, delete all in role “worker”

©2010 HP24

LIST ALL HOSTS

©2010 HP25

VIEW A HOST

©2010 HP26

Benefits over (VM type, disks+)

• Infrastructure can do some late binding install/deploy

• Feeds into the UI

• Web, IDE, build tools, ...

• “Role” CPU/network history can aid placement

• Other templates: network, aggregate clusters

©2010 HP 27

Q&A