from grid to cloud
DESCRIPTION
From the Gaming Scalability event, June 2009 in London (http://gamingscalability.org). Simon will discuss some of the key components of a compute grid infrastructure and highlight some of the key challenges organisations have to meet as their compute grids expand. Simon will also discuss one organisation within the spread betting industry who has recently started using grid technology. Finally Simon will describe how compute grids within the capital markets are beginning to resemble private clouds, and how the underlying infrastructure needs to change to enable these organisation to support a much wider range of applications running on the grid.Simon Waterer is a Senior Solutions Architect with Platform Computing, a leading provider of HPC software. Since joining Platform, Simon has worked with a number of clients within the capital markets and insurance industry to understand their grid computing requirements. Recently Simon has worked with leading organisations within the spread betting industry who also have distributed processing requirements. Prior to working with grid technology Simon has had experience working with a number of other middleware technologies including data caching, messaging middleware and event stream processing.TRANSCRIPT
![Page 1: From Grid to Cloud](https://reader036.vdocuments.us/reader036/viewer/2022062513/555086f6b4c905235b8b4ad5/html5/thumbnails/1.jpg)
From Grid To CloudOnline Gaming High Scalability SIG
Presented by: Simon WatererSolution Architect, Platform Computing
July 2009
![Page 2: From Grid to Cloud](https://reader036.vdocuments.us/reader036/viewer/2022062513/555086f6b4c905235b8b4ad5/html5/thumbnails/2.jpg)
Are Compute Grids Morphing Into Clouds?
12/04/2023 2
Do Cloud Infrastructures Share Features Found In Grids?
How Many Of You Are Using Compute Grid Technology?
![Page 3: From Grid to Cloud](https://reader036.vdocuments.us/reader036/viewer/2022062513/555086f6b4c905235b8b4ad5/html5/thumbnails/3.jpg)
Platform in the Gaming Industry
12/04/2023 3
TIP-EXFeed Handler, Grid Client
Compute Grid
……….
Excel Instances Running on The Grid
Data Cache
Oracle
Wholesale Channel
Client Web Site
Client Mobile
Trader/Modeller
![Page 4: From Grid to Cloud](https://reader036.vdocuments.us/reader036/viewer/2022062513/555086f6b4c905235b8b4ad5/html5/thumbnails/4.jpg)
Running Excel On the Grid
Grid Client
Scheduler
Grid Service/Engine
Excel Instance
Grid Node/Host. Multiple Service/Engines run per CPU/core
Spreadsheet serialised at client and passed by grid middleware to grid node
Dialog Sniffer used for debugging spreadsheets that are to be run on the grid
![Page 5: From Grid to Cloud](https://reader036.vdocuments.us/reader036/viewer/2022062513/555086f6b4c905235b8b4ad5/html5/thumbnails/5.jpg)
The Advantages of Grid?
• Handles allocation of resources– Automatic– Shared Resources
• Handle Failures– The grid provides redundancy
• So What?– Run more workload using fewer resources
12/04/2023 5
![Page 6: From Grid to Cloud](https://reader036.vdocuments.us/reader036/viewer/2022062513/555086f6b4c905235b8b4ad5/html5/thumbnails/6.jpg)
Grid Components
12/04/2023 6
ApplicationInput Output
ClientInput
ServiceOutput
Grid Infrastructure
![Page 7: From Grid to Cloud](https://reader036.vdocuments.us/reader036/viewer/2022062513/555086f6b4c905235b8b4ad5/html5/thumbnails/7.jpg)
Grid Components
• Application on-boarding• Scheduler• Middleware• Workload monitoring• Workload management• Data Grid• Reporting
• Resource allocation• Resource policy• Cluster management• Reporting
12/04/2023 7
Application Orchestration
Resource Orchestration
![Page 8: From Grid to Cloud](https://reader036.vdocuments.us/reader036/viewer/2022062513/555086f6b4c905235b8b4ad5/html5/thumbnails/8.jpg)
Phases of Grid Adoption
12/04/2023 8
LOBLOBLOB
LOB LOB
Phase 1 Phase 2 Phase 3 Phase 4
AA AA A
LOB
LOBLOB
Phase 1: Silo Grid – Grid enable application running on a commodity cluster
Phase 2: LOB Grid – Resource sharing among multiple applications
Phase 3: Enterprise Analytics Grid – Utility computing; enterprise scale and management
Phase 4: Enterprise Grid – Beyond analytics: commercial applications on grid
Phase 1: Silo Grid – Grid enable application running on a commodity cluster
Phase 2: LOB Grid – Resource sharing among multiple applications
Phase 3: Enterprise Analytics Grid – Utility computing; enterprise scale and management
Phase 4: Enterprise Grid – Beyond analytics: commercial applications on grid
A AA A
A AA A
A AA A
A AA A
A AA A
![Page 9: From Grid to Cloud](https://reader036.vdocuments.us/reader036/viewer/2022062513/555086f6b4c905235b8b4ad5/html5/thumbnails/9.jpg)
Grid As A Service
12/04/2023 9
Grid Infrastructure
FX, Equities, Credit Derivatives Dev/Test
• Developers and testers get environments in 15 minutes, instead of waiting 2 weeks
• IT administrators eliminate manual setup and repurposing work
• Automated tracking of utilization with accurate billing to the LOB’s for what they use
• Integrated cloud for Test/Dev through to production application deployment
• Self-service environment reservation with automated, policy-driven VM placement
![Page 10: From Grid to Cloud](https://reader036.vdocuments.us/reader036/viewer/2022062513/555086f6b4c905235b8b4ad5/html5/thumbnails/10.jpg)
04/12/2023 10
Dev Network
UAT Network
Prod Network
LOB CLOB BLOB A
Compute Grid
Free Pool
Jan Contract: 12 machines Jan Contract: 14 machines Jan Contract: 10 machinesSelf-Service Resource Re-allocation Self-Service Resource Re-allocation
Feb Contract: 16 machines Feb Contract: 6 machines Feb Contract: 12 machines
Dev/UAT/Production & Production/Production Sharing
10
![Page 11: From Grid to Cloud](https://reader036.vdocuments.us/reader036/viewer/2022062513/555086f6b4c905235b8b4ad5/html5/thumbnails/11.jpg)
Bursting to non-HPC Servers
HPC Grid
Non-HPC servers
VM
VM VM
11
• Calendar-driven startup/shutdown of VMs• Dynamic addition of non-HPC VMs to HPC Grid
VM
![Page 12: From Grid to Cloud](https://reader036.vdocuments.us/reader036/viewer/2022062513/555086f6b4c905235b8b4ad5/html5/thumbnails/12.jpg)
Cloud Bursting For Peak Demand
GridInfrastructure
ProvisioiningAdaptor
Detect When Resources Are
Exhausted
12
Web 2.0Customer App
GridOverflow Pool
Web 2.0Partner App
Public Clouds
Private Data Centre
• Public Clouds used for cloud-bursting applications
![Page 13: From Grid to Cloud](https://reader036.vdocuments.us/reader036/viewer/2022062513/555086f6b4c905235b8b4ad5/html5/thumbnails/13.jpg)
3 Paths to Cloud Adoption
Time
Sco
pe
of
shar
ing
2003 20152009
Client/ServerSilos
Workgroup/ LOB
VM Cluster• Homogeneous server
consolidation• Limited apps• Basic VM mgmt
Grid• Dynamic workload using
static resources• Policy-based scheduling
Enterprise
Public & Private Cloud
13
Cloud – “A pool of abstracted, highly scalable, and managed infrastructure capable of hosting end-customer applications and billed by consumption” (Forrester)
![Page 14: From Grid to Cloud](https://reader036.vdocuments.us/reader036/viewer/2022062513/555086f6b4c905235b8b4ad5/html5/thumbnails/14.jpg)
Private and Public Cloud
Private Cloud by Corporate IT• Maximize value of underutilized
resources• Mission critical SLAs• High security & compliance
requirements• Enterprise-specific services
Public Cloud by Service Providers• Non-mission critical SLAs• In-house IT has limited scale, scope or
expertise --- SMEs
Public Cloud
Company X
Private Cloud
Company Y
14
![Page 15: From Grid to Cloud](https://reader036.vdocuments.us/reader036/viewer/2022062513/555086f6b4c905235b8b4ad5/html5/thumbnails/15.jpg)
Enterprise Adoption of Cloud
• IT delivering faster services with uniform UIs
• Increased utilization of existing resources
• Controlled overflow to Public clouds to meet unpredictable workload spikes
• Lower CapEx & OpEx
Company X
$ $$ $$$
15
Private Cloud Augmented by Public Cloud
![Page 16: From Grid to Cloud](https://reader036.vdocuments.us/reader036/viewer/2022062513/555086f6b4c905235b8b4ad5/html5/thumbnails/16.jpg)
04/12/2023 16
Different Cloud Services
Software-as-a-Service (SaaS)
Infrastructure-as-a-Service (IaaS)
Platform-as-a-Service (PaaS)
Private Public
![Page 17: From Grid to Cloud](https://reader036.vdocuments.us/reader036/viewer/2022062513/555086f6b4c905235b8b4ad5/html5/thumbnails/17.jpg)
IaaS Usage Model
ESX
Virtual Machine
Virtual Machine
Virtual Machine
Virtual Machine
Virtual Machine
Virtual Machine
PhysicalMachines
IaaS Cloud
Cloud Admin
User
Application Manager
• Sign up for services• Prioritize & allocate
resource quotas
• Define resource inventory
• Publish & deliver services
• Manage costs
• Request & use resources, subject to quota
17
VirtualMachines
![Page 18: From Grid to Cloud](https://reader036.vdocuments.us/reader036/viewer/2022062513/555086f6b4c905235b8b4ad5/html5/thumbnails/18.jpg)
IaaS Benefits
IaaS Cloud
Application Manager
User
CXO
Cloud Admin
• Get machines in minutes instead of days/weeks• Lower costs, pay by
actual usage• Prioritize my
application needs
• Lower IT costs• Faster response to
business• Timely delivery• Increased utilization• Lower CapEx & OpEx• Reduced human errors
18
![Page 19: From Grid to Cloud](https://reader036.vdocuments.us/reader036/viewer/2022062513/555086f6b4c905235b8b4ad5/html5/thumbnails/19.jpg)
04/12/2023 19
IaaS Components
Manual
Allocation Engine
Service Delivery
Resource Integrations
Creating a shared computing
infrastructure from physical & virtual
heterogeneous resources
Delivering app environments according to
workload-aware & resource-aware
policies
![Page 20: From Grid to Cloud](https://reader036.vdocuments.us/reader036/viewer/2022062513/555086f6b4c905235b8b4ad5/html5/thumbnails/20.jpg)
Questions?
![Page 21: From Grid to Cloud](https://reader036.vdocuments.us/reader036/viewer/2022062513/555086f6b4c905235b8b4ad5/html5/thumbnails/21.jpg)
Summary
• Grids• Exhibiting cloud tendencies
• Private Clouds• Resource and workload aware allocation are key• Policy driven
• Platform Symphony• www.platform.com
• Platform ISF Beta• Launched July
• Head in The Cloud• YouTube, Amazon, Stanford University
12/04/2023 21