GPU Computing with Amazon EC2

Deepak Singh@mndoci

P r i n c i p a l P r o d u c t M a n a g e r

Amazon Web Services

4

2

1. Infrastructure

ec2-run-instances

secure

global

on demand

programmable

elastic

instance types

standard (m1)high memory (m2)

high CPU (c1)

t1.micro

high performance

“Our 40-instance (m2.2xlarge) cluster can scan, filter, and aggregate 1 billion rows in 950 milliseconds.”

Mike Driscoll - Metamarkets

cluster computing

MPI

bandwidth intensive

Cluster Compute Instance

2*Intel Xeon 55708 cores w/HT23 GB RAM1.7 TB disk

HVMcc1.4xlarge

10 gig E

PlacementGroup

Placementgroup

full-

bisection

linpack

Cores 7040

Rmax 41.82

Rpeak 82.51

231

November 2010

451

June 2011

42

November 2011

HPC is evolving

2*Intel Xeon 557022 GB RAM1.7 TB disk

HVM2*NVidia M2050

AMBER

JAC NVE: 46.92 ns/day

Factor IX: 13.49 ns/day

Cellulose NVE: 3.07 ns/day

optimizing costs

on-demand

reserved

spot

30,472 cores

$1279/hr

2. Orchestration

AWS CloudFormation

bootstrap

Cloud Init

#cloud-configpackages:! - httpd!runcmd:! - /etc/init.d http start! - echo "<h1>hello, world"</h1> \! ! > /var/www/html/index.html

#!/bin/shec2-run-instances ami-8c1fece5 \! -n 1 \! -t m1.small \! -g deesinghdemo-SG \! -k deesinghdemo-keypair \! --user-data-file \.\cloudconfig.txt

chef/puppet

familiar tools

LSF

Grid Engine

Bright Cluster Manager

combining worlds

MIT Starcluster

$ starcluster start mycluster$ starcluster listclusters

http://www.bioteam.net/2011/03/dude-you-got-some-chef-in-my-starcluster/

Provisions ClusterShared Storage

MonitoringBootstraps StarCluster

Includes 200 GB Public Dataset

Provisioned Stack = Submit jobs to Grid Engine

Image: Chris Dagdigian