hpc over cloud hands-on: cluster automation with amazon ... · corporate presentation web services...
TRANSCRIPT
![Page 1: HPC over Cloud Hands-on: Cluster automation with Amazon ... · Corporate Presentation Web Services • What are WS WS are distributed application components in a client-server model](https://reader033.vdocuments.us/reader033/viewer/2022052019/6032920cda06345b15679328/html5/thumbnails/1.jpg)
HPC over CloudHands-on: Cluster automation with Amazon Web Services
Cristian Mezzanotte
17.04.2012
![Page 2: HPC over Cloud Hands-on: Cluster automation with Amazon ... · Corporate Presentation Web Services • What are WS WS are distributed application components in a client-server model](https://reader033.vdocuments.us/reader033/viewer/2022052019/6032920cda06345b15679328/html5/thumbnails/2.jpg)
© D
atal
ynx A
G
| S
lide
2| 1
9-A
pr-
12
| C
rist
ian
Mez
zan
ott
e | C
orp
ora
te P
rese
nta
tion
Overview
• Datalynx AG
• Web Services and Cloud
• AWS
• Hands-on!
• Conclusions
![Page 3: HPC over Cloud Hands-on: Cluster automation with Amazon ... · Corporate Presentation Web Services • What are WS WS are distributed application components in a client-server model](https://reader033.vdocuments.us/reader033/viewer/2022052019/6032920cda06345b15679328/html5/thumbnails/3.jpg)
© D
atal
ynx A
G
| S
lide
3| 1
9-A
pr-
12
| C
rist
ian
Mez
zan
ott
e | C
orp
ora
te P
rese
nta
tion
Assumptions
• You have a basic knowledge of Cloud Computing concepts
• You know the difference between public, private and hybrid Clouds
• You are familiar with High Performance Computing concepts
![Page 4: HPC over Cloud Hands-on: Cluster automation with Amazon ... · Corporate Presentation Web Services • What are WS WS are distributed application components in a client-server model](https://reader033.vdocuments.us/reader033/viewer/2022052019/6032920cda06345b15679328/html5/thumbnails/4.jpg)
© D
atal
ynx A
G
| S
lide
4| 1
9-A
pr-
12
| C
rist
ian
Mez
zan
ott
e | C
orp
ora
te P
rese
nta
tion
Datalynx AG – www.datalynx.com
• Fast growing, customer-focusd, IT company based in Switzerland
• Competences in IT Architectures, Security, Project Management, Quality Management
• Advice. Create. Manage. Secure. Collaborate.
• Customers: Novartis, Roche, Credit Suisse, Swisscom, Adidas, Swiss Post, etc..
![Page 5: HPC over Cloud Hands-on: Cluster automation with Amazon ... · Corporate Presentation Web Services • What are WS WS are distributed application components in a client-server model](https://reader033.vdocuments.us/reader033/viewer/2022052019/6032920cda06345b15679328/html5/thumbnails/5.jpg)
© D
atal
ynx A
G
| S
lide
5| 1
9-A
pr-
12
| C
rist
ian
Mez
zan
ott
e | C
orp
ora
te P
rese
nta
tion
Web Services
• What are WSWS are distributed application components in aclient-server model
Do not confuse Web Services with Web Pages Web Pages are for human, Web Services are formachines
• Why use WSInteroperability, available on the WWW, open standards
• How do WS workHTTP, XML, SOAP, WSDL, UDDI
![Page 6: HPC over Cloud Hands-on: Cluster automation with Amazon ... · Corporate Presentation Web Services • What are WS WS are distributed application components in a client-server model](https://reader033.vdocuments.us/reader033/viewer/2022052019/6032920cda06345b15679328/html5/thumbnails/6.jpg)
© D
atal
ynx A
G
| S
lide
6| 1
9-A
pr-
12
| C
rist
ian
Mez
zan
ott
e | C
orp
ora
te P
rese
nta
tion
Amazon Web Services
• Public Cloud framework
• Infrastructure as a Service (IaaS)
• Management Console
• API
• Based on Web Services
![Page 7: HPC over Cloud Hands-on: Cluster automation with Amazon ... · Corporate Presentation Web Services • What are WS WS are distributed application components in a client-server model](https://reader033.vdocuments.us/reader033/viewer/2022052019/6032920cda06345b15679328/html5/thumbnails/7.jpg)
© D
atal
ynx A
G
| S
lide
7| 1
9-A
pr-
12
| C
rist
ian
Mez
zan
ott
e | C
orp
ora
te P
rese
nta
tion
Amazon Web Services
Elastic Computing Cloud (EC2)
• Create and select new or templeted images of any Operative System, libraries, data, etc..
• Configure network and security settings
• Start, terminate and monitor your instances
• Manage locations, IP endpoints and storage
• Pay-as-you-go
![Page 8: HPC over Cloud Hands-on: Cluster automation with Amazon ... · Corporate Presentation Web Services • What are WS WS are distributed application components in a client-server model](https://reader033.vdocuments.us/reader033/viewer/2022052019/6032920cda06345b15679328/html5/thumbnails/8.jpg)
© D
atal
ynx A
G
| S
lide
8| 1
9-A
pr-
12
| C
rist
ian
Mez
zan
ott
e | C
orp
ora
te P
rese
nta
tion
Amazon Web Services
Simple Storage Service (Amazon S3)
• Write, read and delete unlimited objects containing up to 5TB of data
• Data security tools
• Authentication
• Sharing
![Page 9: HPC over Cloud Hands-on: Cluster automation with Amazon ... · Corporate Presentation Web Services • What are WS WS are distributed application components in a client-server model](https://reader033.vdocuments.us/reader033/viewer/2022052019/6032920cda06345b15679328/html5/thumbnails/9.jpg)
© D
atal
ynx A
G
| S
lide
9| 1
9-A
pr-
12
| C
rist
ian
Mez
zan
ott
e | C
orp
ora
te P
rese
nta
tion
Amazon Web Services
Cloud Formation
• Template resources
• Create templates of machines and create instances in AWS
• Stack your infrastructure and replicate it into AWS
![Page 10: HPC over Cloud Hands-on: Cluster automation with Amazon ... · Corporate Presentation Web Services • What are WS WS are distributed application components in a client-server model](https://reader033.vdocuments.us/reader033/viewer/2022052019/6032920cda06345b15679328/html5/thumbnails/10.jpg)
© D
atal
ynx A
G
| S
lide
10
| 1
9-A
pr-
12
| C
rist
ian
Mez
zan
ott
e | C
orp
ora
te P
rese
nta
tion
Amazon Web Services
EMR
VPC
![Page 11: HPC over Cloud Hands-on: Cluster automation with Amazon ... · Corporate Presentation Web Services • What are WS WS are distributed application components in a client-server model](https://reader033.vdocuments.us/reader033/viewer/2022052019/6032920cda06345b15679328/html5/thumbnails/11.jpg)
© D
atal
ynx A
G
| S
lide
11
| 1
9-A
pr-
12
| C
rist
ian
Mez
zan
ott
e | C
orp
ora
te P
rese
nta
tion
Amazon Web Services trends
![Page 12: HPC over Cloud Hands-on: Cluster automation with Amazon ... · Corporate Presentation Web Services • What are WS WS are distributed application components in a client-server model](https://reader033.vdocuments.us/reader033/viewer/2022052019/6032920cda06345b15679328/html5/thumbnails/12.jpg)
© D
atal
ynx A
G
| S
lide
12
| 1
9-A
pr-
12
| C
rist
ian
Mez
zan
ott
e | C
orp
ora
te P
rese
nta
tion
The Simulation
• Collaboration with Milan University
• Replica Exchange Molecular Dynamics (REMD) simulation with Amber11
• Pellegrino, Contini et al, Chem. Eur. J 2012, DOI: 10.1002/chem.201104023
![Page 13: HPC over Cloud Hands-on: Cluster automation with Amazon ... · Corporate Presentation Web Services • What are WS WS are distributed application components in a client-server model](https://reader033.vdocuments.us/reader033/viewer/2022052019/6032920cda06345b15679328/html5/thumbnails/13.jpg)
© D
atal
ynx A
G
| S
lide
13
| 1
9-A
pr-
12
| C
rist
ian
Mez
zan
ott
e | C
orp
ora
te P
rese
nta
tion
Cluster Instance
Img_remd
/vol
Img_remd
/vol
Img_remd
/vol
Img_remd
/vol
![Page 14: HPC over Cloud Hands-on: Cluster automation with Amazon ... · Corporate Presentation Web Services • What are WS WS are distributed application components in a client-server model](https://reader033.vdocuments.us/reader033/viewer/2022052019/6032920cda06345b15679328/html5/thumbnails/14.jpg)
© D
atal
ynx A
G
| S
lide
14
| 1
9-A
pr-
12
| C
rist
ian
Mez
zan
ott
e | C
orp
ora
te P
rese
nta
tion
Big data file transfer service
• The amount of data to be processed and moved is growing fast
• Traditional file transfer tools (ftp, scp..) are inadequate
• Gridftp greatly improves performance up to 300 MB/s (LRZ), 200 MB/s (Teragrid)
![Page 15: HPC over Cloud Hands-on: Cluster automation with Amazon ... · Corporate Presentation Web Services • What are WS WS are distributed application components in a client-server model](https://reader033.vdocuments.us/reader033/viewer/2022052019/6032920cda06345b15679328/html5/thumbnails/15.jpg)
© D
atal
ynx A
G
| S
lide
15
| 1
9-A
pr-
12
| C
rist
ian
Mez
zan
ott
e | C
orp
ora
te P
rese
nta
tion
HPC over Cloud vs Traditional HPC
• Availability of resources
• No initial investment on infrastructure
• Pay what you use
• Scalability
• Versatility
• Automation
![Page 16: HPC over Cloud Hands-on: Cluster automation with Amazon ... · Corporate Presentation Web Services • What are WS WS are distributed application components in a client-server model](https://reader033.vdocuments.us/reader033/viewer/2022052019/6032920cda06345b15679328/html5/thumbnails/16.jpg)
© D
atal
ynx A
G
| S
lide
16
| 1
9-A
pr-
12
| C
rist
ian
Mez
zan
ott
e | C
orp
ora
te P
rese
nta
tion
Challenges
• Security
• Billing system integration
• Lower performance in some cases
• Licences in some cases
![Page 17: HPC over Cloud Hands-on: Cluster automation with Amazon ... · Corporate Presentation Web Services • What are WS WS are distributed application components in a client-server model](https://reader033.vdocuments.us/reader033/viewer/2022052019/6032920cda06345b15679328/html5/thumbnails/17.jpg)
© D
atal
ynx A
G
| S
lide
17
| 1
9-A
pr-
12
| C
rist
ian
Mez
zan
ott
e | C
orp
ora
te P
rese
nta
tionThank you!
Contacts: