open source toolchains to manage cloud infrastructure
DESCRIPTION
Open Source Toolchains to Manage Cloud Infrastructure presentation for Cloud Computing Expo East - June 6, 2011. Added APIs (jclouds, fog, libcloud, deltacloud)TRANSCRIPT
![Page 1: Open Source Toolchains to Manage Cloud Infrastructure](https://reader036.vdocuments.us/reader036/viewer/2022062405/554fa3e7b4c90586258b4b40/html5/thumbnails/1.jpg)
Mark R. Hinkle VP of Community Cloud.com Twitter: @clouddotcom @mrhinkle Email: [email protected] [email protected]
Open Source Toolchains to Manage Cloud Infrastructure
![Page 2: Open Source Toolchains to Manage Cloud Infrastructure](https://reader036.vdocuments.us/reader036/viewer/2022062405/554fa3e7b4c90586258b4b40/html5/thumbnails/2.jpg)
2
Introduction • Responsible for Driving Adoption of
CloudStack Open Source Cloud Computing Software
• Former manager of Zenoss Open Source project 100,000 users, 1.5 million downloads
• Former Linux Desktop Advocate (Zealot?) • Former LinuxWorld Magazine Editor-in-Chief • Open Management Consortium Conspirator • Open Desktop Consortium Instigator • Author - “Windows to Linux Business Desktop
Migration” - Thomson • NetDirector Project - Open Source
Con!guration Management Project • Sometimes Author and Blogger at
SocializedSoftware.com/NetworkWorld • Start-up junkie, Glutton for punishment
![Page 3: Open Source Toolchains to Manage Cloud Infrastructure](https://reader036.vdocuments.us/reader036/viewer/2022062405/554fa3e7b4c90586258b4b40/html5/thumbnails/3.jpg)
3
Agenda
• Introductions • Cloud & Systems Management Overview • Open Source Tools for Cloud
Management • Creating Open Source Toolchains • Questions (and Hopefully Answers)
![Page 4: Open Source Toolchains to Manage Cloud Infrastructure](https://reader036.vdocuments.us/reader036/viewer/2022062405/554fa3e7b4c90586258b4b40/html5/thumbnails/4.jpg)
Cloud Computing & Systems Management Overview
![Page 5: Open Source Toolchains to Manage Cloud Infrastructure](https://reader036.vdocuments.us/reader036/viewer/2022062405/554fa3e7b4c90586258b4b40/html5/thumbnails/5.jpg)
5
Jevon’s Paradox and the Cloud
1865, the English economist William Stanley Jevons
…the efficiency with which a resource is used tends to increase (rather than decrease) the rate of consumption of that resource.
…in cloud computing increased consumption can introduces signi!cant management overhead despite higher hardware utilization and other bene!ts
![Page 6: Open Source Toolchains to Manage Cloud Infrastructure](https://reader036.vdocuments.us/reader036/viewer/2022062405/554fa3e7b4c90586258b4b40/html5/thumbnails/6.jpg)
6
Balancing the Equation: Creating the Cloud
+ =
![Page 7: Open Source Toolchains to Manage Cloud Infrastructure](https://reader036.vdocuments.us/reader036/viewer/2022062405/554fa3e7b4c90586258b4b40/html5/thumbnails/7.jpg)
7
Balancing the Equation: Managing the Cloud
+ = ?
![Page 8: Open Source Toolchains to Manage Cloud Infrastructure](https://reader036.vdocuments.us/reader036/viewer/2022062405/554fa3e7b4c90586258b4b40/html5/thumbnails/8.jpg)
8 Automation Balances the Equation • MeatCloud, Can’t Keep up with
Cloud Computing • Devops & Agile IT Philosophy • Script Repetitive Tasks • Integrate Tools • Automate, Automate, Automate
![Page 9: Open Source Toolchains to Manage Cloud Infrastructure](https://reader036.vdocuments.us/reader036/viewer/2022062405/554fa3e7b4c90586258b4b40/html5/thumbnails/9.jpg)
9 DevOps – Agile Management for the Cloud
• Agile, fast moving • Smaller incremental change • Increased coordination across
groups (development/operations) • Eye towards Automation
![Page 10: Open Source Toolchains to Manage Cloud Infrastructure](https://reader036.vdocuments.us/reader036/viewer/2022062405/554fa3e7b4c90586258b4b40/html5/thumbnails/10.jpg)
10
Management Toolchains
Con!guration
Patching and Provisioning
Monitoring Toolchain (n): A set of tools where the output of one tool becomes the input of another tool
![Page 11: Open Source Toolchains to Manage Cloud Infrastructure](https://reader036.vdocuments.us/reader036/viewer/2022062405/554fa3e7b4c90586258b4b40/html5/thumbnails/11.jpg)
11
The Myth of the Nines Availability % Downtime per
Year Downtime per Month
Downtime per Week
99.9% (three nines) 8.76 hours 43.2 minutes 10.1 minutes
99.95% 4.38 hours 21.56 minutes 5.04 minutes
99.99% (four nines) 52.6 minutes 4.32 minutes 1.01 minutes
99.999% (!ve nines) 5.26 minutes 25.9 seconds 6.05 seconds
99.9999% (six nines) 31.5 seconds 2.59 seconds .0605 seconds
Average polling interval for monitoring? 5 minutes? Even superhuman operations people can’t be alerted and take action in under 5 minutes.
One outage per year could drop service level to three nines or worse.
![Page 12: Open Source Toolchains to Manage Cloud Infrastructure](https://reader036.vdocuments.us/reader036/viewer/2022062405/554fa3e7b4c90586258b4b40/html5/thumbnails/12.jpg)
Open Source Tools to Manage the Cloud
![Page 13: Open Source Toolchains to Manage Cloud Infrastructure](https://reader036.vdocuments.us/reader036/viewer/2022062405/554fa3e7b4c90586258b4b40/html5/thumbnails/13.jpg)
13
2011 CLOUD COMPUTING OUTLOOK SURVEY http://www.cloud.com/cloud-computing-outlook
![Page 14: Open Source Toolchains to Manage Cloud Infrastructure](https://reader036.vdocuments.us/reader036/viewer/2022062405/554fa3e7b4c90586258b4b40/html5/thumbnails/14.jpg)
14
4 Types of Management Tools
Provisioning Installation of operating systems and other software
Configuration Management Sets the parameters for servers, can specify installation parameters
Orchestration/Automation Automate tasks across systems
Monitoring Records errors and health of IT infrastructure
![Page 15: Open Source Toolchains to Manage Cloud Infrastructure](https://reader036.vdocuments.us/reader036/viewer/2022062405/554fa3e7b4c90586258b4b40/html5/thumbnails/15.jpg)
15
Cloudy Tools
• Network Capable • Cloud “Aware” • Easy-to-Integrate • Adhere to Open Standards • Lend Themselves to Automation
![Page 16: Open Source Toolchains to Manage Cloud Infrastructure](https://reader036.vdocuments.us/reader036/viewer/2022062405/554fa3e7b4c90586258b4b40/html5/thumbnails/16.jpg)
16 Open Source Provisioning Tools
Year Started Language License Installation Targets
Cobbler 2007 Python GPL Red Hat, OpenSUSE Fedora, Debian, Ubuntu
Fully Automatic Installation (FAI)
2000 Perl GPL Debian
Kickstart ? Python GPL Most .dep and RPM based Linux distros
Spacewalk 2008 Perl, Python, Java GPL Fedora, Centos
Crowbar/Chef 2011 Ruby Apache ?
![Page 17: Open Source Toolchains to Manage Cloud Infrastructure](https://reader036.vdocuments.us/reader036/viewer/2022062405/554fa3e7b4c90586258b4b40/html5/thumbnails/17.jpg)
17 Open Source Configuration Management Tools
Year Started Language License Client/Server
Bcfg2 2003 Python BSD Yes
Cfengine 1993 C Apache Yes
Chef 2009 Ruby Apache Chef Solo – No Chef Server - Yes
Puppet 2004 Ruby GPL yes
![Page 18: Open Source Toolchains to Manage Cloud Infrastructure](https://reader036.vdocuments.us/reader036/viewer/2022062405/554fa3e7b4c90586258b4b40/html5/thumbnails/18.jpg)
18 Open Source Monitoring Tools
Year Started License Language Type of Monitoring
Collection Methods
Cacti (RRDTool) 2001 GPL PHP Performance SNMP, syslog
Nagios 1999 GPL C/PHP Availability SNMP,TCP, ICMP, IPMI, syslog
OpenNMS 2000 GPL Java Availability/Performance
SNMP,
Zabbix 2001 GPL C/PHP Availability/ Performance and more
SNMP, TCP/ICMP, IPMI, Synthetic Transactions
Zenoss 2005 GPL Python Availability, Performance, Event Management
SNMP, ICMP, SSH, syslog, WMI
![Page 19: Open Source Toolchains to Manage Cloud Infrastructure](https://reader036.vdocuments.us/reader036/viewer/2022062405/554fa3e7b4c90586258b4b40/html5/thumbnails/19.jpg)
19 Open Source Automation/Orchestration Tools
Year Started
Language License Client/Server
Support Organization
Capistrano 2006 Ruby MIT Yes None
RunDeck 2010 Java Apache Yes DTO Solutions
Func 2007 Python GPL Yes Fedora Project
MCollective 2009 Ruby Apache Yes PuppetLabs
![Page 20: Open Source Toolchains to Manage Cloud Infrastructure](https://reader036.vdocuments.us/reader036/viewer/2022062405/554fa3e7b4c90586258b4b40/html5/thumbnails/20.jpg)
20 Cloud APIs Aren’t Created Equal Open Source Abstractions
API Language
jclouds java
deltacloud python
Lib cloud python
Fog Ruby
![Page 21: Open Source Toolchains to Manage Cloud Infrastructure](https://reader036.vdocuments.us/reader036/viewer/2022062405/554fa3e7b4c90586258b4b40/html5/thumbnails/21.jpg)
Open Source Management Toolchains
![Page 22: Open Source Toolchains to Manage Cloud Infrastructure](https://reader036.vdocuments.us/reader036/viewer/2022062405/554fa3e7b4c90586258b4b40/html5/thumbnails/22.jpg)
22
Toolchain Concept
Source:http://code.google.com/p/devops-toolchain/wiki/ArchitectureComponents
![Page 23: Open Source Toolchains to Manage Cloud Infrastructure](https://reader036.vdocuments.us/reader036/viewer/2022062405/554fa3e7b4c90586258b4b40/html5/thumbnails/23.jpg)
23
Automated Toolchain
Cloud Image Launch
OS Install
System Con!guration
Application Service Orchestration
Capistrano RunDeck Fabric Func
BCFG2 Cfengine Chef Puppet
Cloud: Eucalyptus OpenStack CloudStack Abiquo
OS Install: Kickstart Cobbler Spacewalk
Command and Control
Con!guration
Bootstrapping Prov
isio
ning
Act
ivity
![Page 24: Open Source Toolchains to Manage Cloud Infrastructure](https://reader036.vdocuments.us/reader036/viewer/2022062405/554fa3e7b4c90586258b4b40/html5/thumbnails/24.jpg)
24
Example: GeekNet
Hundreds of servers, serving web, databases, and other infrastructure for some of the world’s most highly trafficked websites – over 40 million visitors per month.
• Servers are automatically built using con!guration management software (Puppet)
• Discovery tool (Zenoss) !nds infrastructure and populates a CMDB then spits out information to scripts that translate information to BIND con!gurations for DNS
• Monitoring tool adds hosts to polling tool (Nagios) to check servers for availability
• As infrastructure changes systems are updated automatically
• Servers can be spun up and managed in minutes, not hours automatically with little or no human interaction
![Page 25: Open Source Toolchains to Manage Cloud Infrastructure](https://reader036.vdocuments.us/reader036/viewer/2022062405/554fa3e7b4c90586258b4b40/html5/thumbnails/25.jpg)
25 Example Toolchain: Online Gaming Studio
![Page 26: Open Source Toolchains to Manage Cloud Infrastructure](https://reader036.vdocuments.us/reader036/viewer/2022062405/554fa3e7b4c90586258b4b40/html5/thumbnails/26.jpg)
Questions?
![Page 27: Open Source Toolchains to Manage Cloud Infrastructure](https://reader036.vdocuments.us/reader036/viewer/2022062405/554fa3e7b4c90586258b4b40/html5/thumbnails/27.jpg)
27
Contact Me Professional: [email protected] Personal: [email protected]
Professional: [email protected] Personal: [email protected]
Professional: http://open.cloud.com Personal: http://socializedsoftware.com
Twitter: @clouddotcom @mrhinkle
Mark R. Hinkle
VP of Community Open Source Enthusiast
![Page 28: Open Source Toolchains to Manage Cloud Infrastructure](https://reader036.vdocuments.us/reader036/viewer/2022062405/554fa3e7b4c90586258b4b40/html5/thumbnails/28.jpg)
Open Source Toolchains to Manage Cloud Computing
http://www.slideshare.net/socializedsoftware
by Mark R. Hinkle is licensed under a Creative Commons Attribution-ShareAlike 3.0 United
States License.