Download - Rapid scaling in_the_cloud_with_puppet
![Page 2: Rapid scaling in_the_cloud_with_puppet](https://reader036.vdocuments.us/reader036/viewer/2022062418/554bb766b4c9053a298b4a13/html5/thumbnails/2.jpg)
What is Puppet?
Express infrastructure as....• code
o Manage your infrastructure just like softwareo Use version controlo QA changeso Continuous Integrationo Prevent problems from re-occurring
![Page 3: Rapid scaling in_the_cloud_with_puppet](https://reader036.vdocuments.us/reader036/viewer/2022062418/554bb766b4c9053a298b4a13/html5/thumbnails/3.jpg)
What is Puppet?
Express infrastructure as....• code• resources
o What, not howo Relationships not order
![Page 4: Rapid scaling in_the_cloud_with_puppet](https://reader036.vdocuments.us/reader036/viewer/2022062418/554bb766b4c9053a298b4a13/html5/thumbnails/4.jpg)
What is Puppet?
Express infrastructure as....• code• resources• state
o Idempotento What, not how
![Page 5: Rapid scaling in_the_cloud_with_puppet](https://reader036.vdocuments.us/reader036/viewer/2022062418/554bb766b4c9053a298b4a13/html5/thumbnails/5.jpg)
The Cycle
![Page 6: Rapid scaling in_the_cloud_with_puppet](https://reader036.vdocuments.us/reader036/viewer/2022062418/554bb766b4c9053a298b4a13/html5/thumbnails/6.jpg)
The Cycle
![Page 7: Rapid scaling in_the_cloud_with_puppet](https://reader036.vdocuments.us/reader036/viewer/2022062418/554bb766b4c9053a298b4a13/html5/thumbnails/7.jpg)
Resources
Resources are the building blocks of puppetAll resources have:• type• title• attributes
file { '/etc/motd': ensure => file, owner => 'root', content => 'Managed by Puppet', mode => 0755,}
![Page 8: Rapid scaling in_the_cloud_with_puppet](https://reader036.vdocuments.us/reader036/viewer/2022062418/554bb766b4c9053a298b4a13/html5/thumbnails/8.jpg)
The Resource Abstraction Layer (RAL)
The Resource Abstraction Layer allows puppet to introspect the system about resource types.
![Page 9: Rapid scaling in_the_cloud_with_puppet](https://reader036.vdocuments.us/reader036/viewer/2022062418/554bb766b4c9053a298b4a13/html5/thumbnails/9.jpg)
Modules
Modules contain everything puppet needs to manage something.
For example:• apache• bacula• mysql• subversion• etc
![Page 10: Rapid scaling in_the_cloud_with_puppet](https://reader036.vdocuments.us/reader036/viewer/2022062418/554bb766b4c9053a298b4a13/html5/thumbnails/10.jpg)
Modules
Layout:
module_name | -- manifests (puppet code) | -- files (files to serve to clients) | -- templates (ERB templates) | -- lib (puppet plugins)
![Page 11: Rapid scaling in_the_cloud_with_puppet](https://reader036.vdocuments.us/reader036/viewer/2022062418/554bb766b4c9053a298b4a13/html5/thumbnails/11.jpg)
Modules
Where do I get them?
The Forgehttp://forge.puppetlabs.com
Githubhttp://github.comPuppet Module Tool# puppet-module install puppetlabs/apache
![Page 12: Rapid scaling in_the_cloud_with_puppet](https://reader036.vdocuments.us/reader036/viewer/2022062418/554bb766b4c9053a298b4a13/html5/thumbnails/12.jpg)
QAQ: How do I QA my puppet code before pushing to production?
![Page 13: Rapid scaling in_the_cloud_with_puppet](https://reader036.vdocuments.us/reader036/viewer/2022062418/554bb766b4c9053a298b4a13/html5/thumbnails/13.jpg)
QAQ: How do I QA my puppet code before pushing to production?A: Environments!!
![Page 14: Rapid scaling in_the_cloud_with_puppet](https://reader036.vdocuments.us/reader036/viewer/2022062418/554bb766b4c9053a298b4a13/html5/thumbnails/14.jpg)
The GraphPuppet uses a graph to know the relationship between resources
package { 'ntp': ensure => present, }
file { '/etc/ntp.conf': owner => 'root', group => 'root', mode => '0644', require => Package['ntp'], }
service { 'ntpd': ensure => running, enable => true, subscribe => File['/etc/ntp.conf'], }
![Page 15: Rapid scaling in_the_cloud_with_puppet](https://reader036.vdocuments.us/reader036/viewer/2022062418/554bb766b4c9053a298b4a13/html5/thumbnails/15.jpg)
The GraphPuppet uses a graph to know the relationship between resources
![Page 16: Rapid scaling in_the_cloud_with_puppet](https://reader036.vdocuments.us/reader036/viewer/2022062418/554bb766b4c9053a298b4a13/html5/thumbnails/16.jpg)
Puppet is highly customizable
Using Ruby, you can add custom.....• puppet subcommands• types/providers• facts• report processors
![Page 17: Rapid scaling in_the_cloud_with_puppet](https://reader036.vdocuments.us/reader036/viewer/2022062418/554bb766b4c9053a298b4a13/html5/thumbnails/17.jpg)
Puppet is highly customizable
Custom Fact
Facter.add("role") do
setcode do Facter::Util::Resolution.exec("cat /etc/role") end
end
![Page 18: Rapid scaling in_the_cloud_with_puppet](https://reader036.vdocuments.us/reader036/viewer/2022062418/554bb766b4c9053a298b4a13/html5/thumbnails/18.jpg)
Puppet is highly customizable
Custom Report ProcessorPuppet::Reports.register_report(:autoami) do
def process .. do stuff .. end
end
![Page 19: Rapid scaling in_the_cloud_with_puppet](https://reader036.vdocuments.us/reader036/viewer/2022062418/554bb766b4c9053a298b4a13/html5/thumbnails/19.jpg)
Demo
![Page 20: Rapid scaling in_the_cloud_with_puppet](https://reader036.vdocuments.us/reader036/viewer/2022062418/554bb766b4c9053a298b4a13/html5/thumbnails/20.jpg)
Cloud Provisioner
Instant cloud management with puppet
![Page 21: Rapid scaling in_the_cloud_with_puppet](https://reader036.vdocuments.us/reader036/viewer/2022062418/554bb766b4c9053a298b4a13/html5/thumbnails/21.jpg)
Technologies
Puppet Faces
A new API for creating Puppet subcommands and actions. Introduced in 2.7.0
http://www.puppetlabs.com/faces/
Fog
Ruby gem designed to control a variety of cloud services through a unified API.
https://github.com/geemus/fog
![Page 22: Rapid scaling in_the_cloud_with_puppet](https://reader036.vdocuments.us/reader036/viewer/2022062418/554bb766b4c9053a298b4a13/html5/thumbnails/22.jpg)
Simplicity and Scriptability
1. Create a new instance from an AMI2. Install Puppet (from community packages or Puppet Enterprise
installer)3. Generate and sign SSL cert for new agent on master node
# puppet node bootstrap --image ami-d812efb1 --keyname my_keyname --type m1.small --login root --keyfile /path/to/my_keypair.pem --node-group webserver --server puppet.domain.com --enc-ssl --enc-auth-user console --enc-auth-passwd console_pass --enc-port 443
![Page 23: Rapid scaling in_the_cloud_with_puppet](https://reader036.vdocuments.us/reader036/viewer/2022062418/554bb766b4c9053a298b4a13/html5/thumbnails/23.jpg)
Simplicity and Scriptability
List nodes instances
# puppet node_aws listi-d22612b2: created_at: Wed Oct 12 16:50:02 UTC 2011 dns_name: ec2-184-73-33-225.compute-1.amazonaws.com id: i-d22612b2 state: runningi-f1b54b92: created_at: Wed Oct 26 13:46:44 UTC 2011 dns_name: ec2-174-129-228-163.compute-1.amazonaws.com id: i-f1b54b92 state: running
![Page 24: Rapid scaling in_the_cloud_with_puppet](https://reader036.vdocuments.us/reader036/viewer/2022062418/554bb766b4c9053a298b4a13/html5/thumbnails/24.jpg)
Simplicity and Scriptability
Destroy instances
# puppet node terminate ec2-75-101-181-145.compute-1.amazonaws.com
![Page 25: Rapid scaling in_the_cloud_with_puppet](https://reader036.vdocuments.us/reader036/viewer/2022062418/554bb766b4c9053a298b4a13/html5/thumbnails/25.jpg)
Simplicity and Scriptability
require 'puppet'require 'puppet/face'
opts = { :image => 'ami-d812efb1', :keyname => 'my_keypair', :type => 'm1.small', :login => 'root', :keyfile => '/path/to/my_keyfile.pem, :server => 'puppet.mydomain.com'}
Puppet::Face[:node_aws, '0.0.1'].bootstrap(opts)
![Page 26: Rapid scaling in_the_cloud_with_puppet](https://reader036.vdocuments.us/reader036/viewer/2022062418/554bb766b4c9053a298b4a13/html5/thumbnails/26.jpg)
AMI Management
This is a technique, not a tool!
![Page 27: Rapid scaling in_the_cloud_with_puppet](https://reader036.vdocuments.us/reader036/viewer/2022062418/554bb766b4c9053a298b4a13/html5/thumbnails/27.jpg)
AMI Management
• Not funo No, really. It sucks
![Page 28: Rapid scaling in_the_cloud_with_puppet](https://reader036.vdocuments.us/reader036/viewer/2022062418/554bb766b4c9053a298b4a13/html5/thumbnails/28.jpg)
AMI Management
• Not fun• Difficult to know when to update
o Usually requires a human to kick off a process
![Page 29: Rapid scaling in_the_cloud_with_puppet](https://reader036.vdocuments.us/reader036/viewer/2022062418/554bb766b4c9053a298b4a13/html5/thumbnails/29.jpg)
AMI Management
• Not fun• Difficult to know when to update• Needs to be registered with load balancer
o Whoever/whatever updates the AMI needs to register the new AMI with the load balancer and/or auto scaler
![Page 30: Rapid scaling in_the_cloud_with_puppet](https://reader036.vdocuments.us/reader036/viewer/2022062418/554bb766b4c9053a298b4a13/html5/thumbnails/30.jpg)
AMI Management
• Not fun• Difficult to know when to update• Needs to be registered with load balancer• Necessary?
o Many choose to just have puppet always configure a stock AMI
![Page 31: Rapid scaling in_the_cloud_with_puppet](https://reader036.vdocuments.us/reader036/viewer/2022062418/554bb766b4c9053a298b4a13/html5/thumbnails/31.jpg)
Initial Puppet Run
Can be slow if...• You haven't updated your AMIs in a while• You're running on a stock AMI to prevent image
management
![Page 32: Rapid scaling in_the_cloud_with_puppet](https://reader036.vdocuments.us/reader036/viewer/2022062418/554bb766b4c9053a298b4a13/html5/thumbnails/32.jpg)
Best of Both Worlds• Use Cloud Provisioner to spawn new instances of AMIs you
want to manage• Use Puppet custom report processors to detect if anything
changed• Use custom face to snapshot instances if anything changes.
Build new AMI off of snapshot• Register new AMI with load balancer and delete old one
![Page 33: Rapid scaling in_the_cloud_with_puppet](https://reader036.vdocuments.us/reader036/viewer/2022062418/554bb766b4c9053a298b4a13/html5/thumbnails/33.jpg)
EBS backed images (Elastic Block Store)
• Persistent• Allows for snapshots• Many public ones available
![Page 34: Rapid scaling in_the_cloud_with_puppet](https://reader036.vdocuments.us/reader036/viewer/2022062418/554bb766b4c9053a298b4a13/html5/thumbnails/34.jpg)
Autoami
Module containing custom face to manage images and custom report processor
http://github.com/ccaum/puppet-autoami
![Page 35: Rapid scaling in_the_cloud_with_puppet](https://reader036.vdocuments.us/reader036/viewer/2022062418/554bb766b4c9053a298b4a13/html5/thumbnails/35.jpg)
Autoami
Steps:1. Manually release production puppet code in VCS (git/svn)
![Page 36: Rapid scaling in_the_cloud_with_puppet](https://reader036.vdocuments.us/reader036/viewer/2022062418/554bb766b4c9053a298b4a13/html5/thumbnails/36.jpg)
Autoami
Steps:1. Manually release production puppet code in VCS (git/svn)– Launch an instance of every AMI you want to manage
![Page 37: Rapid scaling in_the_cloud_with_puppet](https://reader036.vdocuments.us/reader036/viewer/2022062418/554bb766b4c9053a298b4a13/html5/thumbnails/37.jpg)
Autoami
Steps:1. Manually release production puppet code in VCS (git/svn)– Launch an instance of every AMI you want to manage– Record the certificate name
![Page 38: Rapid scaling in_the_cloud_with_puppet](https://reader036.vdocuments.us/reader036/viewer/2022062418/554bb766b4c9053a298b4a13/html5/thumbnails/38.jpg)
Autoami
Steps:1. Manually release production puppet code in VCS (git/svn)– Launch an instance of every AMI you want to manage– Record the certificate name– Classify the instance with Console
![Page 39: Rapid scaling in_the_cloud_with_puppet](https://reader036.vdocuments.us/reader036/viewer/2022062418/554bb766b4c9053a298b4a13/html5/thumbnails/39.jpg)
Autoami
Steps:1. Manually release production puppet code in VCS (git/svn)– Launch an instance of every AMI you want to manage– Record the certificate name– Classify the instance with Console– Sign the certificate
![Page 40: Rapid scaling in_the_cloud_with_puppet](https://reader036.vdocuments.us/reader036/viewer/2022062418/554bb766b4c9053a298b4a13/html5/thumbnails/40.jpg)
Autoami
Steps:1. Manually release production puppet code in VCS (git/svn)– Launch an instance of every AMI you want to manage– Record the certificate name– Classify the instance with Console– Sign the certificate– Wait for instance to report
![Page 41: Rapid scaling in_the_cloud_with_puppet](https://reader036.vdocuments.us/reader036/viewer/2022062418/554bb766b4c9053a298b4a13/html5/thumbnails/41.jpg)
Autoami
Steps:1. Manually release production puppet code in VCS (git/svn)– Launch an instance of every AMI you want to manage– Record the certificate name– Classify the instance with Console– Sign the certificate– Wait for instance to report– If changes occurred (and were successful), generate new
AMI
![Page 42: Rapid scaling in_the_cloud_with_puppet](https://reader036.vdocuments.us/reader036/viewer/2022062418/554bb766b4c9053a298b4a13/html5/thumbnails/42.jpg)
Autoami
Steps:1. Manually release production puppet code in VCS (git/svn)– Launch an instance of every AMI you want to manage– Record the certificate name– Classify the instance with Console– Sign the certificate– Wait for instance to report– If changes occurred (and were successful), generate new
AMI– Record AMI for load balancers
![Page 43: Rapid scaling in_the_cloud_with_puppet](https://reader036.vdocuments.us/reader036/viewer/2022062418/554bb766b4c9053a298b4a13/html5/thumbnails/43.jpg)
Autoami
Custom Report Processor
Puppet::Reports.register_report(:autoami) do
def process .. do stuff .. endend
![Page 44: Rapid scaling in_the_cloud_with_puppet](https://reader036.vdocuments.us/reader036/viewer/2022062418/554bb766b4c9053a298b4a13/html5/thumbnails/44.jpg)
Autoami
![Page 45: Rapid scaling in_the_cloud_with_puppet](https://reader036.vdocuments.us/reader036/viewer/2022062418/554bb766b4c9053a298b4a13/html5/thumbnails/45.jpg)
Demo