puppet for dummies - proidea · puppet for dummies 4developers - 18 april 2012 ... what is puppet...
TRANSCRIPT
http://joind.in/6328
Puppet forDummies4developers - 18 april 2012Poznań - Poland
woensdag 18 april 12
Joshua Thijssen
Freelance consultant, developer and trainer @ NoxLogic / Techademy
Development in PHP, Python, Perl, C, Java and some sysadmin
Blog: http://adayinthelifeof.nl
Email: [email protected]: @jaytaph
oh hai!
2
woensdag 18 april 12
What is puppet and why should I care?
3
(answer: it’s cool and because I told you so)
woensdag 18 april 12
“People are finally figuring out puppet and how it gets you to the pub by 4pm.
Note that I’ve been at this pub since 2pm.”
- Jorge Castro
4
woensdag 18 april 12
Puppet is a (not necessarily the) solution for the following problem:
How do we setup, manage, synchronize, and upgrade our internal and external
infrastructure?
6
woensdag 18 april 12
LAMPGMVNMCSTRAH-stack
Linux
Apache
MySQL
PHP
Gearman
MongoDB
CouchDBSolr
Tika
Redis
ActiveMQHadoop
Varnish
Ngnix
Memcache
9
woensdag 18 april 12
➡ Solution 1: We don’t,
➡ Solution 2: We outsource,
10
How do we control our infrastructure?
woensdag 18 april 12
➡ Solution 1: We don’t,
➡ Solution 2: We outsource,
➡ Solution 3: We automate the process.
10
How do we control our infrastructure?
woensdag 18 april 12
➡ It’s not funny: you find it more often than not. Especially inside small development companies.
‣ Solution 1: we don’t11
woensdag 18 april 12
➡ It’s not funny: you find it more often than not. Especially inside small development companies.
➡ Internal sysadmin, but he’s too busy with development to do sysadmin.
‣ Solution 1: we don’t11
woensdag 18 april 12
➡ It’s not funny: you find it more often than not. Especially inside small development companies.
➡ Internal sysadmin, but he’s too busy with development to do sysadmin.
➡ We only act on escalation
‣ Solution 1: we don’t11
woensdag 18 april 12
➡ It’s not funny: you find it more often than not. Especially inside small development companies.
➡ Internal sysadmin, but he’s too busy with development to do sysadmin.
➡ We only act on escalation
➡ reactive, not proactive
‣ Solution 1: we don’t11
woensdag 18 april 12
➡ Expensive $LA’s.
➡ What about INTERNAL servers like your development systems and infrastructure?
‣ Solution 2: we outsource12
woensdag 18 april 12
➡ Expensive $LA’s.
➡ What about INTERNAL servers like your development systems and infrastructure?
➡ Fight between stability and agility.
‣ Solution 2: we outsource12
woensdag 18 april 12
➡ Expensive $LA’s.
➡ What about INTERNAL servers like your development systems and infrastructure?
➡ Fight between stability and agility.
➡ Does your hosting company decide on whether you can use PHP5.3???
‣ Solution 2: we outsource12
woensdag 18 april 12
➡ We are in charge.
➡ You can do what you like
‣ Solution 3: we do it ourselves and automate13
woensdag 18 april 12
➡ We are in charge.
➡ You can do what you like
➡ Use: cfEngine, chef, puppet.
‣ Solution 3: we do it ourselves and automate13
woensdag 18 april 12
➡ We are in charge.
➡ You can do what you like
➡ Use: cfEngine, chef, puppet.
➡ When done right, maintenance should not be difficult.
‣ Solution 3: we do it ourselves and automate13
woensdag 18 april 12
➡ Open source configuration management tool.
➡ Written in Ruby
➡ Open source: https://github.com/puppetlabs
➡ Commercial version available (puppet enterprise)
15
woensdag 18 april 12
➡ Don’t tell HOW to do stuff.
➡ Tell WHAT to do.
¹
¹ It’s not actually true, but good enough for now...16
woensdag 18 april 12
➡ Don’t tell HOW to do stuff.
➡ Tell WHAT to do.
¹
¹ It’s not actually true, but good enough for now...
“yum install httpd”“apt-get install apache2”
“install and run the apache webserver”
16
woensdag 18 april 12
Puppetmaster
Puppetclient
Check credentials
Send facts
Returns “catalog”
Report results
19
woensdag 18 april 12
➡ Catalogs are “compiled” manifests
➡ Manifests are puppet definitions
➡ <filename>.pp
➡ Puppet DSL
➡ De-cla-ra-tive language
➡ Version your manifests! (git/svn)
20
woensdag 18 april 12
package { “strace” : ensure => present,}
file { “/home/jaytaph/secret-ingredient.txt” : ensure => present, mode => 0600, user => ‘jaytaph’, group => ‘noxlogic’, content => “beer”, }
21
woensdag 18 april 12
package { “httpd” : ensure => present,}
service { “httpd”: running => true, enable => true, }
22
woensdag 18 april 12
package { “httpd” : ensure => present,}
service { “httpd”: running => true, enable => true, } require => Package[“httpd”],
22
woensdag 18 april 12
‣ Different distributions, different names
Centos / Redhatservice: httpdpackage: httpdconfig: /etc/httpd/conf/httpd.confvhosts: /etc/httpd/conf.d/*.conf
Debian / Ubuntuservice: apache2package: apache2config: /etc/apache2/httpd.confvhosts: /etc/apache2/sites-available
23
woensdag 18 april 12
$operatingsystem is a FACT
package { “webserver”: case $operatingsystem { centos, redhat { $apache = “httpd” } debian, ubuntu { $apache = “apache2” } default : { fail(‘I don’t know this OS/distro’) } }
name => $apache, ensure => installed,
}
24
woensdag 18 april 12
[root@puppetnode1 ~]# facter --puppetarchitecture => x86_64fqdn => puppetnode1.noxlogic.localinterfaces => eth1,eth2,loipaddress_eth1 => 192.168.1.114ipaddress_eth2 => 192.168.56.200kernel => Linuxkernelmajversion => 2.6operatingsystem => CentOSoperatingsystemrelease => 6.0processor0 => Intel(R) Core(TM)2 Duo CPU T7500 @ 2.20GHzpuppetversion => 2.6.9
‣ A simple list with info (also useable in your own tools)25
woensdag 18 april 12
node default { $def_packages = [ “mc”, “strace”, “sysstat” ] package { $def_packages : ensure => latest, }}
/etc/puppet/manifests/site.pp:
‣ “Main” manifest26
woensdag 18 april 12
Defining nodes - regular expressions
node /^web\d+\.example\.local$/ { package { “httpd” : ensure => latest, }}node /^db\d+\.example\.local$/ { package { “mysql-server” : ensure => installed, }}
27
woensdag 18 april 12
node basenode { user { “jaytaph” : ensure => present, gid => 1000, uid => 1000, home => “/home/jaytaph”, shell => “/bin/sh”, password => “supersecrethashedpassword”, managehome => true, }}node /^.+\.example\.local/ inherits basenode { ...}
‣ Node inheritance28
woensdag 18 april 12
class webserver { service { “apache”: ensure => running, require => Package[“apache”],
} package { “apache” : ensure => installed, }
}
‣ Group together into a class29
woensdag 18 april 12
class webserver { service { “apache”: ensure => running, require => Package[“apache”],
} package { “apache” : ensure => installed, }
}
file { “vhost_${webserver_name}” : path => “/etc/httpd/conf/10-vhost.conf”, content => template(“vhost.template.erb”), notify => Service[“httpd”], }
‣ Group together into a class29
woensdag 18 april 12
<virtualHost <%= ipaddress %>:80> ServerName <%= webserver_name %> ServerAlias <%= webserver_alias %> DocumentRoot <%= webserver_docroot %>
</virtualHost>
vhost.template.erb
30‣ ERB templates can contain custom variables and facts
woensdag 18 april 12
node “web01.example.local” inherits base { $webserver_name = “web01.example.local” $webserver_alias = “www.example.local” $webserver_docroot = “/var/www/web01” include webserver}node “web02.example.local” inherits base { $webserver_name = “web02.example.local” $webserver_alias = “crm.example.local” $webserver_docroot = “/var/www/web02” include webserver}
31
woensdag 18 april 12
➡ A puppet module is a collection of resources, classes, templates.
➡ Used for easy distribution and code-reuse.
➡ Self-contained, run out-of-the-box
32
woensdag 18 april 12
➡ puppetforge / github
➡ Create your own (and share!).
➡ Use the ones from puppet enterprise edition.
➡ Use the standard layout / best practices
33
woensdag 18 april 12
class ntp::install { package{"ntpd": ensure => latest }}class ntp::config { File{ require => Class["ntp::install"], notify => Class["ntp::service"], owner => "root", group => "root", mode => 644 } file{"/etc/ntp.conf": source => "puppet:///ntp/ntp.conf"; "/etc/ntp/step-tickers": source => "puppet:///ntp/step-tickers"; }}class ntp::service { service{"ntp": ensure => running, enable => true, require => Class["ntp::config"], }}class ntp { include ntp::install, ntp::config, ntp::service}
34
woensdag 18 april 12
➡ (Unit)test your modules
➡ Test them with: puppet apply --noop
➡ More advanced testing: cucumber / cucumber-puppet (BDD)
35
woensdag 18 april 12
http://docs.puppetlabs.com/references/stable/type.html
➡ Almost everything.
➡ standard 48 different resource types
➡ Ranging from “file” to “cron” to “ssh_key” to “user” to “selinux”.
➡ Can control your Cisco routers and windows machines too (sortakinda)
36
woensdag 18 april 12
http://media.techtarget.com/digitalguide/images/Misc/puppetDashboard.gif
37
woensdag 18 april 12
39
➡ Puppet went from v0.25 to v2.6.
➡ REST interface since 2.6. XMLRPC before that.
➡ One binary to rule them all (puppet).
➡ Puppet v2.7 switched from GPLv2 to apache2.0 license.
woensdag 18 april 12
➡ --test does not mean dry-run!(--noop does).
➡ It’s not object oriented. (puppet class != php class)
➡ It’s a declarative language.
40
woensdag 18 april 12
➡ Puppet agent “calls” the master every 30 minutes.
➡ But what about realtime command & control?
➡ “Puppet kick”... (meh)
➡ MCollective (Marionette Collective)
42
woensdag 18 april 12
➡ Which systems running a database and have 16GB or less?
➡ Which systems are using <50% of available memory?
➡ Restart all apache services in timezone GMT+5.
43
woensdag 18 april 12
ACTIVEMQClient
MCollectiveServer
NodeMiddlewareClient
MCollectiveServer
MCollectiveServer
‣ Middleware takes care of distribution,‣ queued, broadcast etc..
Collective
44
woensdag 18 april 12
http://docs.puppetlabs.com/mcollective/reference/basic/subcollectives.html45
woensdag 18 april 12
Filter out nodes based on facts
$ mc-facts operatingsystemReport for fact: operatingsystem
CentOS found 3 times Debian found 14 times Solaris found 4 times
$ mc-facts -W operatingsystem=Centos operatingsystemreleaseReport for fact: operatingsystemrelease
6.0 found 1 times 5.6 found 2 times
46
woensdag 18 april 12
➡ Display all running processes
➡ Run or deploy software
➡ Restart services
➡ Start puppet agent
➡ Upgrade your systems
47
woensdag 18 april 12
➡ Configuration management tool.
➡ Focusses on “what” instead of “how”.
➡ Scales from 1 to 100K+ systems.
➡ Uses descriptive manifests.
49
woensdag 18 april 12
➡ Useful for sysadmins and developers.
➡ Keeps your infrastructure in sync.
➡ Keeps your infrastructure versioned.
➡ MCollective controls your hosts based on facts, not names.
50
woensdag 18 april 12
There is no reason NOT to control your infrastructure.
Having only 3 servers is NOT a reason.
51
You will be able to join the rest of us in the pub early.
woensdag 18 april 12
http://farm1.static.flickr.com/73/163450213_18478d3aa6_d.jpg 52
woensdag 18 april 12
Please rate my talk on joind.in: http://joind.in/6328
Thank you
53
Find me on twitter: @jaytaph
Find me for development and training: www.noxlogic.nl
Find me on email: [email protected]
Find me for blogs: www.adayinthelifeof.nl
woensdag 18 april 12