using puppet to pull strings - darksim · using puppet to pull strings: ... what is puppet and why...

Using Puppet to Pull Strings:a Gentle Introduction to Puppet

Thomas Uphill

http://ramblings.narrabilis.com

http://goo.gl/FZvBD

example fileshttp://ramblings.narrabilis.com/Talks/puppet-picc.tar.bz2

Latest Versionon Google Docs

http://goo.gl/FZvBD

http://goo.gl/FZvBD

http://ramblings.narrabilis.com/Talks/puppet-picc.tar.bz2

http://ramblings.narrabilis.com/Talks/puppet-picc.tar.bz2

Overview

What is Puppet and why do I want it?

Puppet 101Introductory Material

Puppet 102Advanced Material

Questions sporadically throughout.

What is puppet?

Think of it as a language.

Describe state, not steps.

Paint a picture of your ideal and most clean system.

Puppet does the rest!

puppetlabs

● Luke Kanies in 2005● http://puppetlabs.com/● puppet Enterprise

○ commercial support○ windows support○ puppet forge

Why do I want it?

you can do cool things with itHuman-parsable.ensure that all your hosts are running ssh

service {"ssh": ensure => running;}

Puppet 101Resource Abstraction LayerSyntaxtrifectacore types(file, package, service)puppet apply / puppet resourcefacter/variablesconditionalsbooleans/arithmeticin

templatestypes II(augeas, exec, cron)classesfunctionsfilebuckets

versions

0.20 0.25 2.6.0 2.7.0

2.7.21

0.20 0.25 2.6.0 2.7.0

2.7.21

3.03.0

3.1.13.1.1

learning puppet vm

http://docs.puppetlabs.com/learning/

VMwareOVFVirtualBoxKVM

Get the Learning Puppet VM

learning puppet vm

KVM

[root@hv] vmware-mount -f learn_puppet.vmdk /mnt[root@hv] qemu-img /mnt/flat -Oqcow2 learn_puppet.qcow2

Resource Abstraction Layer

types providers

ubuntu

SLES

macosx

Windows

user

file

package

service

exec

mount

group

host

RHELCENTOS

Springdale

manifest

pluginmodule

Language (preview)

classattributetypenode

factvariable

class class

Syntax

type { "title":

attribute => value,

attribute => value

}

Syntax

type { "title":

attribute => value,

attribute => value

}

syntax.pp

file {"testfile": mode => 0644, owner => root path => "/etc/test", ensure => present}

[root@learn ~] # puppet parser validate syntax.pp err: Could not parse for environment production: Syntax error at 'path'; expected '}' at /etc/puppetlabs/puppet/manifests/syntax.pp:4err: Try 'puppet help parser validate' for usage

trifecta

file

package

service

core types cheat sheet

docs.puppetlabs.com/puppet_core_types_cheatsheet.pdf

fileAttributes

ensure

pathtitle: source

content

target

recurse

purge

owner

group

mode

file {"issue":path => "/etc/motd",content => "Hello World!"}

/etc/motd

Hello World!

distributing file contents

distributing file contents 2

Attributes

target

recursefile {"issue":path => "/etc/motd",content => file("/etc/hello")}

/etc/motd

Hello File!

/etc/hello

Hello File!


file {"issue":path => "/etc/motd",content => template("/etc/hello.erb")}

/etc/motd

Hello learn!

/etc/hello.erb

Hello <%= hostname %>!

puppet apply (serverless operation)

apply changes using a locally defined manifest

[root@learn ~] # puppet apply mymanifest.pp


Attributes

file {"issue":path => "/etc/motd",source =>"puppet:///files/hello"}

/etc/motd

Hello puppet!

/etc/puppet/manifests/hello

Hello puppet!

packageAttributes

ensure

nametitle: source

henson.local

demo VMs

learn jim

kermitpiggy

serviceAttributes

ensure

nametitle: enable

hasrestarthasstatusrestartstatus

ordering/dependencies

file

package

service

host.conf httpd

httpd

DAG

host.conf httpd

httpd

directed acyclic graph

DirectedAcyclicGraph

Referencing objects - case

Defining a new object uses lower case:

Referencing an existing object uses upper case:

Be careful not to define objects twice!

service["sshd"] { ensure => running }

require => Service["sshd"]

/etc/puppet/lopsa/ordering.pp

package {'httpd': ensure => present}service {"httpd": enable => true, ensure => running, hasrestart => true, hasstatus => true, require => [Package['httpd'],File['host.conf']]}

file {"host.conf": path => "/etc/httpd/conf.d/$name", mode => 0644, owner => 'apache', group => 'apache', content => "<VirtualHost *:80> ServerName kermit.henson.local DocumentRoot /var/www/html/kermit </Virtualhost>", require => Package['httpd']}

ordering/dependencies

puppet resource

[root@learn: ~] $ puppet resource --typesaugeascomputercronexecfilefilebucketgroup...

[root@learn: ~] $ puppet resource mount swapmount { 'swap': ensure => 'unmounted', device => '/dev/vg00/swapvol', dump => '0', fstype => 'swap', options => 'defaults', pass => '0', target => '/etc/fstab',}

memory

Facter/Facts

processors

ipaddress

architecture

dns

VM?

selinux

timezonesystem

$processor0$ipaddress$architecture$fqdn$is_virtual$memorysize$selinux$timezone

variables

use as parametersuse in conditionalsfactsuser defined

$processor0

$myvar = "My Variable"

$::processor0

conditionals

● if/else/unless● case● selector

● booleans

booleans

$x = 1$y = 2($x == $y)($x > $y)($x < $y)($x != $y)($x < $y) and !($x < $y)

false

false

true

true

false

arithmetic

$x=1$y=2$x+$x == $y$x-$x$y/2$y >> 1$y << 1

true

1

1

4

0

if/else/unlessif $myvar == "My Variable" {file {'/my/variable':

content => "All is good", ensure => present

}} elsif $myvar {

file {'/my/variable':content => $myvar,ensure => present

}} else {

file {'/my/variable':content => "bad var",ensure => present

}}

$variablefile = "/my/variable"if $myvar == "My Variable" {file {$variablefile:

content => "All is good", ensure => present

}} elsif $myvar {

file {$variablefile:content => $myvar,ensure => present

}} else {

file {$variablefile:content => "bad var",ensure => present

}}

if/else/unlessunless

not yet implementedif !(condition)

if !($myvar) {file {"/my/file":

content => "no myvar set",ensure => present

}}

casecase $::hostname {

/^ldap/: { include ldapserver }/^www/: { include webserver }/^dns/: { include bind }/^mx[12]/: { include mxprimary }/^mx[3-9]/: { include mxsecondary }default: { include base }

}

Regular Expressions

match =~if $::hostname =~ /^ldap/ {

include ldapserver}

not match !~if $::hostname !~ /test/ {

include production}

casecase $::hostname {

/^ldap/: { include ldapserver }/^www/: { include webserver }/^dns/: { include bind }/^mx[12]/: { include mxprimary }/^mx[3-9]/: { include mxsecondary }default: { include base }

}

Regular Expressions● capture

/^ldap(\d+)/ { include "ldap$1" }

selectorternary operator (C)x == y ? "they are equal" : "they are different"

attribute => $fact ? {'value' => 'result','other_value' => 'other_result',default => 'default_result'

}

"in"if $::hostname in ['www','web'] {

service {'httpd':ensure => true

}}

if $::kernelversion in ['2.6.35-22','2.6.38.6-26.rc1.fc15'] {

service {'sshd':ensure => false

}}

backus naur (bnf)<exp> ::= <exp> <arithop> <exp> | <exp> <boolop> <exp> | <exp> <compop> <exp> | <exp> <matchop> <regex> | ! <exp> | - <exp> | "(" <exp> ")" | <rightvalue>

<arithop> ::= "+" | "-" | "/" | "*" | "<<" | ">>"<boolop> ::= "and" | "or"<compop> ::= "==" | "!=" | ">" | ">=" | "<=" | "<"<matchop> ::= "=~" | "!~"

<rightvalue> ::= <variable> | <function-call> | <literals><literals> ::= <float> | <integer> | <hex-integer> | \

<octal-integer> | <quoted-string><regex> ::= '/regex/'

http://docs.puppetlabs.com/guides/language_guide.html

templates

ERB syntax<% Ruby code %>

<%= Ruby expression %><%# comment %>

-%> no newline<%% replace with <%%%> replace with %>

templates (conditionals)

puppet ERB<%= @fact %> replace with fact/global variable

conditionalclient.conf.erb<% if @ipaddress_eth0 != "NONE" %>ServerName <%= @printserver %><% end %>

templates (iteration)

iteration$nameservers = [ 'ns1.example.com',

'ns2.example.com','ns3.example.com']

$searchdomains = [ 'inside.example.com','outside.example.com','under.example.com']

file {"resolvconf":path => "/etc/resolv.conf",mode => 0644, owner => root, group => root,content => template('resolvconf.erb')

}




file {"resolvconf":path => "/etc/resolv.conf",mode => 0644, owner => root, group => root,content => template('resolvconf.erb')

}

templates (concatenation)




file {"resolvconf":path => "/etc/resolv.conf",mode => 0644, owner => root, group => root,content => template('header.erb','resolvconf.erb')

}

types II

cronexecaugeas

cronAttributes

command

hourname: minute

monthmonthday

user

weekday

execAttributes

command

createstitle: cwd

environmentgrouplogoutput

onlyif

path

refreshrefreshonlyreturnsunless

tool to transform files into objectsconfig changes made to the objectsaugtool and augparse command line toolsWhy?

● manipulate parts of a file● don't clobber another modules work● start from a known good config

augeasAttributes

changes

contexttitle: force

nameonlyif

augeas

sshd_config

augeas{ "sshd password authentication": context => "/files/etc/ssh/sshd_config", changes => [ "set PasswordAuthentication no" ], notify => Service["sshd"]}

class

classes

attributetype

attributetype variable

variable

classinclude otherclass

class syntax

class name {type {title: attributes => values },type {title: attributes => values }

}

class hierarchy

parent nephew

top/global

child

functions

includefilejoinsplitdefinedrequire

templaterealizenotice

http://docs.puppetlabs.com/references/stable/function.html



inline_template

● apply template to variables ● pass to a parameter# iterate over the interfaces# print out their addresses into ifcfg-files$ifs = split($interfaces,',')

define inline_template_test { file {"/tmp/ifcfg-$name": content => inline_template("IPADDR=<%= @ipaddress_${name} %>\n") }}

inline_template_test { $ifs: }/tmp/ifcfg-eth0IPADDR=192.168.122.4

filebuckets

Defaults to local-only modeServer mode:filebucket { puppet: server => "puppet.example.edu" }

Command-line utility:# puppet filebucket --local backup /etc/puppet/puppet.conf /etc/puppet/puppet.conf: be50b3e9acc2c2de8df194b1466fd2c1

# puppet filebucket --local restore /etc/puppet/puppet.conf \be50b3e9acc2c2de8df194b1466fd2c1

vim

.vimrc

" puppet style guide modefiletype plugin indent on

Puppet 102advanced languageserver configuration

plusignmentscopeparameterized classesrun stagesresource chainingpuppetmaster (server operation)webrickfileserverclient configurationdebuggingpuppetcasite.pptype defaultsreferencing/upper case

puppet 102 modulespluginscustom factscustom typespluginsyncpassengerpushingreportingencenvironmentsstored configurationsvirtualizing resourcesexported resourcessystem integration/firewallnagios

plusignment

class openssh {

file { "/etc/ssh/sshd_config":

content => "..."; }

service {

"sshd":

ensure => running,

subscribe => File["/etc/ssh/sshd_config"]; }

class keydistribute {

file {

"/etc/ssh/host_rsa_key":

content => "...";

"/etc/ssh/host_dsa_key":

content => "..."; }

Service["sshd"] {

subscribe +> [ File["/etc/ssh/host_rsa_key"],

File["/etc/ssh/host_dsa_key"] ] }

}

}

scope

class base

hostname

class dns

$::hostname

class dns

class base

hostname

scopeinclude parent::child$var = "top level"class parent { $var = "from parent" $hostname = "hostname from parent"}class parent::child inherits parent { $var = "from parent::child" notice ( "parent::var => ",$parent::var ) notice ( "var => ",$var ) notice ( "hostname => ",$hostname ) notice ( "::hostname => ",$::hostname )}

http://docs.puppetlabs.com/guides/scope_and_puppet.html



parameterized classes# unparameterizedinclude resolverclass resolver { $dnsservers = ['8.8.8.8','8.8.4.4'] file {"resolver-conf": path => "/tmp/resolv.conf", content => template("/root/resolver-conf.erb")

}}

parameterized classes# parameterizedclass {"resolver": dnsservers => ['192.168.1.1','192.168.4.1']}class resolver ($dnsservers =['8.8.8.8','8.8.4.4']) { file {"resolver-conf": path => "/tmp/resolv.conf", content => template("/root/resolver-conf.erb")

}}

parameterized classes# parameterized2

class resolver ($mydnsservers = '') {

$maindnsservers = '8.8.8.8,8.8.4.4'

if $mydnsservers {

$dnsservers = split(sprintf("%s,%s",$mydnsservers,$maindnsservers),",")

} else {

$dnsservers = split($maindnsservers,",")

}

file {'resolver-conf':

path => '/tmp/resolv.conf',

content => template('/etc/picc/resolver-conf.erb')

}

}

class {'resolver':

mydnsservers => "192.168.1.1,192.168.4.1"

}

custom definesMove beyond the singletons of typical puppet manifests.

Defines are used to model repeatable chunks of puppet code.

Common examples include versioning repositories and apache virtual hosts.

Not much to explain, let's just take a look at an example...

class git { package{ "git": } file {'/var/lib/git': ensure => 'directory', mode => 0755, owner => 0, group => 0 }}

define gitrepo($owner, $group, $ispublic = false) { include git $mode = $ispublic ? { true => 2774, false => 2770 }

file { "/var/lib/git/${title}": ensure => directory, owner => "${owner}", group => "${group}", mode => $mode, require => [Package['git'],File['/var/lib/git']] }

exec { "${title}_initialize": command => '/usr/bin/git --bare init .', cwd => "/var/lib/git/${title}", unless => "/bin/test -f /var/lib/git/${title}/HEAD", require => [Packages['git'],File["/var/lib/git/${title}"]] }}

"chain" operatorsClass["setup_repos"] -> Class["install_packages"] -> Class["ldap::client"]

Package["openssh-server"] -> File["/etc/ssh/sshd_config"] -> Service["sshd"] <- File["/etc/ssh/ssh_host_rsa_key.pub"]

Run stages & resource chaining

setup_repos install_packages ldap::client

before => Class['install_packages', 'ldap::client']

before => Class['ldap::client']require => Class['setup_repos']

require => Class['setup_repos', 'install_packages']

Run stages - the parameterized waystage {

"pre":before => Stage["main"];

"post":require => Stage["main"];

}

node "server.example.com" {class {

"setup_repos":stage => "pre";

"install_packages":stage => "main";

"ldap::client":stage => "post";

}}

puppetmaster

Allows all puppet work to be done in a single location and then distributed via a pull-based puppet client.

puppetmaster

Webserver

DHCPserver

DNSserver

Databaseserver

TCPport 8140

SSLx509

puppetmaster

● Pairs well with a versioning system such as git or svn to create living manifests.

● Options controlled through [master] section in puppet.conf

● "puppet-server" package. Creates user/group puppet.

● Simplest case is webrick, works out of the box.

webrick

Works without any additional configuration

service puppetmaster start

iptables -A INPUT -m state --state NEW -m tcp -p tcp --dport 8140 -j ACCEPT

[root@jim ~] # lsof -i |grep 8140puppetmas 3211 puppet 6u IPv4 17188 0t0 TCP *:8140 (LISTEN)

puppetmaster as a fileserver

If you use "source => puppet:///" in any of your file definitions, this is how those files get distributed.

/etc/puppet/fileserver.conf

[files]path /var/lib/puppet/filesallow *.example.comallow *.friendly.example.orgdeny *.malice.example.netdeny impostor.friendly.example.org

puppet.conf

auth.conf

fileserver.conf

manifests/site.pp

server configuration

server configuration/etc/puppet/puppet.conf

[main]logdir = /var/log/puppetrundir = /var/run/puppetssldir = $vardir/ssl

[agent]classfile = $vardir/classes.txtlocalconfig = $vardir/localconfig

server configuration

/etc/puppet/fileserver.conf

[files]path /var/lib/puppet/filesallow *.example.com

[modules]allow *.example.com

[facts]path /var/lib/puppet/factsallow *.example.com

[plugins]path /var/lib/puppet/pluginsallow *.example.com

client configuration

/etc/sysconfig/puppet

PUPPET_SERVER=puppet.example.com#PUPPET_PORT#PUPPET_LOG#PUPPET_EXTRA_OPTS=--waitforcert=500

/etc/puppet/puppet.conf

[agent]server = puppet.example.com

it's go time.

on the master:service puppetmaster start

on the client:service puppet start

...hope for the best.

an aside...debugging clients

[root@client ~] # service puppet stop

Run and print the log output to the shell:[root@client ~] # puppet agent -o --no-daemonize -vor[root@client ~] # puppet agent -t

View even more information on the shell:[root@client ~] # puppet agent -o --no-daemonize --debug

View a crazy, incomprehensible amount of info on the shell:[root@client ~] # puppet agent -o --no-daemonize --trace

info: Creating a new SSL key for kermit.henson.localwarning: peer certificate won't be verified in this SSL sessioninfo: Caching certificate for cawarning: peer certificate won't be verified in this SSL sessionwarning: peer certificate won't be verified in this SSL sessioninfo: Creating a new SSL certificate request for kermit.henson.localinfo: Certificate Request fingerprint (md5): F4:E2:10:61:1C:F7:F8:E4:03:98:14:B0:F7:47:C8:9Ewarning: peer certificate won't be verified in this SSL sessionwarning: peer certificate won't be verified in this SSL sessionwarning: peer certificate won't be verified in this SSL sessionExiting; no certificate found and waitforcert is disabled

puppetca

puppetca

Distributes and manages x509 certificates.

Command line utility: puppet cert (puppetca for oldtimers)

Supports auto-signing

/etc/puppet/autosign.conf

*.example.com*.subdomain.example.net

it's go time

server[root@jim puppet]# puppet cert list

kermit.henson.local (F4:E2:10:61:1C:F7:F8:E4:03:98:14:B0:F7:47:C8:9E)

[root@jim puppet]# puppet cert sign kermit.henson.local

notice: Signed certificate request for kermit.henson.local

notice: Removing file Puppet::SSL::CertificateRequest kermit.henson.local at '/var/lib/puppet/ssl/ca/requests/kermit.henson.local.pem'

it's go time[root@kermit ~]# puppet agent -v --no-daemonize -o

warning: peer certificate won't be verified in this SSL session

info: Caching certificate for kermit.henson.local

info: Caching certificate_revocation_list for ca

err: Could not retrieve catalog from remote server: Error 400 on SERVER: Could not find default node or by name with 'kermit.henson.local, kermit.henson, kermit' on node kermit.henson.local

notice: Using cached catalog

err: Could not retrieve catalog; skipping run

[root@enchantress ~]#

site.pp

master manifestneeds to define each node or a default node

simplest* site.ppnode default {}

site.pp

import 'nodes.pp'import 'functions.pp'import 'variables.pp'import 'defaults.pp'

type defaults (defaults.pp)capitalize the type: ...but do not give a title

These "attribute => value" pairs will now be the defaults for all objects of type "file". You are free to overwrite them later.

File {user => root,group => root,mode => 0644

}

tip: tag template

tagprinter.erb

tags----<% tags.each do |tag| -%><%= tag %><% end -%>

classes-------<% classes.each do |cls| -%><%= cls %><% end -%>

variables-------<% scope.to_hash.keys.each do |var| -%><%= var -%> => <%= scope.lookupvar(var) %><% end -%>

modules

{modulepath}└── {modulename} └── files └── manifests └── templates

● a way to organize your manifests● promotes code reuse● provides namespaces● can push code to clients

modules

{modulepath}└── {modulename} └── files └── manifests └── init.pp └── templates

modules

init.pp

class modulename {file {"example.conf":

path => "/tmp/example.conf",source => "puppet:///modules/modulename/example.

conf"}

}

{modulepath}/{modulename}/files/example.conf

{modulepath}/{modulename}/manifests/init.pp

modules

subclass.pp

class modulename::subclass {file {"subclass.conf":

path => "/tmp/subclass.conf",source => "puppet:///modules/modulename/subclass.conf"

}}

{modulepath}/{modulename}/files/subclass.conf

{modulepath}/{modulename}/manifests/subclass.pp

modules

subclass.pp

class modulename::subclass {file {"subclass.conf":

path => "/tmp/subclass.conf",content => template("modulename/subclass.erb")

}}

{modulepath}/{modulename}/templates/subclass.erb

module cheat sheethttp://docs.puppetlabs.com/module_cheat_sheet.pdf

style guide

http://docs.puppetlabs.com/guides/style_guide.html

● two-space soft tabs (no literal tab characters)● no trailing white space● 80 character line width● fat comma arrows (=>) within blocks of attributes● # for comments● 'my variable' unless "my ${variable}"● ensure first● defaults in site.pp● separate files for all classes in a module

forge

http://forge.puppetlabs.com/

> 2.7.14puppet module< 2.7.14puppet-module

plugins{modulepath}└── {modulename} └── files └── lib └── facter └── puppet └── parser └── functions └── provider └── exec └── package └── anytype └── type └── util

custom facts

/etc/puppet/modules/videocard/lib/facter/videocard.rb

# Josko Plazonic - lifted from Josko March 14, 2011 by Thomas Uphillrequire 'facter'

Facter.add("videocard") doconfine :kernel => :linuxENV["PATH"]="/bin:/sbin:/usr/bin:/usr/sbin"setcode do

controllers = []lspciexists = system "/bin/bash -c 'which lspci

>&/dev//null'"if $?.exitstatus == 0

output = %x{lspci}output.each {|s|

controllers.push($1) if s =~ /VGA compatible controller: (.*)/

}endcontrollers

endend

custom types

/etc/puppet/modules/newtype/lib/puppet/type/newtype.rb# this is beyond the context of this talk...# best to look on http://forge.puppetlabs.com/

Puppet::Type.newtype(:newtype)donewproperty(:myprop, :array_matching => :all)doend

end

pluginsync

plugins synced from master to the clients

/etc/puppet/module

something

libfacter puppet

/var/lib/puppet

libfacter puppet

pluginsync

videocard example, videocard.rbpuppet.conf -> [agent] pluginsync = true

info: Retrieving pluginnotice: /File[/var/lib/puppet/lib/videocard.rb]/ensure: defined info: Loading downloaded plugin var/lib/puppet/lib/videocard.rb

Modules + Facts + TemplatesControl access to system users via cgroups

Filestructure:

/etc/puppet/modules/cgroups└── templates └── cgconfig.conf└── files └── cgrules.conf└── manifests └── init.pp└── lib └── facter └── cgroups.rb

Modules + Facts + Templates

manifests/init.pp class cgroups {

package {"libcgroup":

ensure => present;}

service {"cgconfig":

ensure => running,enable => true,require => Package["libcgroup"];

"cgred":ensure => running,enable => true,require => [ Package["libcgroup"], Service["cgconfig"]

];}

file {"/etc/cgconfig.conf":

content => template("cgroups/cgconfig.conf"),notify => [ Service["cgconfig"], Service["cgred"] ];

"/etc/cgrules.conf":source => "puppet:///modules/cgroups/cgrules.conf",notify => [ Service["cgconfig"], Service["cgred"] ];

}}


files/cgrules.conf@staff cpu staff/

@student cpu,memory users/@faculty cpu,memory users/@grad cpu,memory users/@guest cpu,memory users/@alumni cpu,memory users/

* cpu system/


templates/cgconfig.confgroup staff {

cpu {cpu.shares = 20;

}}

group users {cpu {

cpu.shares = 60;}memory {

memory.limit_in_bytes = <%= mem80pct %>;memory.memsw.limit_in_bytes = <%= memswap80pct %>;

}}

group system {cpu {

cpu.shares = 20;}


lib/facter/cgroups.rbFacter.add(:mem80pct) do setcode do Facter::Util::Resolution.exec('\

free -bto | \grep Mem | \awk \'{OFMT = "%.0f"; print $2*.8}\'\

') endend

Facter.add(:memswap80pct) do setcode do Facter::Util::Resolution.exec('\

free -bto | \grep Total | \awk \'{OFMT = "%.0f"; print $2*.8}\'\

') endend

Production - Use passengerApache a better webserver than webrick, mongrel support is deprecated.

Much more stable.

config.ru - must be owned by puppet

Passenger - Apache configurationLoadModule passenger_module modules/mod_passenger.so<IfModule mod_passenger.c> PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-3.0.2 PassengerRuby /usr/bin/ruby</IfModule>RackAutoDetect OffRailsAutoDetect OffListen 8140<VirtualHost *:8140> RequestHeader set X-SSL-Subject %{SSL_CLIENT_S_DN}e RequestHeader set X-Client-DN %{SSL_CLIENT_S_DN}e RequestHeader set X-Client-Verify %{SSL_CLIENT_VERIFY}e DocumentRoot /etc/puppet/rack/public/ RackBaseURI /</VirtualHost>

Performance PassengerPuppet docs recommend the use of the following apache config directives:

# Refresh old puppetmaster processes after 5 minutes...PassengerPoolIdleTime 300# Set to num_clients * 1.15, 15% more than required...PassengerMaxPoolSize 50# Puppetmaster takes a while, just use one global queue...PassengerUseGlobalQueue on# We don't need most of the passenger stuff, disable it...PassengerHighPerformance on

Keep an eye on memory usage!

Puppet pushing

Puppet is strictly pull-based by design.

Cannot truly push changes.

"puppetrunner" allows the master to remotely trigger the agent to run.

/etc/puppet/puppet.conf[agent] listen = true

Reporting

Agent creates a state.yaml file detailing the results of each run. Have the agent send the report to the master:

[agent] report = true

Report plugins parse state.yaml from the agents and sends the data off to some other utility.

[master]reports=<report_format>

Reporting Engines

Common report formats include:● http● log● rrdgraph● tagmail● puppetdashboard (PE Console)● foreman

Providers are located in:${RUBY_LIBDIR}/puppet/reports/*.rb

Puppet dashboard

Foreman

foreman

Can extend an existing puppet database, removing need for puppet/foreman syncing scripts.

Extensible, can control DHCP, DNS, etc

http://theforeman.org/

ENC

external node classifier● script● ldap

● classes● top scope variables● environment

ENC

external node classifier

● execute a script to determine classes to apply to a node.

● script receives fqdn as argument 1● must return yaml:

○ classes○ top scope variables○ environment

ENC

/usr/bin/puppet_node_classifier#!/usr/bin/ruby

require 'yaml'

# create an empty hash@enc = Hash.new@enc["classes"] = Hash.new@enc["classes"]["base"] = Hash.new@enc["parameters"] = Hash.new@enc["environment"] = 'production'

if ARGV[0] =~ /^www/ @enc["classes"]["webserver"] = Hash.newend

if ARGV[0] =~ /^kermi/ @enc["classes"]["kermit"] = Hash.newendputs @enc.to_yamlexit(0)

ENC#!/bin/bashset -eset -u

ENC_BASE_URL="https://localhost:443/nodes"

curl -k -H "Accept: text/yaml" "${ENC_BASE_URL}/${1}"

ENC LDAP[master]

node_terminus = ldapldapserver = ldap.henson.localldapbase = ou=hosts,dc=henson,dc=local

schema:attributeTypes:

puppetclassparentnodeenvironmentpuppetvarpuppetClient

# kermit.henson.local servers, hosts,kermit.henson.localdn: cn=kermit.henson.local,ou=servers,ou=hosts,dc=henson,dc=localcn: kermit.henson.localipHostNumber: 192.168.122.4puppetclass: serverpuppetclass: logpolicypuppetclass: netwatch::serverobjectClass: topobjectClass: iphostobjectClass: puppetclient

environments

[myenviro]modulepath = $confdir/environments/myenviro/modulesmanifest = $confdir/environments/myenviro/site.ppmanifestdir = $confdir/environments/myenviro/manifeststemplatedir = $confdir/environments/myenviro/templates

create separate workspaces for clients, change the manifests/modules based on the environment setting

environments[master]

modulepath = $confdir/environments/$environment/modules:$confdir/modules

manifest = $confdir/environments/$environment/manifests/site.pp

[production]

modulespath = $confdir/production/modules

manifest = $confdir/production/manifests/site.pp

[dev]

modulespath = $confdir/test/modules

manifest = $confdir/test/manifests/site.pp

http://docs.puppetlabs.com/guides/environment.html



environments + git● make new branches

git branch bobsbranch

● ssh-keys from git to puppetmaster

● update environments directory on commit to gitpuppet.git/hooks/post-receive

● tell agent to use the new environmentpuppet agent -t --environment bobsbranch

or/etc/puppet/puppet.conf[agent]environment=bobsbranch




Stored Configurations

Enables use of more advanced puppet features.

Used as a cache, speeds up catalog creation.

puppet.conf[master]

storeconfigs=truedbadapter=mysqldbname=puppetmasterdbuser=puppetmasterdbpassword=superdupersecretdbserver=localhost

fact_names

inventory_facts

param_values

param_namesinventory_nodes

resources

hostsfact_values

source_files

puppet_tagsresource_tags

Stored Configurations DB Schema

Stored Configurations DB Schema

Sample entry from "resources" table:id: 11518title: node001.example.comrestype: Sshkeyhost_id: 28source_file_id: 3exported: 1line: 307updated_at: 2012-04-23 15:25:15created_at: 2012-04-23 15:25:15

Virtualizing Resources

Create a resource, but don't actually send it to the host, keep it local to this manifest.

@ operator

@file { ... }@service { ... }@my_custom_type { ... }

The inverse - resource realization

Ok I changed my mind, send this resource to the host.

<| |> operator, or more recently, the realize() function.

File <| |> realize(File["title"])Service <| |> realize(Service["title"])My_custom_type <| |> realize(My_custom_type["title"])

Realize use case

base.pp:class base {

file {"/tmp/a":

mode => 644;}@user {

"lopsa":ensure => present;

}}

class manifest1 inherits base {

realize(User["lopsa"])...

}


realize(User["lopsa"])...

}


...}

exported resources cheat sheethttp://ramblings.narrabilis.com/Talks/exported_resources_cheatsheet.pdf

http://goo.gl/8KAsM

http://ramblings.narrabilis.com/Talks/exported_resources_cheatsheet.pdf

http://ramblings.narrabilis.com/Talks/exported_resources_cheatsheet.pdf

http://goo.gl/8KAsM

http://goo.gl/8KAsM

Exporting resources

Just sets the "exported" flag in mysql. These resources are now visible to all hosts.

@@ operator

@@file { ... }@@service { ... }@@my_custom_type { ... }

See where this is heading?

In base.pp:

@@sshkey { "${::fqdn}": ensure => present, type => "ssh-rsa", key => "${::sshrsakey}", require => Service["sshd"];}

Sshkey <<| |>>

Done! Now, all of your machines know every other machine's SSH key.

The SSH warning, "are you sure you want to continue? (yes/no):" is a thing of the past!

Of course, this only works for machines within the puppet infrastructure.

Exported resources "gotcha"Be careful when picking the title of any resource that's to be exported.

@@file["/etc/motd"] { mode => 644 }

You'd think this would be fine, but you'll get a "cannot redefine" error if more than one host runs it. Instead, do something like:

@@file {"/etc/motd_${::fqdn}":

path => "/etc/motd",mode => 644;

}

Grouping exported resources

Remember the builtin "tag" parameter?

@@sshkey{ "${::fqdn}": key => ${::sshrsakey}, tag => "group1";}

Sshkey <<| tag == "group1" |>>

Multiple group membership

Tags can be set as an array, but collected based on one value.@@sshkey{ "${::fqdn}": key => "${::sshrsakey}", tag => [ "internal", "external" ];}if $::hostname =~ /internal.example.com/ { Sshkey <<| tag == "internal" |>>}else {Sshkey <<| tag == "external" |>>}

Modify-on-collect

Interact with resources that only exist in the local manifest.

service { "sshd": ensure => running;}

Sshkey <<| |>> { notify => Service["sshd"]}

Systems Integration

Using exported resources, one can build a network adaptive to IP address & hostname changes.

Most common cases are system installation and some form of monitoring.

Puppet has builtin types for integrating with nagios. We'll add in bits to handle monitoring system firewall rules.

simple server/client autoconfig

serverA192.168.1.1

80 192.168.1.2

client


File (tell client where to find the server)client stores config in /etc/service.confclass server {

@@file {"server_$::fqdn":

path => '/etc/service.conf',

content => inline_template('server=<%= ipaddress -%>'),

tag => 'server-config'

}

}

class client {

File <<| tag == 'server-config' |>>

}


Firewall (tell server to allow client)class client {

@@firewall {"80 allow client $::fqdn port 80":

proto => 'tcp',

source => "$::ipaddress",

dport => "80",

action => 'accept',

tag => 'allow-client'

}

}

class server {

Firewall <<| tag == 'allow-client' |>>

}

https://github.com/puppetlabs/puppetlabs-firewall



Integration with anaconda/kickstart%post

chvt 6

/bin/false

count=0

exit=1

while [ $exit -ne 0 ]; do

puppet agent -t \

--server puppetmaster.example.com --waitforcert 60 >/dev/tty6 2>&1

exit=$?

count=$((count + 1))

if [ $count >= 5 ]; then

break

fi

done

if [ $exit == 0 ]; then

chkconfig puppet on

else

read "Puppet didn't run successfully, investigate"

fi

%end

%packages --nobase

@core

puppet

%end

hiera

● solves the nested if problem● set variables based on facts● hierarchical

Final Q & A

Thanks for attending!

Any questions?

Still awake...?

NaginatorStarted as a module/plugin. Became so useful, stable, and well-tested, it was cooked straight into base installs.

We'll use this following example as a "final review" of sorts. The example will get more complex the further into the manifest we go.

In the following examples, assume we've already created (or downloaded from the puppet forge!) the modules "apache" and "iptables".

nagios_host

Attributes

address

aliastitle: check_command

IP Address for Connections

check_period

contactsensurehost_nametarget

Display Addresscheck_ping,check_ssh,etce.g. "24x7","workhours"Who shall we notify?presentabsentFQDN for connections.Filename

nagios_service

Attributes

check_command

check_intervaltitle: check_period

notification_interval

ensure

host_name

max_check_attempts

target

check_ping,check_ssh,etcChecking frequency"24x7","workhours",etcNotification frequencypresentabsentCorresponds to host definitionFailures before notification occurs.File destination

Bare minimum nagios manifest [1/9]class nagios {

package {"nagios":

ensure => present;"nagios-plugins-all":

ensure => present;}service {

"nagios":ensure => running,enable => true,require => Package["nagios"];

}include apacheinclude apache::sslonly

...

Bare minimum nagios manifest [2/9]file {

# Directory for all of the nagios config files..."/etc/nagios/conf":

ensure => directory,owner => "nagios",group => "nagios",recurse => true,ignore => [ ".svn", ".git", "CVS" ],source => "puppet:///filesroot/nagios/etc/conf",mode => 750,notify => Service["nagios"];

# Apache configuration file for nagios..."/etc/httpd/conf.d/nagios.conf":

require => Package["nagios"],notify => Service["httpd"],content => template("/path/to/file");

...

Bare minimum nagios manifest [3/9]...

# Nagios CGI interface configuration file..."/etc/nagios/cgi.conf":

require => Package["nagios"],notify => Service["nagios"],content => template("/path/to/file");

# main nagios configuration file..."/etc/nagios/nagios.cfg":

require => Package["nagios"],notify => Service["nagios"],content => template("/path/to/file");

}

...

Bare minimum nagios manifest [4/9]# We're going to manage the NRPE config file, but export# it to the clients instead of installing it locally.# This way, we can fill in a template with nagios' IP

address.@@file {

"/etc/nagios/nrpe.cfg":content => template("/path/to/file"),tag => "nagios";

}# While we're at it, lets export a firewall rule.# You might consider sprucing this up in production...@@iptables {

"9995 allow monitor":proto => "any",source => "${::ipaddress}",jump => "ACCEPT",

tag => "monitor";}

...

Bare minimum nagios manifest [5/9]...

# To list all of the nagios definitions here would use# up several more slides for minimal scholastic gain, so# this can be a homework assignment instead!

# Here, we need to create all of the nagios types# we've referenced throughout this example.# This includes:# The "24x7" nagios_timeperiod# The "mycontact" nagios_contact# nagios_command defines (check_ping, check_ssh,

host_notify)

# It's not hard, just refer to the puppet type reference.# Create the objects, fill in the attributes, and you'll# have your own fully-puppetized nagios installation up# and running in no time!

...

Bare minimum nagios manifest [6/9]# We can now collect host & service objects from client

nodes:Nagios_host <<| tag == "nagios" |>>Nagios_service <<| tag == "nagios" |>># Client manifests will have the text "include nagios::

client"class client {

package {"nagios-plugins-all":

ensure => present;"nrpe":

ensure => present;}service {

"nrpe":ensure => running,enable => true,require => Package["nrpe"];

}...

Bare minimum nagios manifest [7/9]# Export our host definition:@@nagios_host {

"${hostname}":ensure => present,address => "${ipaddress}",tag => "nagios",alias => "${fqdn}",target => "/etc/nagios/conf/dynamic_hosts.cfg",check_command => "check_ping!100!500",check_interval => "3",contacts => "mycontact",check_period => "24x7",max_check_attempts => 3,require => File["/etc/nagios/conf"],notify => Service["nagios"];

}

...

Bare minimum nagios manifest [8/9]# Begin to export our service definition(s)# Set defaults first in case we decide to add more later.Nagios_service {

target => "/etc/nagios/conf/dynamic_services.cfg",use => "generic-service",is_volatile => "0",host_name => "${hostname}",contacts => "mycontact",notification_period => "24x7",require => [ Nagios_host["${hostname}"],

File["/etc/nagios/conf"] ], notify => Service["nagios"]

} ...

Bare minimum nagios manifest [9/9]# Now do the actual service export. Just append another# N definitions here and they'll get auto-added to

nagios.@@nagios_service {

"${hostname}_check_ssh":ensure => present,service_description => "SSH",check_command => "check_ssh",check_period => "24x7",check_interval => "3",retry_interval => "3",max_check_attempts => "3",tag => "nagios",notification_interval => "720";

}# Final order of business, collect the nrpe config file:File <<| tag == "nagios" |>> { notify => Service["nrpe"]

}}

}

further reading● pro puppet

James Turnbull , Jeffrey McCunehttp://www.apress.com/9781430230571

● learning puppet - puppetlabshttp://docs.puppetlabs.com/learning/

● types documentation - puppetlabshttp://docs.puppetlabs.com/references/stable/type.html

● puppetlabs pdfshttp://info.puppetlabs.com/download-pdfs.html

● function referencehttp://docs.puppetlabs.com/references/stable/function.html

using puppet to pull strings - darksim · using puppet to pull strings: ... what is puppet and why...

Documents