puppet buero20 presentation

PuppetAutomated System Configuration Management

Martin Alfke <[email protected]>

1

Wednesday, December 8, 2010

Agenda• Part I - Puppet Basics

• General + Communication

• Manifests, Modules, Templates + Functions

• Part II - Puppet Workshop

• Part III - Working with Puppet

• GIT/SVN for Puppet

• Production / Test / Development

• Monitoring

2


General• “Put simply, Puppet is a system for automating

system administration tasks”

• Puppet...

• is a declarative language for expressing system configuration

• is a client-server distribution

• Requirements:

• Ruby > 1.8.1 < 1.9

• Facter

3


Communication• Security

• SSL certificate based authentication

• manual signing of certificate requests

• Layers:

• Configuration Language

• Transaction layer

• Resource Abstraction Layer

4


Supported Platforms• Linux

• Debian / Ubuntu / Fedora / CentOS / RHEL / OEL / Mandriva / SuSE / Gentoo

• BSD

• FreeBSD / OpenBSD

• Other Unix

• OS X / Solaris / HP-UX

• Windows - coming in 2010

5


Functional Overview• Clients connect to

Puppet Master

• Puppet Master send clients description of tasks

• Puppet Master stores Clients reports

• Reports can be imported into dashboard database

• Dashboard web interface to reports

6


Facter

7

/usr/bin/facterarchitecture => amd64domain => buero20.localfacterversion => 1.5.7fqdn => puppet.buero20.local...interfaces => eth0,eth1ipaddress => 10.0.2.15...operatingsystem => Debianprocessorcount => 1


Puppet Configuration Language - 1-6

8

• manifests/site.pp

• Global file with node definitions

• modules/<name>/manifests/init.pp

• Module initialization

• Use lower case for names (modules, templates, functions, defines, exec, resources,...)



9

• Resources

• user - create or remove users

• group - create or remove groups

• package install or remove distribution packages

• file - create directories, symlinks, copy files

• cron - add cron jobs

• service - run or stop services like daemons



10

• Classes

• aggregate resources for easier use

• subclasses (=nested classes) for modularity

• parameterised classes for more flexible handling

• classes support inheritance



11

• Definitions

• reusable objects

• Modules

• combine collections of resources, classes and definitions



12

• Chaining resources

• make sure that a service is restarted after filechange

• make sure that config file is copied prior starting a service

• make sure that a package is installed prior starting the service



13

• Nodes

• connect modules and clases to systems

• nodenames are short hostname, fqdn or “default”


Manifests

• Define static resourcesfile { “/etc/passwd”:

owner => root,group => root,mode => 644,

}

• Static resources have full path and name.

14


Manifests with facter Variables

file { “sshconfig”:name => $operatingsystem ? {

solaris => “/usr/local/etc/ssh/sshd_config”,default => “/etc/ssh/sshd_config”,

},owner => root,group => root,mode => 644,

}

• Using facter variables inside a definition

15


Manifest with Sub-Classesclass mysql {

class client { class packages { package { "mysql-client": ensure => installed } } } class server { class packages { package { "mysql-server": ensure => installed } package { "mysql-common": ensure => installed }

16


Manifests with Exec

file {"/etc/apt/keys/pgp_key.asc": owner => root, group => root, mode => 640, source => "puppet://$server/files/etc/apt/keys/pgp_key.asc"}exec { "/usr/bin/apt-key add /etc/apt/keys/pgp_key.asc": unless => "/bin/sh -c '[ `/usr/bin/apt-key list | grep buildd | \ wc -l` -eq 1 ]'"}

17


Manifests with Subscriptionfile {"/etc/apt/keys/puppet.key": owner => root, group => root, mode => 640, source => "puppet:///files/etc/apt/keys/puppet.key"}exec { subscribe-base-config-puppet-key: command => "/usr/bin/apt-key add /etc/apt/keys/puppet.key; \ /usr/bin/apt-get update", logoutput => false, refreshonly => true, subscribe => File["/etc/apt/keys/puppet.key"]}

18


Modules - Directory structure

• Directory structure - e.g. /etc/ssh/sshd_configmodule/sshd/

manifests/init.pp

files/etc/

ssh/sshd_config

• Modules require strict directories naming.

19


Modules - Initialization Manifest

• modules/manifests/sshd/init.ppclass sshd {

file { “/etc/ssh/sshd_config”:mode => 644,source => “puppet:///modules/sshd/etc/ssh/sshd_config”,

}}

• init.pp manifest will be integrated automatically when class name is equal to module name

20


Templates - Directory Structure

• Directory structure + content - e.g. Network settingsnetwork/

manifests/init.pp

templates/network.erb

• Templates require strict directory naming (like modules)

21


Templates - Initialization Manifest

• Manifests - init.ppfile { “/etc/sysconfig/network”:

content => template(“templates/network.erb”),}

• Templatess - network.erbNETWORKING=yesHOSTNAME=<%= hostname %>NOZEROCONF=yes

• Templates may use facter variables

22


Functions• Directory structure e.g. read parameter

from configuration file using facter:lib/

facter/function.rb

• Content of library functions function.rb:require ‘facter’ Facter.add(“PUPPET_FUNCTION”) do %x{/bin/grep -E “^PUPPET_FUNCTION=” /etc/puppet_function | sed -e ‘s/*.=//’ } .chomp endend

23









• Monitoring

24


Puppet Workshop• Installation - Puppet master and client on puppet master only

• Initialization

• Installation - Puppet client on puppet client only

• Modules

• User Management

• Apache sites configuration

• Templating for /etc/hosts

• Setup Reporting and Dashboard

25


Puppet Workshop - Installation - 1-5

•check requirements:

• ruby --version

• ruby -rshadow -e’print “OK\n”’

26



•from source

• fetch and extract source

• wget http://puppetlabs.com/downloads/facter/facter-1.5.8.tar.gz

• wget http://puppetlabs.com/downloads/puppet/puppet-2.6.2.tar.gz

27



• install

• ruby install.rb

• mkdir /etc/puppet

28



•configuration

• puppet --mkuser

• puppet --genconfig > /etc/puppet/puppet.conf

• vi /etc/hosts - add entry for nodename puppet if not existing

29



•manifests/site.pp

• add empty section for default nodenode default {

notice(“default node”)}

30


Puppet Workshop - Initialization

•first start of puppet:

• puppetd --test

•puppet CA

• check client certificate

• puppetca --list

• puppetca --list --all

31


Puppet Workshop - Modules - 1-2

•File Structure

• mkdir -p modules/<name>/{manifests,files}

•modules/<name>/manifests/init.ppclass <name> {

notice(“module <name>”)}

32


Puppet Workshop - Modules - 2-2

•including modules in manifests/site.ppnode default {

include <name>}

33


Puppet Workshop - Account Module - 1-6

• User Management

• create your personal login

• create home directory

1. Module directoriesmkdir -p modules/users/{manifests,files}

34


Puppet Workshop - Account Module - 2-62. Module init.pp

vi modules/users/manifests/init.pp

class users {user{ "martin":!home!! ! ! ! ! => "/home/martin",!managehome! => true,!shell! ! ! ! ! ! => "/bin/bash",!comment!! ! ! => "Martin Alfke",!ensure! ! ! ! ! => present,

35



#!uid! ! ! ! ! ! => 0,#!gid !! ! ! ! ! => 0,# password ! ! => '0OfNn.f5krlF2',#!allowdupe ! ! => true,

}}

36



3. modify site.pp

vi manifests/site.pp

node default {! include users}

37


Puppet Workshop - Account Module - 5-61. create new filemkdir -p modules/users/files/home/martin/www/

cat > modules/users/files/home/martin/www/index.html << EOF<html><head><title>My testsite</title></head><body>foo</body></html>EOF

38


Puppet Workshop - Account Module - 6-62. Module init.ppadd to modules/users/manifests/init.ppclass users {

......file {“/home/martin/www”:! ensure => directory,}file{“home/martin/www/index.html”:! source => “puppet:///modules/users/home/martin/www/index.html”,}

}

39


Puppet Workshop - Apache Module - 1-6• Apache sites Management

• packages

• your own vhost config

1. Module directoriesmkdir -p modules/apache/{manifests,files}mkdir -p modules/apache/files/etc/apache2/sites-available/

40


Puppet Workshop - Apache Module - 2-62. your vhost definition

cat > modules/apache/files/etc/apache2/sites-available/blit-test << EOFListen 88NameVirtualHost *:88<VirtualHost *:88>! DocumentRoot /home/martin/www</VirtualHost>EOF

41


Puppet Workshop - Apache Module - 3-62. Module init.ppvi modules/apache/manifests/init.pp

class apache {! package{“apache2”: ensure!! ! => present }! package{“php5-mysql”: ensure! => present }! file{“/etc/apache2/sites-available/blit-test”:! ! source => “puppet:///modules/apache/etc/apache2/sites-available/blit-test”,! }}

42


Puppet Workshop - Apache Module - 4-6

3. Add to node default manifest site.pp

include apache

43


Puppet Workshop - Apache Module - 5-6• Apache sites Management

• enabling sites with function1. Add to apache init.ppclass apache {...define vhost ($ensure = ʻpresentʼ) {! case $ensure {! ! ʻpresentʼ: {! ! exec { “/usr/sbin/a2ensite $name”:! ! ! unless => “/bin/readlink -e /etc/apache2/sites-enabled/$name”! ! }! ! }

44



! ! ʻabsentʼ: {! ! ! exec { “/usr/sbin/a2dissite $name”:! ! ! ! onlyif => “/bin/readlink -e /etc/apache2/sites-enabled/$name”! ! ! }! ! }! ! default: { err (“Unknown ensure value: $ensure) }! }

45



! vhost {“blit-test”:! ! ensure => “present”,! }! vhost{“000-default”:! ! ensure => absent,! }}

46


Puppet Workshop - Templates - 1-2

• File Structure

• mkdir -p modules/<name>/{manifests,templates}

• modules/<name>/manifests/init.ppclass <name> {

notice(“module <name>”)}

47


Puppet Workshop - Templates - 2-2

• including modules in manifests/site.ppnode default {

include <name>}

48


Puppet Workshop - Hosts Template - 1-3

• Hosts Template

• configure entries in /etc/host

1. Module directoriesmkdir -p modules/hosts/{manifests,templates}

49



2. Module init.pp

vi modules/hosts/manifests/init.pp

class hosts {! file{“/etc/hosts”:! ! owner! => root,! ! group!! => root,! ! content!=> template(hosts.erb),! }}

50



3. template hosts.erb

vi templates/hosts.erb

127.0.0.1!localhost<%= ipaddress %>!<%= fqdn %> <%= hostname %>192.168.0.2! puppet192.168.0.4! mysql! mysqlmaster

51


Puppet Workshop - Functions - 1-2

• File Structure

• mkdir -p modules/<name>/lib/

52


Puppet Workshop - Functions - 2-2

• including modules in manifests/site.ppnode default {

include <name>}

53


Puppet Workshop - Facter Function - 1-4

• Facter Function

• provide additional fact

1. Module directoriesmkdir -p modules/facter/lib/facter

54



2. function.rb

vi modules/facter/lib/facter/function.rb

require ʻfacterʼFacter.add(“PUPPET_FUNCTION”) do! setcode do! ! %x{/bin/grep -E “^PUPPET_FUNCTION=” /etc/puppet_function | sed -e ʻs/.*=//ʼ}.chomp! endend

55



3. puppet.confsection [main]pluginsync = true

4. puppet runpuppetd --test

5. call facter puppet functionfacter --puppet | grep puppet_function

56



3. use custom facts in manifestscase $puppet_function {! “MYSQL”:! ! { include mysql }! “APACHE”:!! { include apache }! “PROXY”:! ! { include proxy }...}

57


Puppet Workshop - Dashboard - 1-5

• Installation

• fetch and extract sourc

• wget http://puppetlabs.com/downloads/dashboard/puppet-dashboard-1.0.4.tgz

• install mysql-server

58



• Configuration

• edit /usr/share/puppet-dashboard/config/database.yaml

• Create Database

• cd /usr/share/puppet-dashboard; rake RAILS_ENV db:create or

• mysql -Ne ‘create database dashboard;’

59



• Initialize Database

• cd /usr/share/puppet-dashboard; rake RAILS_ENV db:migrate

• Import Reports

• cd /usr/share/puppet-dashboard; rake RAILS_ENV=production reports:import

60



• Start service

• cd /usr/share/puppet-dashboard; ./bin/server -e production -d

• Review your Dashboard in browser

• http://<your puppetmaster ip>:3000/

61



• add error to manifest (e.g. point source to a non existing file)

• run puppetd

• puppetd --test

• import data

• cd /usr/share/puppet-dashboard; rake RAILS_ENV=production reports:import

• review dashboard

62









• Monitoring

63


Puppet into GIT/SVN

• Why revision control system?

• Co-working

• Branches

• Which RCS System?

• Which ever you prefer

64


Puppet Staging• Production, Test and Development

• /etc/puppet/puppet.conf

• [main] - environment = ...

• [development] - modulepath=/etc/puppet/development/modules

• [testing] - modulepath=/etc/puppet/testing/modules

• [production] - modulepath=/etc/puppet/production/modules

65


Puppet Monitoring

• Puppet Dashboard

• Configure puppet to store results

• [master] section: reports=http, store

• [agent] (v2.6) or [puppetd] section: report=true

• Configure Database (e.g. MySQL)

66


Puppet Dashboard

67


PuppetAutomated System Configuration Management

Martin Alfke <[email protected]>

Thank you !Questions ?

68


puppet buero20 presentation

Technology