using puppet

Copyright 2009 Trend Micro Inc. 1Classification 04/11/2023

Using Puppet

Alex Su

2011/12/26

Copyright 2009 Trend Micro Inc.Trend Micro Confidential

What is a system admin?


Don’t look at me...

I wasn’t the last one to touch it...


One Goal:

Revolutionize

System

Administration


An Analogy

Programming SysAdmin

Low-level, non-portable

Assembly commands and files

Abstract, portable

Java / Python / Ruby Resources


apt-get install openssh-server

vi /etc/ssh/sshd_config

/etc/init.d/ssh start

package { ssh: ensure => installed }

file { sshd_config:

name => “/etc/ssh/sshd_config”,

source => “puppet://server/apps/ssh/sshd

}

service { sshd: ensure => running, }

This

Becomes


Puppet Quick Overview

• Stop administrating your environment and start developing it...• Re-usable code for managing your software & configurations• Provides a Domain Specific Language (DSL) to script with

– Classes, conditionals, selectors, variables, basic math, etc.

• Supports Linux, Solaris, BSD, OS X; Windows in process!


Puppet Module Structure


A Partial List of Puppet types

Packages • Supports 30 different package providers• Abstracted for your OS automatically• Specify ‘installed’, ‘absent’, or ‘latest’ for desired state• Change from ‘installed’ to ‘latest’ and deploy for quick

Upgrade

Services • Supports 10 different ‘init’ frameworks• Control whether a service starts on boot or is required to

be running always• A service can be notified to restart if a configuration file

has been changed

Files/Directories • Specify ownership & permissions• Load content from ‘files/’, ‘templates/’ or custom strings• Create symlinks• Supports 5 types to verify a file checksum• Purge a directory of files not ‘maintained’


Nagios ‘Type’ SupportNagios Service @@nagios_service {

"load_check_${hostname}": service_description => "Load Averages", check_command => "load_check!3!5", host_name => "$fqdn", use => "generic-service";}

Nagios Service Group

@@nagios_servicegroup { "apache_servers": alias => "Apache Servers";}

Nagios Host @@nagios_host { $fqdn: ensure => present, hostgroups => "ldap", use => "generic-host";}

Nagios Host Group

@@nagios_hostgroup { "load_balancers": alias => "Load Balancers";}


Sample site.pp

import "environment"

import "util"

import "constants"

import "bases"

import "nodes"

# global defaults

Exec { path => "/usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin" }


Classes vs. Modules

• Why use the classes directory and the modules directory?

• Classes are more global and usually contain many different modules

• Modules are the smallest unit of measure that Puppet builds from


Sample hadoop master class

class hadoop-master {

include kerberoskdc

include authclient

include ldapserver

include hadoop

include hbase

include pig

}

class pig {

# install packages

$packagelist = ["hadoop-pig"]

# install packages

package { 'base_pig_rpms':

ensure => installed,

name => $packagelist,

}

}


Sample module init.pp

class resolv {

file { "resolv.conf":

path => "/etc/resolv.conf",

content => template("resolv/conf/resolv.conf.erb"),

owner => root,

group => root,

mode => 644,

ensure => file,

}

file { "hosts":

path => "/etc/hosts",

content => template("resolv/conf/hosts.erb"),

owner => root,

group => root,

mode => 644,

ensure => file,

}

}


apt-get install openssh-server

vi /etc/ssh/sshd_config

/etc/init.d/ssh start

Package

Configuration

Service

Configuration should get modified after package installation

Service should restart when configuration changes


package { ssh: ensure => installed }

file { sshd_config:

name => “/etc/ssh/sshd_config”,

source => “puppet://server/apps/ssh/sshd,

}

service { sshd:

ensure => running,

}

after => Package[ssh]

subscribe => [Package[ssh], File[sshd_config]]


What is a template?

• Puppet templates are flat files containing Embedded Ruby (ERB) variables

• hadoop/conf/hadoop-metrics.properties.erb

<% if ganglia_hosts.length > 0 %>

dfs.class=org.apache.hadoop.metrics.ganglia.GangliaContext31

dfs.period=10

dfs.servers=<% ganglia_hosts.each do |host| -%><%= host %> <% end -%>

<% end %>

• resolv/conf/hosts.erb

<% ip_host_map.each do |ip,hosts| -%>

<%= ip %> <%= hosts %>

<% end -%>


What is a node?

• Node definitions look just like classes, including supporting inheritance, but they are special in that when a node (a managed computer running the Puppet client) connects to the Puppet master daemon.

• nodes.pp

or

node 'tm5-master.client.tw.trendnet.org' inherits hadoop_master {}

node 'tm5-master.client.tw.trendnet.org' {

include kerberoskdc

include authclient

include ldapserver

include hadoop

include hbase

include pig

}


Puppet Network Overview

• Configuration allows for manual synchronizations or a set increment• Client or server initiated synchronizations• Client/Server configuration leverages a Certificate Authority (CA) on the • Puppet Master to sign client certificates to verify authenticity• Transmissions of all data between a master & client are encrypted


Every Client:

• Retrieve resource catalog from central server• Determine resource order• Check each resource in turn, fixing if necessary • Rinse and repeat, every 30 minutes


Every Resource:

• Retrieve current state (e.g., by querying dpkg db or doing a stat)

• Compare to desired state• Fix, if necessary (or just log)


tail –f /var/log/message


TM-Puppet

/etc/puppet

files/ manifests/ modules/auth.conf

autosign.conf

puppet.conf

byhost/

nodes.pp

site.pp

host1/

host2/

host3/

hadoop/

manifests/

templates/

init.pp

util.pp

hbase/

pig/

bases.pp


Reference

• Deployment Tools• ERB - Ruby Templating

http://coretech-backend-dev.tw.trendnet.org/wiki/Cloud_Technologies

http://coretech-backend-dev.tw.trendnet.org/wiki/Cloud_Technologies

http://www.ruby-doc.org/stdlib-1.9.3/libdoc/erb/rdoc/ERB.html

http://www.ruby-doc.org/stdlib-1.9.3/libdoc/erb/rdoc/ERB.html


Questions?


THANK YOU!

using puppet

Technology

nagios host

fromtrend microcopyright

puppet templates

puppet client

puppet master daemon

group load

load balancers

classes directory