using puppet

Copyright 2009 Trend Micro Inc. 1Classification 04/11/2023

Using Puppet

Alex Su

2011/12/26

Copyright 2009 Trend Micro Inc.Trend Micro Confidential

What is a system admin?

Copyright 2009 Trend Micro Inc.Trend Micro Confidential

Don’t look at me...

I wasn’t the last one to touch it...

Copyright 2009 Trend Micro Inc.Trend Micro Confidential

One Goal:

Revolutionize

System

Administration

Copyright 2009 Trend Micro Inc.Trend Micro Confidential

An Analogy

Programming SysAdmin

Low-level, non-portable

Assembly commands and files

Abstract, portable

Java / Python / Ruby Resources

Copyright 2009 Trend Micro Inc.Trend Micro Confidential

apt-get install openssh-server

vi /etc/ssh/sshd_config

/etc/init.d/ssh start

package { ssh: ensure => installed }

file { sshd_config:

name => “/etc/ssh/sshd_config”,

source => “puppet://server/apps/ssh/sshd

}

service { sshd: ensure => running, }

This

Becomes

Copyright 2009 Trend Micro Inc.Trend Micro Confidential

Puppet Quick Overview

• Stop administrating your environment and start developing it...• Re-usable code for managing your software & configurations• Provides a Domain Specific Language (DSL) to script with

– Classes, conditionals, selectors, variables, basic math, etc.

• Supports Linux, Solaris, BSD, OS X; Windows in process!

Copyright 2009 Trend Micro Inc.Trend Micro Confidential

Copyright 2009 Trend Micro Inc.Trend Micro Confidential

Copyright 2009 Trend Micro Inc.Trend Micro Confidential

Puppet Module Structure

Copyright 2009 Trend Micro Inc.Trend Micro Confidential

A Partial List of Puppet types

Packages • Supports 30 different package providers• Abstracted for your OS automatically• Specify ‘installed’, ‘absent’, or ‘latest’ for desired state• Change from ‘installed’ to ‘latest’ and deploy for quick

Upgrade

Services • Supports 10 different ‘init’ frameworks• Control whether a service starts on boot or is required to

be running always• A service can be notified to restart if a configuration file

has been changed

Files/Directories • Specify ownership & permissions• Load content from ‘files/’, ‘templates/’ or custom strings• Create symlinks• Supports 5 types to verify a file checksum• Purge a directory of files not ‘maintained’

Copyright 2009 Trend Micro Inc.Trend Micro Confidential

Nagios ‘Type’ SupportNagios Service @@nagios_service {

"load_check_${hostname}": service_description => "Load Averages", check_command => "load_check!3!5", host_name => "$fqdn", use => "generic-service";}

Nagios Service Group

@@nagios_servicegroup { "apache_servers": alias => "Apache Servers";}

Nagios Host @@nagios_host { $fqdn: ensure => present, hostgroups => "ldap", use => "generic-host";}

Nagios Host Group

@@nagios_hostgroup { "load_balancers": alias => "Load Balancers";}

Copyright 2009 Trend Micro Inc.Trend Micro Confidential

Copyright 2009 Trend Micro Inc.Trend Micro Confidential

Copyright 2009 Trend Micro Inc.Trend Micro Confidential

Sample site.pp

import "environment"

import "util"

import "constants"

import "bases"

import "nodes"

# global defaults

Exec { path => "/usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin" }

Copyright 2009 Trend Micro Inc.Trend Micro Confidential

Classes vs. Modules

• Why use the classes directory and the modules directory?

• Classes are more global and usually contain many different modules

• Modules are the smallest unit of measure that Puppet builds from

Copyright 2009 Trend Micro Inc.Trend Micro Confidential

Sample hadoop master class

class hadoop-master {

include kerberoskdc

include authclient

include ldapserver

include hadoop

include hbase

include pig

}

class pig {

# install packages

$packagelist = ["hadoop-pig"]

# install packages

package { 'base_pig_rpms':

ensure => installed,

name => $packagelist,

}

}

Copyright 2009 Trend Micro Inc.Trend Micro Confidential

Sample module init.pp

class resolv {

file { "resolv.conf":

path => "/etc/resolv.conf",

content => template("resolv/conf/resolv.conf.erb"),

owner => root,

group => root,

mode => 644,

ensure => file,

}

file { "hosts":

path => "/etc/hosts",

content => template("resolv/conf/hosts.erb"),

owner => root,

group => root,

mode => 644,

ensure => file,

}

}

Copyright 2009 Trend Micro Inc.Trend Micro Confidential

apt-get install openssh-server

vi /etc/ssh/sshd_config

/etc/init.d/ssh start

Package

Configuration

Service

Configuration should get modified after package installation

Service should restart when configuration changes

Copyright 2009 Trend Micro Inc.Trend Micro Confidential

package { ssh: ensure => installed }

file { sshd_config:

name => “/etc/ssh/sshd_config”,

source => “puppet://server/apps/ssh/sshd,

}

service { sshd:

ensure => running,

}

after => Package[ssh]

subscribe => [Package[ssh], File[sshd_config]]

Copyright 2009 Trend Micro Inc.Trend Micro Confidential

What is a template?

• Puppet templates are flat files containing Embedded Ruby (ERB) variables

• hadoop/conf/hadoop-metrics.properties.erb

<% if ganglia_hosts.length > 0 %>

dfs.class=org.apache.hadoop.metrics.ganglia.GangliaContext31

dfs.period=10

dfs.servers=<% ganglia_hosts.each do |host| -%><%= host %> <% end -%>

<% end %>

• resolv/conf/hosts.erb

<% ip_host_map.each do |ip,hosts| -%>

<%= ip %> <%= hosts %>

<% end -%>

Copyright 2009 Trend Micro Inc.Trend Micro Confidential

What is a node?

• Node definitions look just like classes, including supporting inheritance, but they are special in that when a node (a managed computer running the Puppet client) connects to the Puppet master daemon.

• nodes.pp

or

node 'tm5-master.client.tw.trendnet.org' inherits hadoop_master {}

node 'tm5-master.client.tw.trendnet.org' {

include kerberoskdc

include authclient

include ldapserver

include hadoop

include hbase

include pig

}

Copyright 2009 Trend Micro Inc.Trend Micro Confidential

Puppet Network Overview

• Configuration allows for manual synchronizations or a set increment• Client or server initiated synchronizations• Client/Server configuration leverages a Certificate Authority (CA) on the • Puppet Master to sign client certificates to verify authenticity• Transmissions of all data between a master & client are encrypted

Copyright 2009 Trend Micro Inc.Trend Micro Confidential

Every Client:

• Retrieve resource catalog from central server• Determine resource order• Check each resource in turn, fixing if necessary • Rinse and repeat, every 30 minutes

Copyright 2009 Trend Micro Inc.Trend Micro Confidential

Every Resource:

• Retrieve current state (e.g., by querying dpkg db or doing a stat)

• Compare to desired state• Fix, if necessary (or just log)

Copyright 2009 Trend Micro Inc.Trend Micro Confidential

tail –f /var/log/message

Copyright 2009 Trend Micro Inc.Trend Micro Confidential

TM-Puppet

/etc/puppet

files/ manifests/ modules/auth.conf

autosign.conf

puppet.conf

byhost/

nodes.pp

site.pp

host1/

host2/

host3/

hadoop/

manifests/

templates/

init.pp

util.pp

hbase/

pig/

bases.pp

Copyright 2009 Trend Micro Inc.Trend Micro Confidential

Reference

• Deployment Tools• ERB - Ruby Templating

Copyright 2009 Trend Micro Inc. 29Classification 04/11/2023

Questions?

Copyright 2009 Trend Micro Inc. 30Classification 04/11/2023

THANK YOU!