using puppet
TRANSCRIPT
Copyright 2009 Trend Micro Inc. 1Classification 04/11/2023
Using Puppet
Alex Su
2011/12/26
Copyright 2009 Trend Micro Inc.Trend Micro Confidential
What is a system admin?
Copyright 2009 Trend Micro Inc.Trend Micro Confidential
Don’t look at me...
I wasn’t the last one to touch it...
Copyright 2009 Trend Micro Inc.Trend Micro Confidential
One Goal:
Revolutionize
System
Administration
Copyright 2009 Trend Micro Inc.Trend Micro Confidential
An Analogy
Programming SysAdmin
Low-level, non-portable
Assembly commands and files
Abstract, portable
Java / Python / Ruby Resources
Copyright 2009 Trend Micro Inc.Trend Micro Confidential
apt-get install openssh-server
vi /etc/ssh/sshd_config
/etc/init.d/ssh start
package { ssh: ensure => installed }
file { sshd_config:
name => “/etc/ssh/sshd_config”,
source => “puppet://server/apps/ssh/sshd
}
service { sshd: ensure => running, }
This
Becomes
Copyright 2009 Trend Micro Inc.Trend Micro Confidential
Puppet Quick Overview
• Stop administrating your environment and start developing it...• Re-usable code for managing your software & configurations• Provides a Domain Specific Language (DSL) to script with
– Classes, conditionals, selectors, variables, basic math, etc.
• Supports Linux, Solaris, BSD, OS X; Windows in process!
Copyright 2009 Trend Micro Inc.Trend Micro Confidential
Copyright 2009 Trend Micro Inc.Trend Micro Confidential
Copyright 2009 Trend Micro Inc.Trend Micro Confidential
Puppet Module Structure
Copyright 2009 Trend Micro Inc.Trend Micro Confidential
A Partial List of Puppet types
Packages • Supports 30 different package providers• Abstracted for your OS automatically• Specify ‘installed’, ‘absent’, or ‘latest’ for desired state• Change from ‘installed’ to ‘latest’ and deploy for quick
Upgrade
Services • Supports 10 different ‘init’ frameworks• Control whether a service starts on boot or is required to
be running always• A service can be notified to restart if a configuration file
has been changed
Files/Directories • Specify ownership & permissions• Load content from ‘files/’, ‘templates/’ or custom strings• Create symlinks• Supports 5 types to verify a file checksum• Purge a directory of files not ‘maintained’
Copyright 2009 Trend Micro Inc.Trend Micro Confidential
Nagios ‘Type’ SupportNagios Service @@nagios_service {
"load_check_${hostname}": service_description => "Load Averages", check_command => "load_check!3!5", host_name => "$fqdn", use => "generic-service";}
Nagios Service Group
@@nagios_servicegroup { "apache_servers": alias => "Apache Servers";}
Nagios Host @@nagios_host { $fqdn: ensure => present, hostgroups => "ldap", use => "generic-host";}
Nagios Host Group
@@nagios_hostgroup { "load_balancers": alias => "Load Balancers";}
Copyright 2009 Trend Micro Inc.Trend Micro Confidential
Copyright 2009 Trend Micro Inc.Trend Micro Confidential
Copyright 2009 Trend Micro Inc.Trend Micro Confidential
Sample site.pp
import "environment"
import "util"
import "constants"
import "bases"
import "nodes"
# global defaults
Exec { path => "/usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin" }
Copyright 2009 Trend Micro Inc.Trend Micro Confidential
Classes vs. Modules
• Why use the classes directory and the modules directory?
• Classes are more global and usually contain many different modules
• Modules are the smallest unit of measure that Puppet builds from
Copyright 2009 Trend Micro Inc.Trend Micro Confidential
Sample hadoop master class
class hadoop-master {
include kerberoskdc
include authclient
include ldapserver
include hadoop
include hbase
include pig
}
class pig {
# install packages
$packagelist = ["hadoop-pig"]
# install packages
package { 'base_pig_rpms':
ensure => installed,
name => $packagelist,
}
}
Copyright 2009 Trend Micro Inc.Trend Micro Confidential
Sample module init.pp
class resolv {
file { "resolv.conf":
path => "/etc/resolv.conf",
content => template("resolv/conf/resolv.conf.erb"),
owner => root,
group => root,
mode => 644,
ensure => file,
}
file { "hosts":
path => "/etc/hosts",
content => template("resolv/conf/hosts.erb"),
owner => root,
group => root,
mode => 644,
ensure => file,
}
}
Copyright 2009 Trend Micro Inc.Trend Micro Confidential
apt-get install openssh-server
vi /etc/ssh/sshd_config
/etc/init.d/ssh start
Package
Configuration
Service
Configuration should get modified after package installation
Service should restart when configuration changes
Copyright 2009 Trend Micro Inc.Trend Micro Confidential
package { ssh: ensure => installed }
file { sshd_config:
name => “/etc/ssh/sshd_config”,
source => “puppet://server/apps/ssh/sshd,
}
service { sshd:
ensure => running,
}
after => Package[ssh]
subscribe => [Package[ssh], File[sshd_config]]
Copyright 2009 Trend Micro Inc.Trend Micro Confidential
What is a template?
• Puppet templates are flat files containing Embedded Ruby (ERB) variables
• hadoop/conf/hadoop-metrics.properties.erb
<% if ganglia_hosts.length > 0 %>
dfs.class=org.apache.hadoop.metrics.ganglia.GangliaContext31
dfs.period=10
dfs.servers=<% ganglia_hosts.each do |host| -%><%= host %> <% end -%>
<% end %>
• resolv/conf/hosts.erb
<% ip_host_map.each do |ip,hosts| -%>
<%= ip %> <%= hosts %>
<% end -%>
Copyright 2009 Trend Micro Inc.Trend Micro Confidential
What is a node?
• Node definitions look just like classes, including supporting inheritance, but they are special in that when a node (a managed computer running the Puppet client) connects to the Puppet master daemon.
• nodes.pp
or
node 'tm5-master.client.tw.trendnet.org' inherits hadoop_master {}
node 'tm5-master.client.tw.trendnet.org' {
include kerberoskdc
include authclient
include ldapserver
include hadoop
include hbase
include pig
}
Copyright 2009 Trend Micro Inc.Trend Micro Confidential
Puppet Network Overview
• Configuration allows for manual synchronizations or a set increment• Client or server initiated synchronizations• Client/Server configuration leverages a Certificate Authority (CA) on the • Puppet Master to sign client certificates to verify authenticity• Transmissions of all data between a master & client are encrypted
Copyright 2009 Trend Micro Inc.Trend Micro Confidential
Every Client:
• Retrieve resource catalog from central server• Determine resource order• Check each resource in turn, fixing if necessary • Rinse and repeat, every 30 minutes
Copyright 2009 Trend Micro Inc.Trend Micro Confidential
Every Resource:
• Retrieve current state (e.g., by querying dpkg db or doing a stat)
• Compare to desired state• Fix, if necessary (or just log)
Copyright 2009 Trend Micro Inc.Trend Micro Confidential
tail –f /var/log/message
Copyright 2009 Trend Micro Inc.Trend Micro Confidential
TM-Puppet
/etc/puppet
files/ manifests/ modules/auth.conf
autosign.conf
puppet.conf
byhost/
nodes.pp
site.pp
host1/
host2/
host3/
hadoop/
manifests/
templates/
init.pp
util.pp
hbase/
pig/
bases.pp
Copyright 2009 Trend Micro Inc.Trend Micro Confidential
Reference
• Deployment Tools• ERB - Ruby Templating
Copyright 2009 Trend Micro Inc. 29Classification 04/11/2023
Questions?
Copyright 2009 Trend Micro Inc. 30Classification 04/11/2023
THANK YOU!