How to Create a Next-GenExample42 Puppet Module

in 5 Minutes

Alessandro FranceschiDevOps Days Rome 2012

Prepare for an info overload rushand an Ignite's verbosity world record

Example42 Next-Gen modules...Support for different OS (Default: RedHat / Debian)

Fully parametrized classes

Data separation with Hiera/ENC parameters lookup

Extreme customization options

Module behavior customization options

Full and partial decommissioning support

Optional automatic Monitoring / Firewalling

Optional Puppi integration

Optional Debug and Auditing

Fetch the sourcesgit clone --recursivegit://github.com/example42/puppet-modules-nextgen.git

Cloning into 'puppet-modules-nextgen'...remote: Counting objects: 608, done.remote: Compressing objects: 100% (317/317), done.remote: Total 608 (delta 281), reused 578 (delta 252)Receiving objects: 100% (608/608), 102.59 KiB | 118 KiB/s, done.Resolving deltas: 100% (281/281), done.Submodule 'Example42-documentation' (git://github.com/example42/Example42-documentation.git) registered for path 'Example42-documentation'Submodule 'Example42-tools' (git://github.com/example42/Example42-tools.git) registered for path 'Example42-tools'[...]

Enjoy modules' varietycd puppet-modules-nextgen ; lsExample42-documentation! maven! ! ! ! puppiExample42-tools! ! ! mcollective! ! ! rclocalGemfile!! ! ! ! monitor! ! ! ! redisREADME.rdoc! ! ! ! munin! ! ! ! resolverRakefile!! ! ! ! mysql! ! ! ! rsyncactivemq!! ! ! ! nagios! ! ! ! rsyncsshapache! ! ! ! ! nginx! ! ! ! rvmconcat! ! ! ! ! nrpe!! ! ! ! solrexample42! ! ! ! ntp!! ! ! ! splunkfirewall!! ! ! ! openntpd!! ! ! stdlib42foo!! ! ! ! ! openssh! ! ! ! sudofoo_webapp! ! ! ! openvpn! ! ! ! tartarusforeman!! ! ! ! orientdb!! ! ! tftphaproxy!! ! ! ! pentaho! ! ! ! tomcaticinga! ! ! ! ! php!! ! ! ! vagrantiptables!! ! ! ! postfix! ! ! ! vsftpdjava! ! ! ! ! postgresql! ! ! wgetjboss! ! ! ! ! pupmod-concat! ! wordpressjenkins!! ! ! ! puppet! ! ! ! xinetdlibvirt!! ! ! ! puppetdashboard! ! yumlogstash!! ! ! ! puppetdb

Create a new module from fooExample42-tools/module_clone.shThis script creates a skeleton for a new module based on different Example42 foo module templates.[...]

Enter the name of the new module based on foo: lighttpdCOPYING MODULEbuilding file list ... doneModulefileREADME.rdocRakefilemanifests/manifests/init.ppmanifests/params.ppmanifests/spec.ppspec/spec/spec_helper.rbspec/classes/spec/classes/foo_spec.rb[...]RENAMING FILESRenamed lighttpd/spec/classes/foo_spec.rb to lighttpd/spec/classes/lighttpd_spec.rb---------------------------------------------------CHANGING FILE CONTENTSChanged lighttpd/manifests/init.ppChanged lighttpd/manifests/params.ppChanged lighttpd/manifests/spec.ppChanged lighttpd/ModulefileChanged lighttpd/README.rdocChanged lighttpd/spec/classes/lighttpd_spec.rbChanged lighttpd/spec/spec_helper.rbModule lighttpd createdStart to edit lighttpd/manifests/params.pp to customize it

Explore module's contentsfind lighttpd/lighttpd/

lighttpd/speclighttpd/spec/spec_helper.rblighttpd/spec/classeslighttpd/spec/classes/lighttpd_spec.rblighttpd/Rakefile

lighttpd/Modulefile

lighttpd/manifestslighttpd/manifests/init.pplighttpd/manifests/params.pplighttpd/manifests/spec.pp

lighttpd/README.rdoc

lighttpd/templateslighttpd/templates/spec.erb

git initgit add .git commit -m "Example42-tools/module_clone.sh lighttpd"

Edit params.pp 1/2vi lighttpd/manifests/params.ppclass lighttpd::params { $package = $::operatingsystem ? { default => 'lighttpd', } $service = $::operatingsystem ? { default => 'lighttpd', }[...] $config_dir = $::operatingsystem ? { /(?i:Debian|Ubuntu|Mint)/ => '/etc/lighttpd/conf-enabled', default => '/etc/lighttpd/conf.d', } $config_file_init = $::operatingsystem ? { /(?i:Debian|Ubuntu|Mint)/ => '/etc/default/lighttpd', default => '/etc/sysconfig/lighttpd', } $data_dir = $::operatingsystem ? { /(?i:Debian|Ubuntu|Mint)/ => '/var/run/lighttpd', default => '/var/www', } $log_dir = $::operatingsystem ? { default => '/var/log/lighttpd', }

$log_file = $::operatingsystem ? { default => '/var/log/lighttpd.log', } $port = '80' $protocol = 'tcp'

Edit params.pp 2/2vi lighttpd/manifests/params.pp# You should not need to change the following ones

class lighttpd::params {[...] # General Settings $my_class = '' $source = '' $source_dir = '' $source_dir_purge = false $template = '' $options = '' $service_autorestart = true $version = 'present' $absent = false $disable = false $disableboot = false

### General module variables that can have a site or per module default $monitor = false $monitor_tool = '' $monitor_target = $::ipaddress $firewall = false $firewall_tool = '' $firewall_src = '0.0.0.0/0' $firewall_dst = $::ipaddress $puppi = false $puppi_helper = 'standard' $debug = false $audit_only = false

git status#! modified: manifests/params.pp

git add .git commit -m "Basic support for Ubuntu/RedHat families"

Review and sign modulevi lighttpd/README.rdoc= Puppet module: lighttpdThis is a Puppet module for lighttpd based on the second generation layout ("NextGen") of Example42 Puppet Modules.Made by Alvagante / Example42Official site: http://www.example42.comOfficial git repository: http://github.com/example42/puppet-lighttpdReleased under the terms of Apache 2 License.

== USAGE - Basic management* Install lighttpd with default settings class { 'lighttpd': }

* Use custom source directory for the whole configuration dir class { 'lighttpd': source_dir => 'puppet:///modules/example42/lighttpd/conf/', }

* Use custom template for main config file. Note that template and source arguments are alternative. class { 'lighttpd': template => 'example42/lighttpd/lighttpd.conf.erb', }

* Automatically include a custom subclass class { 'lighttpd': my_class => 'example42::my_lighttpd', }

* Activate automatic monitoring class { 'lighttpd': monitor => true, monitor_tool => [ 'nagios' , 'monit' , 'munin' ], }

git status#! modified: README.rdoc

git add .git commit -m "Changed module's author to ... me"

Explore the module's logic 1/4less lighttpd/manifests/init.pp# = Class: lighttpd# == Parameters## [*my_class*]# Name of a custom class to autoload to manage module's customizations# If defined, lighttpd class will automatically "include $my_class"# Can be defined also by the (top scope) variable $lighttpd_myclass## [*source*]# Sets the content of source parameter for main configuration file# If defined, lighttpd main config file will have the param: source => $source# Can be defined also by the (top scope) variable $lighttpd_source[...]class lighttpd ( $my_class = params_lookup( 'my_class' ), $source = params_lookup( 'source' ), $source_dir = params_lookup( 'source_dir' ), $source_dir_purge = params_lookup( 'source_dir_purge' ), $template = params_lookup( 'template' ), $service_autorestart = params_lookup( 'service_autorestart' , 'global' ), $options = params_lookup( 'options' ), $version = params_lookup( 'version' ), $disable = params_lookup( 'disable' ),[...] $data_dir = params_lookup( 'data_dir' ), $log_dir = params_lookup( 'log_dir' ), $log_file = params_lookup( 'log_file' ), $port = params_lookup( 'port' ), $protocol = params_lookup( 'protocol' ) ) inherits lighttpd::params {

Explore the module's logic 2/4less lighttpd/manifests/init.ppclass lighttpd ([...] ) inherits lighttpd::params {

$bool_source_dir_purge=any2bool($source_dir_purge) $bool_service_autorestart=any2bool($service_autorestart) $bool_absent=any2bool($absent) $bool_disable=any2bool($disable) $bool_disableboot=any2bool($disableboot) $bool_monitor=any2bool($monitor) [...]

### Definition of some variables used in the module $manage_package = $lighttpd::bool_absent ? { true => 'absent', false => $lighttpd::version, }

$manage_service_enable = $lighttpd::bool_disableboot ? { true => false, default => $lighttpd::bool_disable ? { true => false, default => $lighttpd::bool_absent ? { true => false, false => true, }, }, }

[...]

Explore the module's logic 3/4less lighttpd/manifests/init.ppclass lighttpd ([...] ) inherits lighttpd::params {

$manage_package = $lighttpd::bool_absent ? { true => 'absent', false => $lighttpd::version, }

$manage_service_enable = $lighttpd::bool_disableboot ? { true => false, default => $lighttpd::bool_disable ? { true => false, default => $lighttpd::bool_absent ? { true => false, false => true, }, }, } [...] $manage_file_source = $lighttpd::source ? { '' => undef, default => $lighttpd::source, }

$manage_file_content = $lighttpd::template ? { '' => undef, default => template($lighttpd::template), } [...]=>

<=[...] package { 'lighttpd': ensure => $lighttpd::manage_package, name => $lighttpd::package, }

service { 'lighttpd': ensure => $lighttpd::manage_service_ensure, name => $lighttpd::service, enable => $lighttpd::manage_service_enable, hasstatus => $lighttpd::service_status, pattern => $lighttpd::process, require => Package['lighttpd'], }

file { 'lighttpd.conf': ensure => $lighttpd::manage_file, path => $lighttpd::config_file, [...] notify => $lighttpd::manage_service_autorestart, source => $lighttpd::manage_file_source, content => $lighttpd::manage_file_content, replace => $lighttpd::manage_file_replace, }

Explore the module's logic 4/4less lighttpd/manifests/init.pp[...] # The whole lighttpd configuration directory can be recursively overriden if $lighttpd::source_dir { file { 'lighttpd.dir': ensure => directory, path => $lighttpd::config_dir, require => Package['lighttpd'], notify => $lighttpd::manage_service_autorestart, source => $lighttpd::source_dir, recurse => true, purge => $lighttpd::bool_source_dir_purge, replace => $lighttpd::manage_file_replace, audit => $lighttpd::manage_audit, } }

### Include custom class if $my_class is set if $lighttpd::my_class { include $lighttpd::my_class }

### Provide puppi data, if puppi is enabled if $lighttpd::bool_puppi == true { $classvars=get_class_args() puppi::ze { 'lighttpd': ensure => $lighttpd::manage_file, variables => $classvars, helper => $lighttpd::puppi_helper, } }

# Withclass { 'lighttpd': my_class => 'example42::my_lighttpd',}

# You can autoload example42/manifests/my_lighttpd.pp:class example42::my_lighttpd { # My extra custom resources}

Add a parameter to the classvi lighttpd/manifests/params.pp### Module specific parameters$use_ssl = false

vi lighttpd/manifests/init.pp# == Parameters# [*use_ssl*]# Set to true to activate ssl. # In order to use this option you need to use a template that honours it:# template => 'site/lighttpd/lighttpd.conf.erb',# (cp lighttpd/templates/lighttpd.conf.erb site/templates/lighttpd/lighttpd.conf.erb)[...]

class lighttpd ( $use_ssl = params_lookup( 'use_ssl' ), ) inherits lighttpd::params {[...]

$bool_use_ssl=any2bool($use_ssl)[...]

### Include ssl subclass (if it were of any use) # if $lighttpd::bool_use_ssl == true { # include $lighttpd::ssl # fi git status

#! modified: manifests/init.pp#! modified: manifests/params.pp

git add .git commit -m "Added (mostly useless) use_ssl parameter"

Create a sample templatevi lighttpd/templates/lighttpd.conf.erb# File Managed by Puppet## Sample /etc/lighttpd/lighttpd.conf based on Centos6 layout#var.log_root = "<%= scope.lookupvar('lighttpd::log_dir') %>"var.server_root = "<%= scope.lookupvar('lighttpd::data_dir') %>"var.conf_dir = "<%= scope.lookupvar('lighttpd::conf_dir') %>"[...]

<% if bool_use_ssl == true %> ssl.engine = "enable" ssl.pemfile = "/path/to/server.pem"<% end %>[...]

server.max-connections = <%= scope.function_options_lookup(['server.max-connections',‘1024’]) %>server.server.max-keep-alive-idle = <%= scope.function_options_lookup(['server.max-keep-alive-idle',‘5’]) %>server.max-request-size = <%= scope.function_options_lookup(['server.max-request-size',‘0’]) %>server.max-read-idle = <%= scope.function_options_lookup(['server.max-read-idle',‘60’]) %>server.max-write-idle = <%= scope.function_options_lookup(['server.max-write-idle',‘360’]) %>server.kbytes-per-second = <%= scope.function_options_lookup(['server.kbytes-per-second',‘128’]) %>server.connection.kbytes-per-second = <%= scope.function_options_lookup(['connection.kbytes-per-second',‘32’]) %>

git status#! modified: templates/lighttpd.conf.erb

git add .git commit -m "Added sample lighttpd.conf template (not used by default)"

Add a define 1/2vi lighttpd/manifests/dotconf.ppdefine lighttpd::dotconf ( $source = '' , $template = '' , $options = '', $ensure = present ) {

$manage_file_source = $source ? { '' => undef, default => $source, }

$manage_file_content = $template ? { '' => undef, default => template($template), }

file { "Lighttpd_$name.conf": ensure => $ensure, path => "${lighttpd::config_dir}/${name}.conf", mode => $lighttpd::config_file_mode, owner => $lighttpd::config_file_owner, group => $lighttpd::config_file_group, require => Package['lighttpd'], notify => $lighttpd::manage_service_autorestart, source => $manage_file_source, content => $manage_file_content, audit => $lighttpd::manage_audit, }

}git status#! modified: manifests/dotconf.pp

Add rspec test for a definevi spec/defines/lighttpd_dotconf_spec.rbrequire "#{File.join(File.dirname(__FILE__),'..','spec_helper.rb')}"describe 'lighttpd::dotconf' do let(:title) { 'lighttpd::dotconf' } let(:node) { 'rspec.example42.com' } let(:facts) { { :arch => 'i386' , :operatingsystem => 'redhat' } } let(:params) { { 'ensure' => 'present', 'name' => 'www.example42.com', 'source' => 'puppet:///modules/site/lighttpd/www.example42.com.conf', } } describe 'Test lighttpd::dotconf' do it 'should create a lighttpd::dotconf' do should contain_file('Lighttpd_www.example42.com.conf').with_ensure('present') end end describe 'Test lighttpd::dotconf source parameter' do it 'should create a lighttpd::dotconf' do content = catalogue.resource('file', 'Lighttpd_www.example42.com.conf').send(:parameters)[:source] content.should == "puppet:///modules/site/lighttpd/www.example42.com.conf" end end describe 'Test lighttpd::virtualhost decommissioning' do let(:facts) { { :arch => 'i386' , :operatingsystem => 'ubuntu' } } let(:params) { { 'ensure' => 'absent' } } it 'should remove a lighttpd::dotconf file with ensure => absent' do should contain_file('Lighttpd_www.example42.com.conf').with_ensure('absent') end end

git status#! modified: spec/defines/lighttpd_dotconf_spec.rb

git add .git commit -m "Added lighttpd::dotconf spec tests"

Run Testscd lighttpd;./Example42-tools/check-module.sh############################### Executing rake tasks ###############################/usr/local/rvm/rubies/ruby-1.8.7-head/bin/ruby -S rspec spec/classes/lighttpd_spec.rb --format doc --colorRun options: exclude {:broken=>true}lighttpd Test standard installation should contain Package[lighttpd] with ensure => "present" should contain Service[lighttpd] with ensure => "running" should contain Service[lighttpd] with enable => "true" should contain File[lighttpd.conf] with ensure => "present" Test installation of a specific version should contain Package[lighttpd] with ensure => "1.0.42" Test standard installation with monitoring and firewalling should contain Package[lighttpd] with ensure => "present" should contain Service[lighttpd] with ensure => "running" should contain Service[lighttpd] with enable => "true" should contain File[lighttpd.conf] with ensure => "present" should monitor the process should place a firewall rule[...]

############################### Executing puppetlint ################################## ./manifests/init.ppWARNING: line has more than 80 characters on line 108WARNING: line has more than 80 characters on line 437### ./manifests/params.pp### ./manifests/spec.pp

Job DoneModule is ready to use: node test { class { 'lighttpd': # Custom configuration provided as a template created in: # MODULEPATH/site/templates/lighttpd/lighttpd.conf.erb template => 'site/lighttpd/lighttpd.conf.erb' # Custom options used in template: options => { 'server.kbytes-per-second' => 254, 'server.max-keep-alive-idle' => 3, } }}

git remote add origin https://github.com/example42/puppet-lighttpd.git

git push -u origin master

Was it too fast? Get the module:

http://github.com/example42/ puppet-lighttpd.git

Review these slides:

http://slideshare.net/alvagante

Follow Up on Twitter:

@alvagante