Download - Using Puppet in Small Infrastructures
Puppet & Small Infrastructures
Rachel Andrew
@rachelandrew
edgeofmyseat.com
grabaperch.com
Why would a small business use Puppet?
• My background
• Learning Puppet and initial challenges
• Our current use of Puppet
• Why Puppet for small businesses with a handful of servers?
This is my job.
• writer
• tech support person
• bookkeeper
• HR
• filler in of baffling forms from the government
• PHP developer
• front-end web developer
• marketer
• sales person
• public speaker
• … ops person.
Back in my day …
Pre-Puppet
• Infrastructure consisted of a bunch of VPS boxes hosted at Memset
• Configured at different times
• Some set up by me, some by Drew
• Neither of us understood the setups done by the other
• No real handle on what was installed where
Initial setup would be documented but configuration would drift over time as we updated, installed and
fixed things.
“If it ain’t broke, don’t fix it”
Getting Started with Puppet
Puppet or Chef?
https://docs.puppetlabs.com/learning/
https://puppetlabs.com/learn
http://puppetlabs.com/blog/get-more-agile-learn-how-to-automate-one-small-thing-with-puppet-enterprise
“By starting small and getting good at automating one discrete task, you can establish a foundation for bigger automation projects.”
Ideas for small tasks
• cron jobs
• users
• ssh keys
• vhosts
• specific config files - for example a common php.ini
• packages or settings you configure on all servers as standard
Installing packages
package { "sudo": ensure => "installed" }
Using Puppet to create cron jobs.
cron {‘my_cron_job’: command => "php /home/sites/mysite/public_html/perch/core/scheduled/run.php secret", user => root, minute => [1,31], }
Adding standard files.
file {'/etc/php5/apache2/php.ini': ensure => file, source => 'puppet:///modules/hosting/php.ini', notify => Service["apache2"], }
Don’t wait until you have time to rebuild everything. Who ever has
time to rebuild everything?
Not Invented Here.
Is there an existing, well supported module that does this job?
https://forge.puppetlabs.com/supported
Managing Third Party Modules
Dependencies will bite you.
http://garylarizza.com/blog/2014/10/19/on-dependencies-and-order/
“Puppet describes the end-state of the machine, and NOT the order that it’s (Puppet) going to take you to that state”
Where we are now.
• A Puppet Master, PuppetDB is on the same box
• Three webservers
• The “demo server”, also a webserver but of interesting configuration
• PuppetBoard and Scout to see what is happening in Puppet and for monitoring
Webservers
• Puppetlabs Apache, MySQL
• modules/hosting = a module I’ve written than wraps up standard things used on webservers
• make use of hiera for site, database and user values
Discovering Hiera made Puppet make sense to me.
A common.yaml file holds information common to all servers. For example user accounts.
--- users: rachel: comment: "Rachel Andrew" shell: "/bin/bash" home: "/home/rachel" managehome: "true" groups: ['admin','www-admin'] drew: comment: "Drew McLellan" shell: "/bin/bash" home: "/home/drew" managehome: "true" groups: ['admin','www-admin'] ssh_keys: rachel_ssh: user: "rachel" type: "rsa" key: "AAAABB[...]" drew_ssh: user: "drew" type: "rsa" key: "AAAABB[...]"
Information specific to one server is held in node specific YAML files.
eg: vhosts and MySQL databases.
--- apache_vhosts: example.co.uk: port: '8080' docroot: '/home/sites/example/public_html' docroot_group: 'www-admin' servername: 'example.co.uk' serveraliases: ['example.com'] test.co.uk: port: '8080' docroot: '/home/sites/test/public_html' docroot_group: 'www-admin' servername: 'test.co.uk' serveraliases: ['test.com']
mysql_db: db_a: user: 'user_a' password: 'xxxxx' grant: ['all'] db_b: user: 'user_b' password: 'xxxxx' grant: ['all']
The hiera.yaml file.
--- :backends: - yaml
:logger: console :yaml: :datadir: /etc/puppet/hiera
:hierarchy: - "%{::fqdn}" - common
hiera_hash gives an array of users, hosts and databases from the node specific YAML.
I can use that in create_resources within manifests.
$sites = hiera_hash('apache_vhosts')
create_resources('apache::vhost',$sites)
$db = hiera_hash('mysql_db')
create_resources('mysql::db',$db)
http://garylarizza.com/blog/2014/10/24/puppet-workflows-4-using-hiera-in-anger/
“When you come up with a solution using create_resources(), I challenge you to draw up another solution using Puppet code in a Puppet manifest”
Hiera and the demo server.
Standard CMS demos allow everyone access to one install
which is “refreshed” periodically.
We wanted to give everyone a clean demo all of their own.
Hiera can have multiple backends defined.
Hiera can use json as well as YAML.
--- :backends: - yaml - json
:logger: console :yaml: :datadir: /etc/puppet/hiera :json: :datadir: /etc/puppet/hiera
:hierarchy: - '%{fqdn}' - common
deploy.pp
• create a home directory
• grab the site files tarball and untar into the home directory
• get the relevant SQL dump
• grab the config file and replace out db details
• create a database using the import file
• create a vhost
• execute a script to notify Air Traffic Control the site is ready
• json Hiera backend is the source of truth for Puppet as to what sites should be running
• could deploy to multiple servers by writing multiple json files one for each node
• can deploy different versions of Perch - for example to allow someone to try out a beta
• currently deploying and tearing down 50 or 60 sites per day. It just works.
Start small with Puppet, but be aware of non-obvious problems
that Puppet can help solve.
I use Vagrant and Puppet to test and build the site packages locally.
Why should small business and small infrastructures consider
Puppet?
Disaster Recovery
Small companies
• often don’t need hugely redundant infrastructures
• having sites offline for a few hours not critical
• … as long as everything can be restored.
Before Puppet
• Rebuilding our infrastructure would have involved us “trying to remember” what went where.
• Just getting servers reinstalled would have taken a long time.
• Then we would have had to reconfigure every site, every SSH key, one at a time.
With Puppet
• Configuration for each server is held in code, and in an external git repo
• Checkout the modules onto a new Puppet Master
• Spin up new servers and run Puppet which would create all resources - sites, keys etc.
• We could then import any data such as MySQL backups
A good test - can you restore any of your servers into a local VM?
How do we do that thing again?
Puppet allows us to document processes by way of manifests.
The git commit history gives me additional information as to why
something is configured that way.
Please look after this server.
Get an expert up to speed quickly
Ensure knowledge isn’t lost when someone leaves the company
Small businesses are often far more exposed than large ones to
losing knowledge when a key person leaves.
Easier audits and compliance
http://blog.bluemalkin.net/pci-compliance-tips-for-sys-admins/
“It is generally acceptable to show the Puppet modules to the auditor to demonstrate what settings are applied to the PCI servers.”
Speed of setting up new servers
Puppet means I don’t need to spend time and energy remembering how
to do things on our servers.
Moving hosting or to new servers within a hosting company
Getting “stuck” on terrible hosting is a real issue for small businesses
Being Puppetized makes moving the entire infrastructure seem far
less scary.
Modules from the Forge
Modules show best practice ways of achieving tasks.
The Puppet Community
https://docs.puppetlabs.com/community/community_guidelines.html
“We like nice people way better than mean ones!”
Thank you
http://rachelandrew.co.uk/presentations/puppet
@rachelandrew