config managament for development environments iii

gareth rushgrove | morethanseven.net

Configuration Managementfor Development Environments

PuppetConf 22nd September 2011

http://www.flickr.com/photos/36144637@N00/159627088/



Gareth Rushgrove


Work at UK Government Digital Service


Blog at morethanseven.net


Curate devopsweekly.com


http://www.flickr.com/photos/iancarroll/5027441664gareth rushgrove | morethanseven.net

Problems

http://www.flickr.com/photos/iancarroll/5027441664

http://www.flickr.com/photos/iancarroll/5027441664

http://www.flickr.com/photos/34652102@N04/5059217055gareth rushgrove | morethanseven.net

1. Not all developers want to be sysadmins

http://www.flickr.com/photos/34652102@N04/5059217055



1. Sysadmins don’t want devs to be sysadmins




2. New team members getting started time



http://www.flickr.com/photos/biggreymare

3. Running a full set of services locally




4. Works on my machine


⚡ brew info mysqlmysql 5.5.14

$ aptitude show mysql-serverPackage: mysql-serverState: not installedVersion: 5.1.41-3ubuntu12.10

Homebrew is great but...


What’s a few versions between friends?


- An ORDER BY clause was bound to the incorrect substatement when used in UNION context.

- A NOT IN predicate with a subquery containing a HAVING clause could retrieve too many rows, when the subquery itself returned NULL.

- MIN(year_col) could return an incorrect result in some cases.

23 releases and 21 months in-between 5.1.41 and 5.5.14. Here’s some fixed bugs:

And lots more

Spot the cross platform bug (not the security flaw)


On our Mac


⚡ ./server.rb &⚡ curl "http://127.0.0.1:8181/?query=Bob"⚡ curl "http://127.0.0.1:8181/?query=bob"⚡ ls

http://127.0.0.1:8181/?query=Bob

http://127.0.0.1:8181/?query=Bob

http://127.0.0.1:8181/?query=bob

http://127.0.0.1:8181/?query=bob

⚡ ./server.rb &⚡ curl "http://127.0.0.1:8181/?query=Bob"⚡ curl "http://127.0.0.1:8181/?query=bob"⚡ lsBob⚡ cat BobHello bob

On our Mac


http://127.0.0.1:8181/?query=Bob

http://127.0.0.1:8181/?query=Bob

http://127.0.0.1:8181/?query=bob

http://127.0.0.1:8181/?query=bob

On Linux


$ ./server.rb &$ curl "http://127.0.0.1:8181/?query=Bob"$ curl "http://127.0.0.1:8181/?query=bob"$ ls

http://127.0.0.1:8181/?query=Bob

http://127.0.0.1:8181/?query=Bob

http://127.0.0.1:8181/?query=bob

http://127.0.0.1:8181/?query=bob

$ ./server.rb &$ curl "http://127.0.0.1:8181/?query=Bob"$ curl "http://127.0.0.1:8181/?query=bob"$ lsBob bob$ cat BobHello Bob$ cat bobHello bob

On Linux


http://127.0.0.1:8181/?query=Bob

http://127.0.0.1:8181/?query=Bob

http://127.0.0.1:8181/?query=bob

http://127.0.0.1:8181/?query=bob

gareth rushgrove | morethanseven.net http://www.flickr.com/photos/34652102@N04/5059208501

Solutions



http://www.flickr.com/photos/dawilson/2598713027

Virtualisation




VirtualBox


VMware


Virtualisation needs powerful hardware

gareth rushgrove | morethanseven.net http://www.flickr.com/photos/martinoc/477335951

http://flickr.com/photos/psd/102332391/


gareth rushgrove | morethanseven.net http://www.flickr.com/photos/peteradams/2272928740

What about editing code?

http://www.flickr.com/photos/peteradams/2272928740

http://www.flickr.com/photos/peteradams/2272928740


Shared Folders or NFS




Doubledown

Vim


Vagrantup.com


What is Vagrant?


- Automated virtual machine creation using Oracle’s VirtualBox- Automated provisioning of virtual environments using Chef or Puppet- Full SSH access to created environments- Assign a static IP to your VM, accessible from your machine- Forward ports to the host machine- Shared folders allows you to continue using your own editor- Package environments into distributable boxes- Completely tear down environment when you’re done- Easily rebuild a complete environment with a single command


Base boxes




VeeWee



Community boxes

⚡ gem install vagrant⚡ vagrant box add lucid32 http://.../lucid32.box⚡ vagrant init⚡ vagrant up

Vagrant up


http://files.vagrantup.com/lucid32.box

http://files.vagrantup.com/lucid32.box

⚡ lsVagrantfile⚡ vagrant up⚡ vagrant ssh⚡ vagrant reload⚡ vagrant halt⚡ vagrant destroy

Vagrant command line


⚡ vagrant ssh-configHost default HostName 127.0.0.1 User vagrant Port 2222 IdentityFile /Users/.../vagrant-0.8.2/keys/vagrant ...

Export SSH configuration


Vagrant::Config.run do |config| config.vm.box = "lucid32"end

Vagrantfile


Vagrant::Config.run do |config| config.vm.forward_port("web", 80, 8080) config.vm.forward_port("ftp", 21, 4567) config.vm.forward_port("ssh", 22, 2222, :auto => true)end

Port forwarding


Vagrant::Config.run do |config| config.vm.share_folder("folder", "/guest", "../host")end

Shared folders


Vagrant::Config.run do |config| config.vm.boot_mode = :gui config.ssh.forward_agent = true config.vm.network("33.33.33.10") config.vm.customize do |vm| vm.memory_size = 512 endend

Lots more options


Puppet


Vagrant provisioning with Puppet


Vagrant::Config.run do |config| config.vm.provision :puppet do |puppet| puppet.manifests_path = "puppetmanifests" puppet.manifest_file = "newbox.pp" endend

Vagrant provisioning with Puppetmaster


Vagrant::Config.run do |config| config.vm.provision :puppet_server do |puppet| puppet.puppet_server = "puppet.example.com" puppet.puppet_node = "vm.example.com" endend

Chef


Vagrant provisioning with Chef


Vagrant::Config.run do |config| config.vm.provision :chef_solo do |chef| chef.add_recipe = "garethr" chef.cookbooks_path = “cookbooks” endend

Specifying Chef roles


Vagrant::Config.run do |config| config.vm.provision :chef_solo do |chef| chef.roles_path = "roles" chef.add_role("vm") endend

Vagrant::Config.run do |config| config.vm.provision :chef_solo do |chef| chef.recipe_url = "http://github.com/cookbooks.tar.gz" chef.add_recipe "garethr" chef.cookbooks_path = [:vm, "cookbooks"] chef.json.merge!({ :garethr => { :ohmyzsh => "https://github.com/.../oh-my-zsh.git", :dotvim => "https://github.com/garethr/dotvim.git" }}) endend


Remote file

http://cloud.github.com/downloads/garethr/chef-repo/cookbooks.tar.gz

http://cloud.github.com/downloads/garethr/chef-repo/cookbooks.tar.gz

https://github.com/garethr/oh-my-zsh.git

https://github.com/garethr/oh-my-zsh.git

https://github.com/garethr/dotvim.git

https://github.com/garethr/dotvim.git

http://www.flickr.com/photos/s3a/4710416678

Plugins


- Vagrant Hosts - https://github.com/dwt/vagrant-hosts- Sahara - https://github.com/jedi4ever/sahara- Vagrantboxes - https://github.com/garethr/ruby-vagrantboxes



⚡ vagrant provision[default] Running provisioner: Vagrant::Provisioners::Puppet...[default] Running Puppet with base.pp...[default] notice: /Stage[main]//File[/etc/motd]/content: content changed '{md5}a10cc0046a5fad11470513e5f5df9d91' to '{md5}9e5e449fc643d3e88a2cefeb1af7bc2e'[default] [default] notice: /Stage[main]//File[/etc/motd]/mode: mode changed '777' to '644'

Useful for testing puppet modules


http://www.flickr.com/photos/crustyscumbrothersontour/2674351601

Useful for local configuration management





Real world example

gareth rushgrove | morethanseven.netgareth rushgrove | morethanseven.net

- I want my development environment on my local vms- I don’t want a wiki page of instructions- I don’t want to have to manually install anything- I don’t want to care about destroying a virtual machine

⚡ tree├── Vagrantfile└── modules └── garethr ├── manifests │ └── init.pp ├── spec │ └── classes │ └── base.rv │ └── janus.rv │ └── ohmyzsh.rv └────── spec_helper.rb

Puppet layout


Packages I like


class base { $packages = ["zsh", "wget", "curl", "lynx", "git-core", "dvtm", “tree”, "build-essential", "vim-nox"] package { $packages: ensure => "installed" }}

My Zsh configs


$repo = "git://github.com/robbyrussell/oh-my-zsh.git"exec { "ohmyzsh": command => "git clone ${repo} .oh-my-zsh", cwd => "/home/vagrant", creates => "/home/vagrant/.oh-my-zsh", require => Class["base"],}exec { "zshrc": command => "cp .... /home/vagrant/.zshrc", creates => "/home/vagrant/.zshrc", require => Exec["ohmyzsh"],}

My Vim configs


$repo = "git://github.com/carlhuda/janus.git"exec { "get_janus": command => "git clone ${repo} .vim", cwd => "/home/vagrant", creates => "/home/vagrant/.vim", require => Class["base"],}exec { "compile_janus": command => "rake", creates => "/home/vagrant/.zshrc", require => Exec["ohmyzsh"], environment => "HOME=/home/vagrant",}

Testing with Rspec


require 'spec_helper'

describe 'ohmyzsh', :type => :class do it { should create_exec("ohmyzsh") } it { should create_exec("zshrc") } it { should create_class("base") }end

Rspec results


base should create Package[zsh] should create Package[vim-nox] should create Package[git-core]

ohmyzsh should create Exec[ohmyzsh] should create Exec[zshrc] should create Class[base]

Finished in 1.4 seconds10 examples, 0 failures


Conclusions

gareth rushgrove | morethanseven.netgareth rushgrove | morethanseven.net

- Using Virtualisation makes getting started fast- Running the same platform catches bugs early- Using Vagrant makes managing virtual machines easy- Writing configuration as code makes it testable

More information on Vagrant


- IRC - #vagrant on Freenode- Github Issues - https://github.com/mitchellh/vagrant/issues- Google Groups - http://groups.google.com/group/vagrant-up

https://github.com/mitchellh/vagrant/issues

https://github.com/mitchellh/vagrant/issues

http://groups.google.com/group/vagrant-up

http://groups.google.com/group/vagrant-up

Questions?

gareth rushgrove | morethanseven.net http://flickr.com/photos/psd/102332391/



config managament for development environments iii

Technology

macgareth rushgrove

sysadminsgareth rushgrove

comgareth rushgrove

nfsgareth rushgrove

netgareth rushgrove

machinegareth rushgrove

problemsgareth rushgrove

solutionsgareth rushgrove