create disposable test environments with vagrant and puppet

Creating Disposable Test Environments with Vagrant and Puppet

Gene Gotimer, Senior Architect

gene.gotimer@coveros.com

Coveros helps organizations accelerate the delivery of business value through secure, reliable software

About Coveros

Why Disposable Test Environments?

Destructive testing

Known baseline

Available on-demand

Not shared

No vested interest in keeping them long-term

Always up-to-date

Tools Involved

VirtualBox– virtualization software

Vagrant– virtualization automation

Puppet– configuration management and automation– Chef, Ansible, or SaltStack would work equally well

Packer– machine image automation

VirtualBox

Oracle VM VirtualBox

Virtualization software from Oracle

Runs on Windows, Mac, Linux

Runs as an application

Allows us to use local VMs

Easy to install

Works well with Vagrant

https://www.virtualbox.org/

Vagrant

Virtualization workflow software from HashiCorp

Free, open-source

Easy to install

Works well with Puppet, Chef, Shell– many other provisioners

Works well with VirtualBox, VMware, Amazon Web Services– many other providers

https://www.vagrantup.com/

Creating a Vagrant Box

To create a VM:– mkdir starcanada‐vagrant– cd starcanada‐vagrant– vagrant box add hashicorp/precise64– vagrant init hashicorp/precise64– vagrant up

vagrant box add– downloads a “base box”– boxes at https://atlas.hashicorp.com/search

vagrant init– builds a Vagrantfile with the base box

vagrant up– starts the VM

Vagrantfile

Vagrantfile– lots of comments by default– stock Vagrantfile without comments is:

Vagrant.configure(2) do |config|config.vm.box = "hashicorp/precise64"

vagrant up

vagrant up– imports the base box to VirtualBox– makes sure the base box is up to date– sets a unique name for the VM– sets up networking (just NAT by default)– sets up port forwarding (just SSH by default)– boots VM– replaces known, insecure SSH key with a new random key– makes sure VirtualBox Guest Additions are installed– mounts shared folders (/vagrant by default on the VM)– provisions software (nothing by default)

Access Vagrant Box

To access a VM:– vagrant ssh

vagrant ssh– connects to the VM via the forwarded SSH port

requires an SSH client installed– Git (https://msysgit.github.io/)– openssh on Cygwin (http://www.cygwin.com/)– PuTTY (http://www.chiark.greenend.org.uk/~sgtatham/putty/)

requires converting the key format

Rebuild Vagrant Box

To rebuild a VM:– vagrant destroy– vagrant up

vagrant destroy– deletes a VM

vagrant up– starts the VM

Puppet

Configuration management software from PuppetLabs

Vaguely Ruby-based, domain-specific language

Free, open-source

Easy to install

Works well with Vagrant

Similar to Chef, Ansible, SaltStack

https://puppetlabs.com/

Install Apache with Puppet

Modify the Vagrantfile:Vagrant.configure(2) do |config|

config.vm.box = "hashicorp/precise64"config.vm.network "private_network", ip: "192.168.33.10"config.puppet_install.puppet_version = '3.8.1'config.vm.provision "shell", inline: <<‐SHELLsudo puppet module install puppetlabs‐apache

SHELLconfig.vm.provision "puppet" do |puppet|puppet.manifests_path = "manifests"puppet.manifest_file = "site.pp"puppet.module_path = "modules"

endend

Vagrant Networking

config.vm.network "private_network", ip: "192.168.33.10"– sets up a new network interface on the box– private_network = host-only

only this box and other VMs on this box can reach it

Vagrant Modules

config.puppet_install.puppet_version = '3.8.1'– Vagrant module from

https://github.com/mitchellh/vagrant/wiki/Available-Vagrant-Plugins– vagrant‐puppet‐install

installs Puppet version 3.8.1 could have been :latest, but I want control

Shell Provisioning

config.vm.provision "shell", inline: <<‐SHELLsudo puppet module install puppetlabs‐apacheSHELL

– here-doc that runs all the commands until SHELL– this command installs a Puppet module from

https://forge.puppetlabs.com/puppetlabs

Puppet Provisioning

config.vm.provision "puppet" do |puppet|puppet.manifests_path = "manifests"puppet.manifest_file = "site.pp"puppet.module_path = "modules"end

– sets up a standard Puppet layout– commands in manifests/site.pp– reusable modules in modules

Example Puppet Code

Example init.pp file in the modules/website/manifests directory:

class website {class { 'apache': }apache::vhost { "${::fqdn}":vhost_name => '*',default_vhost => true,port => '80',docroot => '/var/www',

}file { '/var/www/index.html':ensure => 'file',content => template('website/index.html.erb'),owner => 'root',group => 'www‐data',mode => '0640',require => Class['apache'],

Installing Apache httpd

class { 'apache:' } – installs Apache httpd server– sets up default configuration

Configuring Apache httpd

apache::vhost { "${::fqdn}":vhost_name => '*',default_vhost => true,port => '80',docroot => '/var/www',

}– sets up default virtual host– listening on port 80– document root is /var/www

Installing Templated Content

file { '/var/www/index.html':ensure => 'file',content => template('website/index.html.erb'),owner => 'root',group => 'www‐data',mode => '0640',require => Class['apache'],

}– copies file from host box– sets owner, group, and permissions

Automation Advantages

Deploy is now automated

Automated = repeatable, easy, quick

Test on the system, make any changes we want, then destroy it, recreate it in a pristine condition

Reuse the deployment scripts in all environments– including production– especially production

Other Possibilities

Template files

Variable substitution/Configuration database– YAML– JSON– Encrypted

Multiple machines

Different providers– Managed– VMware– Amazon Web Services (AWS)

Chef, Ansible, or SaltStack

Packer

Machine image automation from HashiCorp

Free, open-source

Easy to install

Works well with Puppet, Chef, Shell– many other provisioners

Works well with VirtualBox, VMware, Amazon Web Services– many other providers

https://packer.io/

Packer Templates

Packer templates on GitHub from Shiguredo, Inc.

Templates for– CentOS Linux 6.4, 6.5, 6.6, 7.0, 7.1– Scientific Linux 6.4, 6.5, 7.0– Ubuntu Linux 12.04, 14.04

Fork and edit to create you own base boxes

https://github.com/shiguredo/packer-templates

Wrap-Up

Tools Recap

VirtualBox– virtualization software– https://www.virtualbox.org/

Vagrant– virtualization automation– https://www.vagrantup.com/– Boxes: https://atlas.hashicorp.com/search– Plugins:

https://github.com/mitchellh/vagrant/wiki/Available-Vagrant-Plugins

Tools Recap

Puppet– configuration management and automation– https://puppetlabs.com/– Modules: https://forge.puppetlabs.com/puppetlabs

Packer– machine image automation– https://packer.io/– Templates: https://github.com/shiguredo/packer-

templates

Questions?

Gene Gotimergene.gotimer@coveros.comhttp://www.coveros.com@CoverosGene

create disposable test environments with vagrant and puppet

Software

position available immediately: devops engineer devops...

vagrant story

oscar: rapid iteration with vagrant and puppet enterprise -...

harmonious development: standardizing the deployment process...

itb2015 - winning with vagrant, puppet and chef

spdk vagrant box install instructions ·...

vagrant and puppet

infrastructure testing with jenkins, puppet and vagrant -...

devops - yo briefca · 5 x amazon ec2 small instances...

environments line-up! vagrant & puppet 101

topics covered : summary · topics covered : summary ......

provisioning with vagrant & puppet

new batches info - quality thought · puppet docker linux...

getting started with puppet and vagrant (1)

reproducibility through environment capture: part 1:...

qcon sp - shorttalk - virtualização e provisionamento de...

vagrant - pugmi

einführung in puppet und vagrant

better local qa testing with vagrant and puppet

vagrant puppet