puppet camp chicago 2014: docker and puppet: 1+1=3 (intermediate)

Dockerand

Puppet

1+1=3

@jpetazzo

● Wrote dotCloud PAAS deployment tools

– EC2, LXC, Puppet, Python, Shell, ØMQ...● Docker contributor

– Docker-in-Docker, VPN-in-Docker,router-in-Docker...CONTAINERIZE ALL THE THINGS!

● Runs Docker in production,and helps others to do the same

What is Docker?The quick elevator pitch

Docker Engine + Docker Hub

= Docker Platform

Docker Engine

The Docker Engine

● Open Source● Written in Go● Runs containers● On any modern Linux machine

(Intel 64 bits for now)

Containers ?

Containers

● Software delivery mechanism(a bit like a package!)

● Put your application in a container,run it anywhere

● A bit like a VM, but ...

I have four words for you

● CONTAINERS boot faster(than VMs)

● CONTAINERS have less overhead(more consolidation)

● CONTAINERS bring native performance(on bare metal)

● CONTAINERS are cloud-compatible(can run in VMs)

Docker Engine recap

● Approximation:it's an hypervisor to run containers

● Approximation:containers are like VMs, but lighter

● Docker makes containers available to everybody(not just veterans from the last emacs/vim war)

DockerHub

Docker Hub

● Services operated by Docker Inc.● Library of ready-to-use container images● Registry for your container images

(public or private)● Automated builds

(triggered by pushes to GitHub/Bitbucket)● Free for public/open source code, $$ otherwise

Buildingcontainers

Dockerfile

FROM ubuntu:14.04MAINTAINER Docker Team <education@docker.com>

RUN apt-get updateRUN apt-get install -y nginxRUN echo 'Hi, I am in your container' \ >/usr/share/nginx/html/index.html

CMD [ "nginx", "-g", "daemon off;" ]

EXPOSE 80

FROM ubuntu

RUN apt-get -y updateRUN apt-get install -y g++RUN apt-get install -y erlang-dev erlang-manpages erlang-base-hipe ...RUN apt-get install -y libmozjs185-dev libicu-dev libtool ...RUN apt-get install -y make wget

RUN wget http://.../apache-couchdb-1.3.1.tar.gz | tar -C /tmp -zxf-RUN cd /tmp/apache-couchdb-* && ./configure && make install

RUN printf "[httpd]\nport = 8101\nbind_address = 0.0.0.0" > /usr/local/etc/couchdb/local.d/docker.ini

EXPOSE 8101CMD ["/usr/local/bin/couchdb"]

docker build -t jpetazzo/couchdb .

Dockerfilevs.

Shell scripts

Shell scripts

● OK-ish for simple stacks● Tricky to handle all possible situations

(that's why we have proper config management)● Though choice when rebuilding:

– from scratch (but it takes forever!)

– iteratively (but might behave differently!)

Dockerfilevs.

Configuration Management

Configuration Management:the Good

● Deals with low-level stuff● Abstracts some details (distro, sometimes OS)● Ensures convergence to a known state● Library of reusable, composable templates

Configuration Management:the Bad

● Steep learning curve● Generally requires an agent

(or something to trigger e.g. « puppet apply »)● Resource-intensive

(it's OK to run the agent on a 64 GB server,it's less OK to run 100 agents on said server)

Configuration Management

● Reusability is just as good as modules are(i.e. YMMV)

● Not as deterministic as you think● Rollbacks are harder than you think

{ 'openssl' : ensure => present }

{ 'openssl' : ensure => '1.2.3-no-heartbleed-pls' }

Dockerfileto the rescue

Dockerfile

● Doesn't have to deal with « low-level stuff »(hardware, drivers... handled by the host)

● Doesn't need all the goodness of CM(because it doesn't have to converge)

● Partial rebuilds are fast(layered caching rebuilds only what is needed)

● Allows inheritance and composition(FROM <mycustombase>; see also: ONBUILD)

● Easy learning curve(if you know Shell, you already know Dockerfile)

But...

● Doesn't deal with « low-level stuff »(hardware, drivers...)

● Doesn't define resource dependencies(no before/after)

● Doesn't define what runs where

Puppetto the rescue

Before/After

● Use Puppet tosetup hardware(or virtual hardware), install packages, deploy code,run services.

● Use Puppet tosetup hardware(or virtual hardware), install Docker,run containers.

● Use Dockerfilesto install packages,deploy code,run services.

Do one thing,and do it well

First things first

https://github.com/garethr/garethr-docker

https://forge.puppetlabs.com/garethr/docker

Installing Docker with Puppet

include 'docker'

class { 'docker': version => '0.8.1'}

Warm up our image collection

# download the registry imagedocker::image { 'stackbrew/registry':}

# don't download all ubuntu,# just 'precise'docker::image { 'ubuntu': image_tag => 'precise'}

Run containers

docker::run { 'slavedb': image => 'jpetazzo/postgresql' command => '…' ports => ['5432', '22'], links => ['masterdb:master'], use_name => true, volumes => ['/var/lib/postgresql'], volumes_from => '420fc7e8aa20', memory_limit => 100000000, # bytes username => 'postgres', hostname => 'sdb.prod.dckr.io', env => ['FUZZINESS=42', FOO=BAR', 'FOO2=BAR2'], dns => ['8.8.8.8', '8.8.4.4'], restart_service => true

}

Can I use Puppet to build Docker

container images?

YES

Should I use Puppet to build Docker

container images?

NO

OK,let's do it anyway

My other VM is a container

● write a Dockerfile to install Puppet● start tons of containers● run Puppet in them (agent, or one-shot apply)

Good if you want a mix of containers/VM/metal

But slower to deploy, and uses more resources

FROM ubuntu:12.04RUN apt-get install -qy wgetRUN mkdir /puppetWORKDIR /puppetRUN wget -q http://apt.puppetlabs.com/puppetlabs-release-precise.debRUN dpkg -i puppetlabs-release-precise.debRUN apt-get update -qRUN apt-get install -qy puppet-commonCMD puppet agent --no-daemonize --verbose

Sample Dockerfile

Lightweight, portable VMs

● Start containers instead of VMs– I can start 10 containers on this puny laptop!

– You can start those 10 containers too!(Even though you have a totally different laptop!)

– We can start those containers in the Cloud!

● Deploy sshd, syslogd, crond, etc.– You can... But do you have to?

The revolution will be containerized

● write a Dockerfile to install Puppet● … and run Puppet as part of build process● deploy fully baked, « golden » images

Faster to deploy

Easier to rollback

FROM ubuntu:12.04RUN apt-get install -qy wgetRUN mkdir /puppetWORKDIR /puppetRUN wget -q http://apt.puppetlabs.com/puppetlabs-release-precise.debRUN dpkg -i puppetlabs-release-precise.debRUN apt-get update -qRUN apt-get install -qy puppet-commonENV FACTER_HOSTNAME database42ADD ./site.pp /puppet/site.ppRUN puppet apply site.pp

Sample Dockerfile

Beyond Golden

Containers

Get rid of sshd, crond, syslogd...

● Remote access: nsenterhttps://github.com/jpetazzo/nsenter

● Cron:use a separate container

● Logs:use a data container

http://blog.docker.com/2014/06/why-you-dont-need-to-run-sshd-in-docker/

Why?

● Separate orthogonal concerns(don't rebuild your app to change logging, remote access, and other unrelated things)

● Have different policies in prod/dev/QA/etc● Ship lighter containers

Thoughts...

What if we could...

● Run the Puppet agent outside of the container● Run a single agent for many containers● Share the cost of the agent

Thank you!

Shameless promo + Q&A

Tonight:Docker and Mesos meet-up, at BrainTree(requires cloning+teleportation)

The rest of the week:A bunch of talks about Docker & Containers(requires a LinuxCon pass)

http://docker.com/@docker@jpetazzo