drupalcon prague 2013: automate drupal deployments with linux containers, docker and vagrant

Automate Drupal deployments with Linux Containers, Vagrant and Docker

An overview of deployment strategies@ricardoamaro

Free/Opensource software loverSenior Cloud Engineer @AcquiaDrupal.org infrastructure/devopsDrupalist & Linux enthusiast

Father, artist, community facilitator

@ricardoamaro

About me

Vicente e Dália

About us

1. The sad VirtualMachine story

2. Containers and non-containers

3. Drupal on LXC

4. How to Puppetize a container

5. Docker & LXC

6. Shipping containers with Drupal

today’s agenda

Hardware virtualization or platform virtualization refers to the creation of a virtual machine that acts like a real computer with an operating system.

Software executed on these virtual machines is separated from the underlying hardware resources.

What is virtualization?

Cloud infrastructure providers like Amazon Web Service sell virtual machines. EC2 revenue is expected to surpass $1B in revenue this year. That's a lot of VMs…

Why should i care?Increase

+ efficiency+ availability+ security

Reduce

- costs- hardware- energy

Virtual Machine platforms

➢ We are also paying for lot of avoidable overhead.

➢ The Virtual Machine is a full-blown operating system image.

➢ This is a heavyweight solution to run applications in the cloud.

The sad Virtual Machine story...

What is the solution?

Containers used to be terrible, but not anymoreContainers used to be terrible, but not anymore

A new concept, a new hope

Because LXC is ready to roll!

On any recent Linux Kernel near you!

Source : http://www.linuxjournal.com/content/containers%E2%80%94not-virtual-machines%E2%80%94are-future-cloud

Virtual Machines vs Containers

Virtualization and paravirtualization require a full operating system image for each instance.

Source : http://www.linuxjournal.com/content/containers%E2%80%94not-virtual-machines%E2%80%94are-future-cloud

Virtual Machines vs Containers

Containers can share a single Linux Kernel and, optionally, other binary and library resources.

The time to provision

Source : http://www.linuxjournal.com/content/containers%E2%80%94not-virtual-machines%E2%80%94are-future-cloud

mount /dev/sda /targetchroot /target

but that had no resource and security isolation goals for multi-tenant designs...

From the simple concept of “chroot”

source: http://openvz.org

CpuDevicesProcessesMemoryDisk spaceNetwork

Wha

t if y

ou co

uld

cont

rol..

.

Openvz & LXC

Needcontrol over specifichost resources

cgroupsControl Groups provide a mechanism for aggregating/partitioning sets of tasks, and all their future children, into hierarchical groups with specialized behaviour.

~$ ls /sys/fs/cgroupblkio cpu cpuacct cpuset devices freezer hugetlb memory perf_event

example:

lxc-cgroup -n foo cpuset.cpus "0,3"

Containers & Cgroups

https://www.kernel.org/doc/Documentation/cgroups/cgroups.txt

ricardo@ricardo-box:~$ sudo lxc-checkconfig Kernel configuration not found at /proc/config.gz; searching...Kernel configuration found at /boot/config-3.8.0-26-generic--- Namespaces ---Namespaces: enabledUtsname namespace: enabledIpc namespace: enabledPid namespace: enabledUser namespace: missingNetwork namespace: enabledMultiple /dev/pts instances: enabled

--- Control groups ---Cgroup: enabledCgroup clone_children flag: enabledCgroup device: enabledCgroup sched: enabledCgroup cpu account: enabledCgroup memory controller: enabledCgroup cpuset: enabled

--- Misc ---Veth pair device: enabledMacvlan: enabledVlan: enabledFile capabilities: enabled

Note : Before booting a new kernel, you can check its configurationusage : CONFIG=/path/to/config /usr/bin/lxc-checkconfig

LXC on Ubuntu

Since Ubuntu 12.04, containers are constrained by apparmor by default

- /usr/bin/lxc-start is automatically transitioned to its own profile, where it is only allowed to mount into the

container’s tree.

- The default policy attempts to protect the host from accidental container abuses – such as writing to /proc/sysrq-

trigger and /proc/mem,

- Each container configuration can specify a custom profile.

On Ubuntu 13.04 - We are able to exploit user namespaces and support stacked apparmor profiles

- Apport hooks for better debug support,

- Greater scriptability by providing a liblxc api.

By 14.04User namespace should support container use by unprivileged users.

Other resources:

http://www.ibm.com/developerworks/linux/library/l-lxc-security/index.html

https://wiki.ubuntu.com/LxcSecurity

http://wiki.ubuntu.com/UserNamespace

LXC Security with Apparmor

Wait…I don’t have to use heavy virtualboxes?

Let’s start with Vagrant and puppetize it!

You just need that guy

You will get:

1. Drupal (latest version)

2. Nginx

3. Php + php-fpm

4. Mysql

5. Phpmyadmin

6. xhprof

7. xdebug

8. composer

https://github.com/ricardoamaro/drupal-lxc-vagrant-docker

My contribution to Drupal Containers

Install latest Vagrant from: http://downloads.vagrantup.com/tags/v1.2.7 or later.

Install lxc + redir.

sudo dpkg -i vagrant_1.2.7_x86_64.deb

sudo apt-get install lxc redir

Vagrant LXC (demo) - Install

Get the code from:https://github.com/ricardoamaro/drupal-lxc-vagrant-docker

git clone git@github.com:ricardoamaro/drupal-lxc-vagrant-docker.

git

cd ~/drupal-lxc-vagrant-docker

1 - Clone the code

vagrant plugin install vagrant-lxc

vagrant up --provider=lxc

sudo lxc-ls --fancy

# redirect port 80 to the host

sudo redir --lport=80 --cport=80 --caddr={container ip} &

# and/or edit the /etc/hosts file with:

${IP} drupal phpmyadmin xhprof

2 - Get the plugin & deploy

Now…

I have to

build this

every time?

use Docker

Docker Who??

this Docker

and ship them has containers

Ship containers? Build Once, Run Anywhere

Install docker:

sudo apt-get -y install dockercurl get.docker.io | sudo sh -x

Import container to docker:

sudo tar -C /var/lib/lxc/{container name}/rootfs/ -c . | sudo docker import - dev/drupal

Start docker:

sudo docker run -i -t -p :80 dev/drupal /bin/bash

The image is already pushed to https://index.docker.io, and can be pulled using:

sudo docker pull ricardoamaro/drupal

You can ship your image into a Docker container

https://github.com/ricardoamaro/docker-drupal

https://github.com/ricardoamaro/docker-drupal-nginx

Or... build it the Docker way:

the Commands: attach Attach to a running container

commit Create a new image from a container's changes

diff Inspect changes on a container's filesystem

export Stream the contents of a container as a tar archive

history Show the history of an image

images List images

import Create a new filesystem image from the contents of a tarball

info Display system-wide information

inspect Return low-level information on a container

kill Kill a running container

login Register or Login to the docker registry server

logs Fetch the logs of a container

port Lookup the public-facing port which is NAT-ed to PRIVATE_PORT

ps List containers

pull Pull an image or a repository to the docker registry server

push Push an image or a repository to the docker registry server

restart Restart a running container

rm Remove a container

rmi Remove an image

run Run a command in a new container

start Start a stopped container

stop Stop a running container

tag Tag an image into a repository

version Show the docker version information

wait Block until a container stops, then print its exit code

The docker is awesome!

the Apihttp://docs.docker.io/en/latest/api/registry_index_spec/

the Registryhttp://docs.docker.io/en/latest/api/index_api/

Docker on Docker (v0.6)

Container layers to be used for hosting applications

Continuous Deployments & Development

Changes to the container can be committed to the central index or rolled back

Just commit the good apples

Openstack and Docker...

The future has a bonus extra:http://blog.docker.io/2013/06/openstack-docker-manage-linux-containers-with-nova/https://wiki.openstack.org/wiki/Docker

“Nova is intended to be modular and easy to extend and adapt. It supports manydifferent hypervisors (KVM and Xen to name a few), different database backends(SQLite, MySQL, and PostgreSQL, for instance), different types of userdatabases (LDAP or SQL), etc.”

And it supports Docker containers!

This project is open-source and available at: https://github.com/dotcloud/openstack-docker.

...with the Nova driver

Develop the box in layersUse only one Linux KernelDeploy quicklyBuild Once, Run Anywhere

Awesomeness!

@ricardoamaro

Questions?

Locate this session at the DrupalCon Prague website:https://prague2013.drupal.org/node/388

Click the “Take the survey” link

THANK YOU!

@ricardoamaro

Locate this session at the DrupalCon Prague website:https://prague2013.drupal.org/node/388

Click the “Take the survey” link