Scale Big With Docker

February 2014—Docker 0.8.0

Outline

● Why should I care?● The container metaphor● Very quick demo● Working with Docker● Building images● Docker future

Outline

● Why should I care?● The container metaphor● Very quick demo● Working with Docker● Building images● Docker future

Deploy everything

● webapps● backends● SQL, NoSQL● big data● message queues● … and more

Deploy almost everywhere

● Linux servers● VMs or bare metal● Any distro● Kernel 3.8 (or RHEL 2.6.32)

Deploy reliably & consistently

● If it works locally, it will work on the server● With exactly the same behavior● Regardless of versions● Regardless of distros● Regardless of dependencies

Deploy easily

● Don't learn a new CM tool● Reuse components and recipes● Don't install anything fancy locally

(One single local VM to handle all your apps)

Deploy at scale

● Build once● Deploy anywhere● Deploy quickly

– Images are lightweight– Docker runtime itself is lightweight– Fast start/stop times

● Manage Docker with its REST API

Outline

● Why should I care?● The container metaphor● Very quick demo● Working with Docker● Building images● Docker future

Problem: shipping goods? ? ? ? ? ? ?

? ? ? ? ? ? ?

? ? ? ? ? ? ?

? ? ? ? ? ? ?

? ? ? ? ? ? ?

? ? ? ? ? ? ?

Solution:the intermodal shipping container

Problem: shipping code

djangoweb frontend ? ? ? ? ? ?

node.jsasync API ? ? ? ? ? ?

background workers ? ? ? ? ? ?

SQL database

? ? ? ? ? ?

distributed DB, big data

? ? ? ? ? ?

message queue

? ? ? ? ? ?

mylaptop

your laptop

QA staging prod on cloud VM

prod on bare metal

Solution:the Linux container

High level approach:it's a lightweight VM

● I can SSH into it● I have root access● I can apt-get/yum install packages ● I don't see the host system (or other containers)

« Machine Container »

Low level approach:it's chroot on steroids

● I can run a single program● container = isolated process(es)● share kernel with host● no device emulation → no overhead

« Application Container »

Outline

● Why should I care?● The container metaphor● Very quick demo● Working with Docker● Building images● Docker future

Outline

● Why should I care?● The container metaphor● Very quick demo● Working with Docker● Building images● Docker future

One-time setup

● On your servers (Linux)– Packages (Ubuntu, Debian, Fedora, Gentoo, Arch...)

– Single binary install (Golang FTW!)

– Easy provisioning on Rackspace, Digital Ocean, EC2, GCE...

● On your dev env (Linux, OS X, Windows)– Vagrantfile

– boot2docker (25 MB VM image)

– Natively (if you run Linux)

●

The Docker workflow 1/2

● Work in dev environment(local machine or container)

● Other services can be perfect copies of prod● Whenever you want to test « for real »:

– Build in seconds

– Run instantly

The Docker workflow 2/2

● Push your build to a registry (public or private)● Run it in CI/CD● Run it in production● Rejoice

Something goes wrong?Just re-run previous version (it's still there!)

I'm starting a new project!

● Each service will be in its own container(s)● Build an image for each service● Pin dependencies (packages etc.) accurately● Link services together

→ Recommended workflow!

I'm Dockerizing some code!

● Making a single service runnable on Docker● Create a « Dockerfile »

(Think Makefile, Vagrantfile...)● If it's on GitHub, create a Trusted Build

(will automatically build ready-to-run images)● In any case, register it on index.docker.io

I'm migrating to Docker!

● Easy if there is only one service per server– Dockerize the service

– Containerize administrative functions (logs...)

● Otherwise, try to split services● You don't need to move everything to Docker● Don't use containers as VMs, even if you can!

I'm already using Chef/Puppet/Salt/Ansible/...!

● Plan A: generic images– Put your configuration management tool in an image

– Run that image just like you would run a VM

– Great for mixed environments

● Plan B: specialized images– Put your configuration management tool in an image

– Run that image, freeze the result, run it!

– Faster and safer

Outline

● Why should I care?● The container metaphor● Very quick demo● Working with Docker● Building images● Docker future

Authoring imageswith run/commit

1) docker run ubuntu bash

2) apt-get install this and that

3) docker commit <containerid> <imagename>

4) docker run <imagename> bash

5) git clone git://.../mycode

6) pip install -r requirements.txt

7) docker commit <containerid> <imagename>

8) repeat steps 4-7 as necessary

9) docker tag <imagename> <user/image>

10) docker push <user/image>

Authoring imageswith run/commit

● Pros– Convenient, nothing to learn

– Can roll back/forward if needed

● Cons– Manual process

– Iterative changes stack up

– Full rebuilds are boring, error-prone

Authoring imageswith a Dockerfile

FROM ubuntu

RUN apt-get -y updateRUN apt-get install -y g++RUN apt-get install -y erlang-dev erlang-manpages erlang-base-hipe ...RUN apt-get install -y libmozjs185-dev libicu-dev libtool ...RUN apt-get install -y make wget

RUN wget http://.../apache-couchdb-1.3.1.tar.gz | tar -C /tmp -zxf-RUN cd /tmp/apache-couchdb-* && ./configure && make install

RUN printf "[httpd]\nport = 8101\nbind_address = 0.0.0.0" > /usr/local/etc/couchdb/local.d/docker.ini

EXPOSE 8101CMD ["/usr/local/bin/couchdb"]

docker build -t jpetazzo/couchdb .

Authoring imageswith a Dockerfile

● Almost nothing to learn● Rebuilds are easy● Caching system makes rebuilds faster● Single file to define the whole environment!

Outline

● Why should I care?● The container metaphor● Very quick demo● Working with Docker● Building images● Docker future

Docker: the community

● Docker: >330 contributors● Docker inc.: <20 engineers ● latest milestone (0.8): >120 contributors● ~50% of all commits by external contributors● GitHub repository: >1400 forks

...and counting!

Coming Soon

● Network acceleration● Container-specific metrics● Consolidated logging● Plugins (compute backends...)

Those things are already possible,but will soon be part of the core.

Docker 1.0

● Multi-arch, multi-OS● Stable control API● Stable plugin API ● Resiliency● Signature● Clustering

Thank you! Questions?

http://docker.io/

http://docker.com/

https://github.com/dotcloud/docker

@docker

@jpetazzo