why everyone is excited about docker (and you should too...) - carlo bonamico - codemotion milan...
TRANSCRIPT
Carlo Bonamico
Why everyone is excited about Docker (and you should too...)
[email protected] - NIS [email protected] – Genova Java User Group
Twitter: @carlobonamico
MILAN november 28th/29th 2014
Didn't you hear these at least once? Can you deploy my application? It's for Linux...
Yes, but which distribution? release? patch?
Why the deploy failed? Didn't you test the app? Yes, but the production environment is slightly different
The new version is having some issues, can you rollback to the previous one, please? Yes, but it will take some hours – if John hasn't already
gone home – in that case he'll do it on monday
Can you debug this production problem? Yes, but I need an hosted DEV environment as I can't
run all the needed VMs on my laptop
In the beginning...
We moved from physical server to VMs performance and
resource usage issues
Got more security and hardware independence but creating a VM
still takes time
Some hosting / cloud providers took advantage of kernel-level virtualization LXC OpenVZ
But out-of-reach of the common man Dev Try setting up LXC...
Then in 2013 (please fake drum roll)
Solomon Hykes (@solomonstre) started Docker as an internal project within dotCloud.com hosting to make Linux Containers an order of magnitude easier and more powerful, too
Open Sourced in March 2013 in a week, it went to the top projects on GitHub
https://github.com/docker/docker
A catalyst for innovative work on containers shortly, several other key projects converged
Fast Forward to mid-2014
Major Open Source project with contributions from all the big names in IT Google, RedHat, OpenShift, Ubuntu ...
DotCloud → Docker Inc. https://www.docker.com/
All cloud big and small names are in a rush to provide Docker hosting Amazon, CloudFoundry, Linode, Digital Ocean… and Microsoft !
both for Azure and soon for the new Windows Server...
But why should I join the party?
In short, Docker makes creating Development Test and Production
environments an order of magnitude simpler faster and completely portable
across both local and cloud infrastructure
Docker hello world...
What's happening here? user@laptop:~$ docker pull ubuntu:14.04.1
user@laptop:~$ docker run ubuntu:14.04.1 echo "Hello World"
Hello World
And here? user@laptop:~$ docker run t i ubuntu:14.04.1 /bin/bash
root@d1fa8fcb4518:/# ls
bin boot dev etc home lib lib64 media mnt optproc root run sbin srv sys tmp usr var
root@d1fa8fcb4518:/# python
bash: python: command not found
root@d1fa8fcb4518:/#
Behind the scenes...
I run the docker cli (Command Line Interface )user@laptop:~$ docker run t i ubuntu:14.04.1 /bin/bash
the CLI connects to docker daemon by REST API, which asks the Linux kernel to create a new container
d1fa8fcb4518
and runs /bin/bash in it, soroot@d1fa8fcb4518:/# ls
bin boot dev etc home lib lib64 media ...
lists the filesystem of the container (!= from host OS)root@d1fa8fcb4518:/# python
bash: python: command not found
So what's inside Docker?
Isolation layer based on kernel namespaces separate process trees, network, user IDs and mounted
file systems
Resource isolation through cgroups CPU, memory, block I/O and network
Standard interface through libcontainer based on libvirt, LXC and systemd-nspawn
And more...
How are data & containers stored?
AUFS Another Union Filesystem possibly other snapshotting fs (zfs) / block device (LVM)
Layered approach rootfs → kernel layer bootfs → a Linux distribution emacs apache application
Copy-on-Write approach – à la subversion (SVN)
Containers, Images and Index
A Container is a running instance can run 100-1000 containers per host
An Image is a static snapshot in turn based on a series of layers unique hash for each layer, so
Images are basically versioned (think git) can be tagged ubuntu:14.04.1 can be updated by applying layer deltas
Images can be stored in an Index local and remote indexes (think maven / npm repos)
So a container is like a lighter/better Virtual Machine?
Well...
VM vs Container
A Virtual Machine needs an hypervisor and a full OS inside
Bigger footprint RAM needed Storage space
Tend to be slower 2 filesystems, 2 OSes
Strong resource management
A Container talks to the host kernel
Smaller footprint no RAM needed for
Guest OS differential storage
Tend to be faster direct CPU access
Less sophisticated resource management
VM vs Container
Great! but tell me about security
Are containers less secure than Vms? the answer is nuanced... https://docs.docker.com/articles/security/
Can I use Docker in Production? Sure! many Internet companies trust it
But a container still needs good System Administration & InfoSec practices! limiting privileges, avoiding unsecure defaults, etc...
http://www.slideshare.net/jpetazzo/dockerlinuxcontainerslxcandsecurity
http://opensource.com/business/14/7/dockersecurityselinux
Avoid This!
Docker workflow
Start with a dockerfile
Define an image for running Tomcat 7 inspired by https://registry.hub.docker.com/_/tomcat/
FROM java:7jre
RUN groupadd r tomcat && useradd r createhome g tomcat tomcat
ENV CATALINA_HOME /usr/local/tomcat
ENV PATH $CATALINA_HOME/bin:$PATH
RUN mkdir p "$CATALINA_HOME" && chown tomcat:tomcat "$CATALINA_HOME"
WORKDIR $CATALINA_HOME
USER tomcat
ENV TOMCAT_MAJOR 7
ENV TOMCAT_VERSION 7.0.57
Dockerfiles - continued
ENV TOMCAT_TGZ_URL https://www.apache.org/dist/tomcat/tomcat$TOMCAT_MAJOR/v$TOMCAT_VERSION/bin/apachetomcat$TOMCAT_VERSION.tar.gzRUN curl SL "$TOMCAT_TGZ_URL" o tomcat.tar.gz \
&& curl SL "$TOMCAT_TGZ_URL.asc" o tomcat.tar.gz.asc \
&& tar xvf tomcat.tar.gz stripcomponents=1 \
&& rm bin/*.bat \
&& rm tomcat.tar.gz*
EXPOSE 8080
CMD ["catalina.sh", "run"]
Public repo of Dockerfiles, with automatic build http://dockerfile.github.io/
Building an image
Build the image from the Dockerfiledocker build .
You can then do further edits, then docker build .
And archive the image locallydocker commit 38b73dfecc3c dockersimplesamplesweb
And tag itdocker tag 47432ccfea81 dockersimplesamplesweb:1.0
List local imagesdocker images
Starting a container
Start a container interactivelydocker run i t dockersimplesamplesweb /bin/bash
Start a container as a daemon using defaul entrypoint
docker run d dockersimplesamplesweb:1.0
Check running containersdocker ps
And stopping it docker stop <<id>>
Check also stopped containersdocker ps a
Attaching to a running container
Using nsenterdocker inspect format "{{ .State.Pid }}"
determined_bardeen
nsenter target $PID mount uts ipc net pid
Storing and Sharing data
Creating a Container to host a data Volume#Dockerfile
FROM busybox
VOLUME /var/lib/mysql
CMD /bin/sh
Create the Imagedocker build tag carlobonamico/datastore
Create the Containerdocker run d name pgsql_data v /var/lib/pgsql/ carlobonamico/datastore
Attach the volume to another containerdocker run d volumesfrom pgsql_data cb/postgresdb
Publishing
To the Central Registrydocker push carlobonamico/dockersimplesamplesweb
need a free account on https://hub.docker.com/
see result at https://registry.hub.docker.com/u/carlobonamico/dockersimplesamplesweb/
Tag and publish to a private repository docker tag 8dbd9e392a96 mylocalrepo:5000/dockersimplesamplesweb
You need https://github.com/docker/dockerregistry
Deploy to the cloud
On cloud serverdocker pull carlobonamico/dockersimplesamplesweb:2
Run itdocker run d carlobonamico/dockers...samplesweb:2
Upgrade itdocker pull carlobonamico/dockers...samplesweb:2.1
Run itdocker run d carlobonamico/dockers...samplesweb:2.1
Rollback to previous versiondocker run d carlobonamico/dockers...samplesweb:2
So what do I get?
If I am a Dev recreate complex environments on a laptop
If I am a Tester easy to recreate applications deployments and data
If I am an Ops person less configuration effort more standardization
In general lots of pre-packaged components
https://registry.hub.docker.com/
quickly deploy (groups of) packages even multiple versions at the same time
But many other benefits
to come...
Docker and DevOps
Docker gives a common, seamless collaboration model and workflow between Dev and Ops clearer separation of responsibilities
Docker and DevOps by Gene Kim https://www.youtube.com/watch?v=SaHbtEeu37M
DevOps is a software development method that stresses communication, collaboration and integration between software developers and IT professionals, as a response to the interdependence of Dev and Ops. http://en.wikipedia.org/wiki/DevOps
Docker helps Continuous Delivery
4 Practices of Continuous Delivery (from the book) Build binaries only once
package them in containers
Same mechanism to deploy to every environment and move the containers across environments
Smoke test your deployment, & If anything fails, stop the line!
Continuous Delivery of value to users through a constant flow of incremental product/service improvements along the entire pipeline Idea → Implementation → Test → Deploy → Prod
http://continuousdelivery.com/
Docker helps with CD's 8 principles
Releasing/deploying MUST be repeatable and reliable containers
Automate everything! docker is fully scriptable and has an API
If somethings difficult or painful, do it more often containers are quick to deploy many times a day
Keep everything in source control including dockerfiles!
Done means “released” it's containers all the way to production
Build quality in! containers support frequent and realistic testing
Everybody has responsibility for the release process see DevOps slide...
Improve continuously
What do I put in a Docker image?
The traditional Way VM-like approach
SSH, init.d several apps in the same container
http://phusion.github.io/baseimage-docker/ https://registry.hub.docker.com/u/phusion/baseimage/ Useful in the transition phase or to run existing SW
The Docker Way run a service per container purists say a single process per container!
From a single container
To many containers
Two key drivers Scalability Microservices
Microservices
Componentization via Services Organized around Business Capabilities Products not Projects Smart endpoints and dumb pipes Decentralized Governance Decentralized Data Management Infrastructure Automation Design for failure Evolutionary Design
Instead of big, monolitic, black-hole-like single appimplement a network of collaborating simple services
http://martinfowler.com/articles/microservices.html
“a bit like SOA, but done right”
It looks like Docker is a perfect match!
SOLID Design Principles
Apply @unclebobmartin S.O.L.I.D. principles to entire architecture Separation of Concerns → microservices Open for extension, Closed for modification →
Immutable Infrastructure never “change” a container: add a new one with the new version then discard the old one
http://blog.codeship.com/immutableinfrastructure/
Liskov Substitution Principle → APIs, service contracts Interface Segregation Principle → micro-APIs Dependency Inversion Principle → container linking
Linking containers
Run a DB and give it a name
docker run d name db postgres:9.3.5
Run a Web serverdocker rm f carlobonamico/web
does not see the db
Run a Web Server linked to the DB with automatic local dns alias registration
docker run d P link db:db carlobonamico/dssweb
Principles of Package Design How do I split functionality across Containers?
REP The Release Reuse Equivalency Principle The granule of reuse is the granule of release
CCP The Common Closure Principle Classes that change together are packaged together
CRP The Common Reuse Principle Classes that are used together are packaged together
ADP The Acyclic Dependencies Principle The dependency graph must have no cycles
SDP The Stable Dependencies Principle Depend in the direction of stability
SAP The Stable Abstractions Principle Abstractness increases with stability
Thank you again, Uncle Bobhttp://butunclebob.com/ArticleS.UncleBob.PrinciplesOfOod
Managing Development and Production clusters
Fig http://www.fig.sh/ create DEV environm.
fig.ymlweb: build: . command: catalina .sh run links: db ports: "8000:8000"db: image: postgres
Then (think vagrant-up) fig up
open source https://github.com/g
ooglecloudplatform/kubernetes
manage prod clusters use it on Google
Compute Engine or download on
premises
Ansible and Docker
So I do not need a configuration management system anymore?
Well, you still need to
Create images Manage the Docker host
Ansible to the rescue!
Ansible
Simple yet incredibly powerful Open Sourceconfiguration management and orchestration tool Infrastructure as data
http://www.slideshare.net/carlo.bonamico/infrastructureasdatawithansibleforeasiercontinuousdelivery
Ansible can support Docker in two ways http://www.ansible.com/docker
1) Manage the docker host with docker module e.g. create a container running Tomcat
docker: image=centos command="service tomcat6 start" ports=808
Building Images with Ansible
2) Copy and launch ansible playbook in Dockerfile http://www.ansible.com/2014/02/12/installingandbuildingdockerwithansible
https://github.com/CaptTofu/ansibledockerpresentation
Use base image with ansible from https://registry.hub.docker.com/repos/ansible/
FROM ansible/ubuntu14.04ansibleMAINTAINER yournameRUN git clone http://github.com/user/myapp.git /tmp/myappWORKDIR /tmp/myappADD inventory /etc/ansible/hostsRUN ansibleplaybook myapp.yml c localEXPOSE 22 3000ENTRYPOINT [“/home/app/tomcat/bin/catalina.sh run”]
So, where do I start?
Try the samples https://github.com/carlobonamico/dockersimplesamples
Great interactive tutorial at https://docs.docker.com/ https://docs.docker.com/articles/dockerfile_bestpractices/
Try Docker in the Cloud with Koding ide
http://learn.koding.com/guides/whatisdocker/
References
Cloud architectures http://sites.oreilly.com/odewahn/ddsfieldguide/ http://12factor.net/
Microservices https://skillsmatter.com/conferences/6312mucon http://douglassquirrel.com/microservices/
Distributions to put around and inside a container? https://coreos.com/
Docker and Windows http://weblogs.asp.net/scottgu/dockerandmicrosoftintegratingdockerwithwindowsserverandmicrosoftazure
Thank you!
Other presentations http://www.slideshare.net/carlo.bonamico/presentations
Follow me on Twitter @carlobonamico
updates on Docker, Ansible, Continuous Delivery and some AngularJS!
Contact me [email protected] / [email protected]
My company http://www.nispro.it
Running on Mac/Windows
Boot2docker A minimalistic VM – just SSH + docker http://boot2docker.io/
Download and launch the installer https://github.com/boot2docker/windowsinstaller/releases/latest
Launch docker Boot2Docker Start