exploring docker in ci/cd
Post on 07-Aug-2015
110 Views
Preview:
TRANSCRIPT
CI/CD with Docker
Henry Huang
Agengda
• Docker brings benefits on CI/CD process
• Coding: IDE with Docker
• Build with Docker
• Jenkins with Docker
– 1 Master x Slaves
– Masters x Slaves
Docker Provides
• Easy to prepare dev build environments
• No languages/libraries dependencies
• Build starts in seconds - more faster
• Increased robustness
• Simplified deployment on CI nodes
• Faster Rollback & Canary Release
• Easy to scale on CI nodes
• Well compatible with existing CI tools
CI/CD Process
Coding
• IDE with Docker
– Can run your code any where
If it runs on your laptop it will run on the integration server too
– Don't have to bother about libraries, conflicts or installing GCC, etc.
– Easy to build with different language packs or libraries, etc.
Coding • Requirements for IDE with Docker
– Dockerfile • Syntax highlighting • Autocomplete • Syntax validation
– Compose yml file • Define inter container relation as links and volumes • Run multiple containers with one click
– IDE build system • Run containers from the IDE with different language packs
or library requirement, etc.
– IDE runtime system • Launch runtime environment inside the container for
further troubleshooting and testing
Docker Plugin for IDE
• Doclipser
• Sublime Docker
• IntelliJ IDEA 14.1
• Eclipse JBoss Tools
• Visual Studio 2015 RC Tools for Docker - Preview extension
Summary Page: http://domeide.github.io/
Build
• Currently, we cannot meet the CI principles: – Every commit (to baseline) should be built
– Keep the build fast
– Make it easy to get the latest deliverables
• Problems: – [Dragged] Update dev build environment
– [Dragged/Fragile] Build the source files
– [Complicated] Get the latest/specified deliverables
Build with Docker
• Easy to prepare dev build environments
• No languages/libraries dependencies
• Build starts in seconds - more faster
• Increased robustness
• Simplified deployment on CI nodes
• Faster Rollback & Canary Release
• Easy to scale on CI nodes
• Well compatible with existing CI tools
Example: Way of Building Docker
• Chicken and Egg – build the latest docker binary in old version of docker container
• Dockerfile: provides the necessary dependencies and environment to build – COPY source files into Docker build image
• hack/make.sh: provides the build script for – binary, cross, unit test, integration-cli-test, etc.
• Verify the new version via dind (docker-in-docker) • Deliverables
– Docker Images – Binary
• mounting host folder to the dev container • Via `docker cp`
dind - build docker
# git clone https://github.com/docker/docker.git
# cd docker; docker build -t docker-dev .
# docker run --privileged --rm -ti docker-dev /bin/bash root@383dd9ee7613:/go/src/github.com/docker/docker# hack/make.sh binary
---> Making bundle: binary (in bundles/1.8.0-dev/binary)
Building: bundles/1.8.0-dev/binary/docker-1.8.0-dev
Created binary: bundles/1.8.0-dev/binary/docker-1.8.0-dev
root@383dd9ee7613:/go/src/github.com/docker/docker# docker
bash: docker: command not found
… …
root@383dd9ee7613:/go/src/github.com/docker/docker# docker -dD
root@383dd9ee7613:/go/src/github.com/docker/docker# docker run hello-world
dind - hello world
Jenkins to build docker
• https://jenkins.dockerproject.org/job/Docker%20Master/
Docker Registry • Docker Registry
– V1: named as registry (python, boto) • https://github.com/docker/docker-registry
– V2: named as distribution (re-write from scratch, golang) • https://github.com/docker/distribution
• Storage model – inmemory – Filesystem – S3 – azure: Microsoft Azure Blob Storage – rados: Ceph Object Storage
• Frontend UI – https://github.com/kwk/docker-registry-frontend – No support V2
Docker Registry
# http://docs.docker.com/registry/deploying/
• Take Dockerfile
– HTTP
• Add “--insecure-registry” into docker daemon options # docker pull private-registry.iwsaas:5000/centos:latest
– HTTPS: SSL private key and certificate
• Take Compose
– Extra data volumes, etc.
Multiple Registries
• Mirroring (still in proposal)
• Cross repository (included in the v2.1 milestone)
Jenkins with Docker
• Build/Push docker images – Execute docker CLI in jobs: case #1
– Docker build step plugin: case #2 • https://wiki.jenkins-ci.org/display/JENKINS/Docker+build+step+plugin
• Execute tasks inside docker container – Docker plugin (dind): case #3
• https://wiki.jenkins-ci.org/display/JENKINS/Docker+Plugin
– Patched docker plugin (master and slave in the same container level): case #4 • https://github.com/henrysher/docker-plugin
Demo for case 4
• Add new options into docker daemon marked in red: OPTIONS='-H tcp://0.0.0.0:4243 -H unix:///var/run/docker.sock --selinux-enabled'
• Run the docker image “iwsaas/jenkins” as a Jenkins container # docker run -d -p 8080:8080 --name jenkins iwsaas/jenkins
b675af71332782a9bbfe6d92469d9a280ae751593b460520ac6bd6ab047b8b70
• Configure Docker Plugins – Docker URL(docker0 virtual bridge address)
• http://172.17.42.1:4243
– Docker Template (iwsaas/jenkins-slave) • Default username/password: jenkins/jenkins
• Create a “test” job and execute some basic commands
Case 4 - Console Output
• Execute the command inside the container
– Container ID: e380cfb050a1
Jenkins Problems
• Multiple Masters
– Sync & Standby
• Resource isolation & allocation (for slaves)
– Multiple tasks running on one slave
– CPU, Memory, Disk, etc. per task?
Mesos + Marathon +Docker+ Jenkins
• Mesos: kernel of distribute systems
• Marathon: init/upstart for mesos
Mesos + Marathon + Docker + Jenkins
• Mesos: kernel of distribute systems
• Marathon: init/upstart for mesos
top related