continuous delivery with jenkins, docker and mesos/marathon - jbcnconf
TRANSCRIPT
CONTINUOUS DELIVERY WITH JENKINS, DOCKER
AND MARATHON/MESOSJulia Mateo
@juliamateodc@duchessswiss http://jduchess.ch/
Monday, June 29, 15
GOAL
Monday, June 29, 15
CONTINUOUS DEPLOYMENT VS CONTINUOUS DELIVERY
• Continuous delivery : series of practices designed to ensure that code can be rapidly and safely deployed to production
• Continuous deployment is the next step of continuous delivery: Every change that passes the automated tests is deployed to production automatically
Monday, June 29, 15
CONTINUOUS DELIVERY
Source : http://xebia.github.io/cd-with-docker/#/
Monday, June 29, 15
From https://www.docker.com/whatisdocker/
Using Docker for Continuous Delivery
• Docker is an open platform for developers and sysadmins to build, ship, and run distributed applications.
Monday, June 29, 15
DOCKER VS VMS
Server
Host OS
Hypervisor
GuestOS
Guest OS
Guest OS
Guest OS
Libs/Bins
Libs/Bins
Libs/Bins
App 1 App 2 App 3 App 4
Libs/Bins
Server
Host OS
Docker Engine
Libs/Bins Libs/Bins
App 1 App 2 App 3 App 4
c1 c2 c3 c4
VM1 VM2 VM3 VM4
Monday, June 29, 15
CONTAINERS AND IMAGES
DockerImages Docker
Images
libcontainer, Union FilesystemCentos Ubuntu
Jettyadd App.warContainer 1
OracleDBContainer 2
Read only
WritableWritable
Read only
Monday, June 29, 15
TO SUM UP :
• Simple : container as deployment unit
• Fast : Only need to build/upload the ∆ image
• Robust : less scope of error than custom deployment scripts
• Other advantages : docker facilitates fast rollback
Monday, June 29, 15
FIRST STEP : DEPLOY TO ONE NODE
Monday, June 29, 15
THE WEB APP TO DEPLOY
Monday, June 29, 15
DEPLOY WITH DOCKER : TEST ENVIRONMENT
Localhost
Monday, June 29, 15
DEPLOY WITH DOCKER : TEST ENVIRONMENT
Localhost
Monday, June 29, 15
DEPLOY WITH DOCKER : TEST ENVIRONMENT
Localhost
Monday, June 29, 15
DEPLOY WITH DOCKER : TEST ENVIRONMENT
Localhost
Monday, June 29, 15
DEPLOY WITH DOCKER : TEST ENVIRONMENT
Localhost
Monday, June 29, 15
DEPLOY WITH DOCKER : TEST ENVIRONMENT
Localhost
docker publish plugin
Monday, June 29, 15
DEPLOY WITH DOCKER : TEST ENVIRONMENT
Localhost
docker publish plugin
ssh jenkins plugin
Monday, June 29, 15
DEPLOY WITH DOCKER : TEST ENVIRONMENT
Localhost
docker publish plugin
ssh jenkins plugin
Monday, June 29, 15
DOCKER PUBLISH PLUGIN
Monday, June 29, 15
USING DOCKER INTO DOCKER
Monday, June 29, 15
DEPLOY WITH DOCKER : TEST ENVIRONMENT
Localhost
docker publish plugin
ssh jenkins plugin
Monday, June 29, 15
DEPLOY WITH DOCKER : TEST ENVIRONMENT
Localhost
docker publish plugin
ssh jenkins plugin
Monday, June 29, 15
DOCKER PRIVATE REPOSITORY
• Add the registry’s location to the repository name : my.registry.address:port/repositoryname
Monday, June 29, 15
DEPLOY WITH DOCKER : TEST ENVIRONMENT
Localhost
docker publish plugin
ssh jenkins plugin
Monday, June 29, 15
DEPLOY WITH DOCKER : TEST ENVIRONMENT
Localhost
docker publish plugin
ssh jenkins plugin
Monday, June 29, 15
JENKINS SSH PLUGIN
Monday, June 29, 15
DEPLOY WITH DOCKER : TEST ENVIRONMENT
Localhost
docker publish plugin
ssh jenkins plugin
Node 1
Node 2
Monday, June 29, 15
DEPLOY WITH DOCKER : TEST ENVIRONMENT
Localhost
docker publish plugin
ssh jenkins plugin
Node 1
Node 2
Container 1
Container 2
Container 3
Container 4
Container 5
Monday, June 29, 15
PROBLEMS
• Scalability
• Fault tolerance
Monday, June 29, 15
NEXT STEP : DEPLOY TO A MESOS
CLUSTER
Monday, June 29, 15
DEPLOY WITH DOCKER : TEST ENVIRONMENT
Localhost
docker publish pluginnode1Container 1
Container 3
Container 4Container 5
Container 2
Http request plugin
Master
Executor
Marathon
Executor
Monday, June 29, 15
• Abstraction of cluster resources
• Share resources across multiple frameworks (versions of the same fwk)
• Resource fair sharing : alternative to static partitioning
•Data locality
What is Mesos ?
Monday, June 29, 15
MESOS
ZKMaster Master
Executor Executor Executor
Executor ExecutorExecutor
Executor Executor Executor
Slaves
Monday, June 29, 15
HOW DOES IT WORK ?
http://mesos.apache.org/documentation/latest/mesos-architecture/
Monday, June 29, 15
USING MARATHON WITH MESOS
•Marathon is a Mesos framework written in Scala
• Provides easy deployment of Docker containers
•Manages of long running apps
• Rest API for developers
Monday, June 29, 15
MARATHON
ZKMaster Master
Executor Executor Executor
Executor ExecutorExecutor
Executor Executor Executor
Slaves
Monday, June 29, 15
MARATHON
ZKMaster Master
Executor Executor Executor
Executor ExecutorExecutor
Executor Executor Executor
Slaves
Marathon
Monday, June 29, 15
DEPLOY WITH DOCKER : TEST ENVIRONMENT
Localhost
docker publish plugin
Http request plugin
Master
Executor
Marathon
Executor
Monday, June 29, 15
JENKINS : HTTP REQUEST PLUGIN
Monday, June 29, 15
DEMO
Monday, June 29, 15
USING HAPROXY FOR LOAD BALANCING
Monday, June 29, 15
DEPLOYING TO PRODUCTION
• Canary releasing
• Blue Green deployment
Monday, June 29, 15
FEEDBACK FROM DEVELOPER PERSPECTIVE
•Many technologies in a short time : ansible, docker, mesos, marathon..
• All of them are quite recent technologies
Monday, June 29, 15
FEEDBACK FROM DEVELOPER PERSPECTIVE
•Docker, Mesos/Marathon : developer friendly
• Big and growing community
Monday, June 29, 15
NEXT IMPROVEMENTS
• Fine grained proxy configuration, service discovery...
Monday, June 29, 15
THANK YOU !
QUESTIONS
?Monday, June 29, 15