docker containers & the future of drupal testing
DESCRIPTION
Story of an investigation to improve cloud The sad VirtualMachine story Containers and non-containers DEMO - Drupal Docker Drupal Testbots story in a Glance Docker as a testing automation factor DEMO - Docker Tesbot Integration pathTRANSCRIPT
Docker Containers: Testing automation and development in Drupal.
Amsterdam - DevOps meetup, September 2014
A new strategy for automating Drupal testing@ricardoamaro
Free/Opensource software loverSenior Cloud Engineer @AcquiaDrupal.org infrastructure/devopsDrupalist & Linux enthusiast
Father, artist, community facilitator
@ricardoamarohttps://drupal.org/user/666176
About me
Story of an investigation to improve cloud
1. The sad VirtualMachine story
2. Containers and non-containers
3. DEMO - Drupal Docker
4. Drupal Testbots story in a Glance
5. Docker as a testing automation factor
6. DEMO - Docker Tesbot
7. Integration path
today’s meetup
Hardware virtualization or platform virtualization refers to the creation of a virtual machine that acts like a real computer with an operating system.
Software executed on these virtual machines is separated from the underlying hardware resources.
What is virtualization?
Cloud infrastructure providers like Amazon Web Service sell virtual machines. EC2 revenue is expected to surpass $1B in revenue this year. That's a lot of VMs…
Why should i care?Increase
+ efficiency+ availability+ security
Reduce
- costs- hardware- energy
Virtual Machine platforms
➢ We are also paying for lot of avoidable overhead.
➢ The Virtual Machine is a full-blown operating system image.
➢ This is a heavyweight solution to run applications in the cloud.
The sad Virtual Machine story...
What is the solution?
Containers used to be terrible, but not anymoreContainers used to be terrible, but not anymore
A new concept, a new hope
Because Docker has arrived!
On any recent Linux Kernel near you!
Source : http://www.linuxjournal.com/content/containers%E2%80%94not-virtual-machines%E2%80%94are-future-cloud
Virtual Machines vs Containers
Virtualization and paravirtualization require a full operating system image for each instance.
Source : http://www.linuxjournal.com/content/containers%E2%80%94not-virtual-machines%E2%80%94are-future-cloud
Virtual Machines vs Containers
Containers can share a single Linux Kernel and, optionally, other binary and library resources.
Source : http://www.linuxjournal.com/content/containers%E2%80%94not-virtual-machines%E2%80%94are-future-cloud
Virtual Machines vs Containers
Virtualization and paravirtualization require a full operating system image for each instance.
Containers can share a single operating system and, optionally, other binary and library resources.
The time to provision
Source : http://www.linuxjournal.com/content/containers%E2%80%94not-virtual-machines%E2%80%94are-future-cloud
mount /dev/sda /targetchroot /target
but that had no resource and security isolation goals for multi-tenant designs...
From the simple concept of “chroot”
source: http://openvz.org
CpuDevicesProcessesMemoryDisk spaceNetwork
Wha
t if y
ou co
uld
cont
rol..
.
Docker
Needcontrol over specifichost resources
cgroupsControl Groups provide a mechanism for aggregating/partitioning sets of tasks, and all their future children, into hierarchical groups with specialized behaviour.
~$ ls /sys/fs/cgroupblkio cpu cpuacct cpuset devices freezer hugetlb memory perf_event
Containers & Cgroups
https://www.kernel.org/doc/Documentation/cgroups/cgroups.txt
use Docker
Docker Who??
this Docker
and ship them as containers
Ship containers? Build Once, Run Anywhere
https://github.com/ricardoamaro/docker-drupal
https://github.com/ricardoamaro/docker-drupal-nginx
Build and ship Drupal the Docker way:
the Commands:
attach Attach to a running container
commit Create a new image from a container's changes
diff Inspect changes on a container's filesystem
export Stream the contents of a container as a tar archive
history Show the history of an image
images List images
import Create a new filesystem image from the contents of a tarball
info Display system-wide information
inspect Return low-level information on a container
kill Kill a running container
login Register or Login to the docker registry server
logs Fetch the logs of a container
port Lookup the public-facing port which is NAT-ed to PRIVATE_PORT
ps List containers
pull Pull an image or a repository to the docker registry server
push Push an image or a repository to the docker registry server
restart Restart a running container
rm Remove a container
rmi Remove an image
run Run a command in a new container
start Start a stopped container
stop Stop a running container
tag Tag an image into a repository
version Show the docker version information
wait Block until a container stops, then print its exit code
The docker is awesome!the Apihttp://docs.docker.io/en/latest/api/registry_index_spec/
the Builderhttp://docs.docker.com/reference/builder/
the Registryhttp://docs.docker.io/en/latest/api/index_api/
Docker on Docker > v0.6
Changes to the container can be committed to the central index or rolled back
Continuous Deployments, Development & Testing... Keep only the good apples
How does this picture help the Drupal testing and the jobrunner?
TESTBOTS
Testbot is `Sad`
Actual running testbots were created in 2007 and revamped in 2009, need modernizing in order to overcome several pending issues that make them difficult to debug, go faster and be less prone to error.
Since they are full virtual servers, they consume much cpu, memory and IO. A part from that developers have ever increasing demands. Multiple environments and databases need to be tested and contained.
Testbots(jobrunner)
Runs the actual test
drupal.org
Issue queue starts processand displays
results
qa.drupal.org
keeps track of jobs and detailed
info
Current Drupal testing workflow (simplified):
Integration round table:
Testbot1
drupal.org
Issue queue starts processand displays
results
resultsdrupal.org
keeps track of jobs and detailed
info
New proposed Drupal testing workflow:
Jenkins
Dispatcher
Testbot2
Testbot3
Testbot4
Testbotn
Docker Container types:
- PHP5.3 / PHP5.4 / PHP5.5 / PHP5.6- MariaDB - MySQL- Sqlite - Postgresand others...
API
@ricardoamaro
Docker Containers: Testing automation
and development in Drupal.Amsterdam - DevOps meetup, September 2014
@ricardoamaro
Thank you!