docker offshore - packaging applications for hard to reach datacenter
DESCRIPTION
What if your cooperate network infrastructure consists out of topological islands which aren’t connected permanently to the internet? How do you handle data synchronization among data centers if bandwidth is changing all the time? How do you deploy your applications in such an environment and how do you manage application monitoring? Is it possible to use cloud services, SaaS, IaaS and so on in such a setup and how to manage it? What is necessary to handle video data, images and other documents and keep everything in sync. Join us to learn how we use Docker, Chef, AWS and friends in such an unusual business case.TRANSCRIPT
Docker offshore – packaging applications for hard to reach data- centers
Joern Barthel
Warsaw, 26.09.2014
2
About us > 30 IT consultants
Offices in Berlin & Warsaw
Core areas of expertise Cloud consulting and change management Architecture and Development Operations
Warsawdevopsdays 2014
3
Our preferred mode of operations Do (almost) everything on IaaS
Advantages: Reduce procuration risks: control to developers (Almost) everything-as-code Provisioning
Requirements: Internet which is ubiquitous, cheap, low-latency and high
bandwidth
Warsawdevopsdays 2014
4Warsawdevopsdays 2014
5Warsawdevopsdays 2014
6
Not so fast, not so reliable Almost never fast and reliable off shore
Few mbit/s to up to 350 mbit/s via O3b Second(s) to ~200ms not counting packet loss
Never cheap
Sometimes fast and reliable on shore
Off shore performance depending on location, age of the vessel, geopolitics
Complex regulatory requirements
Large data centers nevertheless
Warsawdevopsdays 2014
7
Many environment, same requirements Different ships at different locations = different environments
On top of that: deployment to regular data centres
Shipping VMs not possible
Provisioning cannot easily guarantee deterministic behaviour
Warsawdevopsdays 2014
8
Current toolchain Management and reviews: Github and Enterprise
CI with Travis Pro and Atlassian Bamboo
Provisioning with Puppet and Chef
CD: not so much
Instead: Package (fpm, Uber-jars), provision, deploy
Warsawdevopsdays 2014
9
Pain points Every environment requires somewhat custom provisioning
No easy determinism
Complex software to manage runtime requirements for complex software
Package all the things – LTS politics
Warsawdevopsdays 2014
10
A future stack Begun using Docker in 0.6 for internal tooling
Shipping Docker images = (almost) no more provisioning
Provisioning becomes part of the package
Immutable build that is shared between environments
Configured through environment variables which are passed to the container
In short: don’t build & provision packages – package everything with Docker!
Warsawdevopsdays 2014
11
Introduction to Docker Go client/server
Userland virtualization
Vastly different scope
Abstract vs. concrete
No concept of changing state from A to B
Warsawdevopsdays 2014
12
Components Build
Dockerfiles = series of command Immutable images = snapshots of state Commands and context Relationships and unions Caching
Distribute
Run
Warsawdevopsdays 2014
13
Components Build
Distribute Naming images Push and pull to/from repositories Only transfer images the client does not know about
Run
Warsawdevopsdays 2014
14
Components Build
Distribute
Run Container process R/W image
Warsawdevopsdays 2014
15
Packaging & Provisioning revisited Applications: names and tags
registry.kreuzwerker.de/jira:6.3.6 registry.kreuzwerker.de/jira:7c38a67 registry.kreuzwerker.de/jira:production
Result: immutable but not (exactly) reproducible
Saving valuable bandwidth & space: Shared images: synchronize only the diffs Special case: dependency managers Patch management
Warsawdevopsdays 2014
16
Running containers Pull, start, stop
Configuration via environment
Separation of roles if desired
What’s left in the host OS? CoreOS, Project Atomic
Warsawdevopsdays 2014
17
New toolchain Management and reviews: still Github and Enterprise
CI: still Travis Pro and Atlassian Bamboo
CD: almost - Quay.io and Bamboo + Docker Private Registry
Warsawdevopsdays 2014
18
What is gained? Concrete instead of abstract provisioning
Identical, immutable builds which are configured at runtime for different environments
Easy determinism
No more packages!
Simple separation of duties where needed or wanted
Warsawdevopsdays 2014
19Warsawdevopsdays 2014
Questions please!
20Warsawdevopsdays 2014
Dziękuję!