Docker DeploymentsAt Scale and In Production

Karl MatthiasLead Site Reliability Engineer

Friday, March 13, 15

Where We Came From

About 5 applications made up the whole site

Server pool grew but didn’t drastically change

Deployment tooling is all hand-crafted

1-2 deployment specialists on each team

FAIRLY STABLE, TRADITIONAL SETUP

Friday, March 13, 15

Where We Were GoingShipping a large new product that would double server count

SOA: Break up those monolithic applications into ~25 new services

Go from 1-2 deploys a day to up to 1 deploy per day per service

Double the number of developers each year

ORGANIZATIONAL CHANGES

Friday, March 13, 15

Fix Deployment

Deployment was the riskiest part of the new era

Only worry about stateless apps for right now

Define some application standards

Then fix the process

HIGHEST RISK

Friday, March 13, 15

Traditional DeploymentMIGHT BE FAMILIAR

Friday, March 13, 15

Docker DeploymentONE HANDOFF

Friday, March 13, 15

Clean separation between development and operations concerns

Supports heterogeneous app languages

Build one set of reliable deployment tools

Discover dependencies and conflicts in testing

Ship exactly what you tested

Keep configuration out of the application repository

Why Docker?

Friday, March 13, 15

Repeatable Deployments

Rolling, zero-down-time deploys

Any ops engineer can deploy any app

Configuration is not in the application repo

CENTURION

Friday, March 13, 15

Taking it to Production

Repeatable deployments: Centurion

Service Discovery: SmartStack (Airbnb) on Apache Zookeeper

Visualization: Bosun (name likely to change)

Monitoring: Initially Nagios and our check_docker, increasingly customized tooling

THE WHOLE PACKAGE

Friday, March 13, 15

Bosun is our visualization framework (evolving to do discovery)

It just finds new nodes and services

Gossip-based so it scales well

Works in development with no infrastructure

See all the instances of a service on all hosts

What’s Running?SEE THE CLUSTER

Friday, March 13, 15

The Future

Centurion to deploy into Mesos/Marathon

Discovery through gossip-based Bosun (in progress)

Dynamic scheduling of hosts by Marathon

Dynamic monitoring from Bosun data

WHERE TO FROM HERE?

Friday, March 13, 15

Links

@relistan - Karl Matthias

Docker Book: http://goo.gl/AM943M

https://github.com/newrelic/centurion

https://github.com/newrelic/check_docker

https://github.com/newrelic/go_nagios

Friday, March 13, 15