scale your docker containers with mesos
TRANSCRIPT
Docker containers at scale withApache MesosDistributed Systems Engineer
Apache Mesos PMC
Timothy Chen
@tnachen
Mesoswhat? Marathon
Chronos
Demo!
Status quo is static partitioning and use of virtual machines
Add some virtual machines
Provision Hadoop
Provision a web service
Moar data, moar Hadoop
Mesos let us treat a cluster ofnodes...
As one big computer
Not as individual machines
Not as VMs
But as computational resourceslike cores, memory, disks, etc.
240 million monthly activeusers
500 million tweets per dayUp to 150k tweets per secondMore than 100TB per day of
compressed data
“Mesos is the cornerstone of our elasticcompute infrastructure — it's how we build allour new services and is critical for Twitter'scontinued success at scale. It's one of the
primary keys to our data center efficiency.” —Chris Fry, SVP of Engineering at Twitter
Containerization in Mesos, a brief history
Mesos is...A top-level Apache project A cluster resource negotiator Scalable to 10,000s of nodes Fault-tolerant, battle-tested An SDK for distributed apps
Mesos for all the things
Marathon Mesoswhat?
Chronos
Demo!
a self-serve interface to your cluster
distributed "init" for long-running services
a private fault-tolerant PaaS
Marathon does it!Start, stop, scale, update appsNice web interface, APIHighly available, no SPoFNative Docker supportFully featured REST APIPluggable event busRolling deploy / restartApplication health checksArtifact staging
Chronos
Mesoswhat?
Marathon
Demo!
Introducing Chronos
a scheduler for batch and one-off jobs
Distribute a graph of jobs
Dependency graph for execution
FeaturesDistributed job schedulerWeb interface, APIHighly available, no SPoFNative Docker supportEasy scheduling with repeating intervals
Chronos REST
PUT chronos-node:8080/scheduler/job/job1
GET chronos-node:8080/scheduler/jobs
DELETE chronos-node:8080/scheduler/task/kill/job2
Demo!
Mesoswhat?
Marathon
Chronos
Thanks!Come and talk to me
P.S., we're hiring!