How I automated myself out of my job

with Apache Mesos

Michail Kolcow, 2017

About me• 12+ years in IT • Java, Scala, Lisp, Python, … • Tatar, Russian, English, Latvian, Polish • Spark, Hadoop, Couchbase • I love castles & craft beer • 2 lovely kids: Kira & Adam

The most British thing ever?

The most British thing ever

The most British thing ever

Kettle-o-strophe

Kettle-o-strophe is real

Autoscaling to prevent the future kettle-o-strophe

Dinorwig Power Station, Wales, UK

Why do we care about kettles?

My first kettle-o-strophe

My first kettle-o-strophe

• Start of the main product development: 2015 • Autoscaling groups, Cloudwatch & Elastic IP • Mandatory code review • ~100 microservices • Fully automated deployment w/Ansible &

Jenkins • Scala, Go, Java • Postgres, Couchbase, Influx, GraphDB • Non-distributed dev teams & DevOps

My first kettle-o-strophe

My first kettle-o-strophe

My first kettle-o-strophe

My second kettle-o-strophe

My second kettle-o-strophe

• Start of the main product development: 1948 • ~ 1000 of apps, code from different epochs • Distributed teams: EU, US, Australia • Highly motivated staff • Some apps got their own build & deployment

automation process • Scala, Java, Node, Python, Ruby, whatever… • Oracle, Sybase, Riak, Postgres, Couchbase,

Influx, GraphDB, choose your own DB

My second kettle-o-strophe

DATACENTRES ARE NOT IMMUTABLE!!!

To avoid the inevitable

To avoid the inevitable

To avoid the inevitable

Tech we use

Tech we build

Developers’ pipelines

Mesos as a DC abstraction

Mesos as a DC abstraction

Mesos is a distributed kernel

How it helps

How it helps

We need more abstractions

To track the tasks

To deploy the apps

It’s demo time

DEMO