diego container scheduler

Diego: Container Scheduler

hello!Hristo Iliev

[email protected]

Gopher

DIEGO?DEA in Go

Gopher

SteerAmbush

Loot=

ManageScheduleEvacuate

Gopher

DEA in Go

1.Architecture

101

Responsibilities

Stage & Run apps

Responsibilities

Responsibilities

Stage & Run apps

DEA

App 1 App 2

App 3 App 1

App 2

Maps traffic to container

s

Responsibilities

ResponsibilitiesMaps

traffic to container

s

DEA

App 1 App 2

App 3 App 1

App 2

Router

Maps traffic to container

s Router

DEA

App 1

Responsibilities

CC: APIHM:

Eventual consistency

Responsibilities

CC: APIHM:


Actual state

App 1 App 2

App 3 App 1

App 2

Responsibilities

CC: APIHM:


Actual state

App 1 App 2

App 3 App 1

App 2

Desired state

App 2App 3

App 2

Responsibilities

Communication

Responsibilities

DEA vs Diego

DEA + HM✘ Manage✘ Schedule

DEA vs Diego

Diego✘ Manage✘ Schedule

2.Why?

What’s wrong with DEA?

Onsi Fakhouri(VP of Engineering for CF)

Diego: Re-envisioning the Elastic Runtime (CF Summit 2014)

https://www.youtube.com/watch?v=1OkmVTFhfLY

https://www.youtube.com/watch?v=1OkmVTFhfLY

✘ Tight coupling✘ Separation of concern

Problems

✘ Tight coupling✘ Separation of concern

Hard to extend & test

Problems

✘ Triangular dependencies

Problems



Consistency

Concurrency

Problems

✘ Orchestration

Consistency

Concurrency

Problems

✘ Coupling & Separation


✘ Orchestration


Consistency

Concurrency

Problems

3.How

… was Diego made?

DEA History✘Health Manager

RubyEventual consistency

DEA History✘Health Manager

RubyEventual consistency

✘HM9000Goµ-servicesetcd

https://www.cloudfoundry.org/hm9000-ready-for-launch/

https://www.cloudfoundry.org/hm9000-ready-for-launch/

4.How

… can I use it?

Migration

DEA pool

App 1 App 2

App 3 App 1

App 2

Existing

Migration

DEA pool

App 1 App 2

App 3 App 1

App 2

Cell pool

App 4 App 5

App 4

New

Releases✘CF-Release

DEA pool

✘Diego-ReleaseCell poolModifies CF-Release

5.How

… does it work?

Eric malm(Diego Product Manager)

Diego Runtime Overview (CF Summit 2016)

https://www.youtube.com/watch?v=iv5EpheLLh0



Apps

CC-Bridge Dependencies

bbs

etcd

cflinuxfs2

file-server

nsync

tps

garden

docker-app

cc-uploader

consul

stager

converger

rep

buildpack-app

windows-app

CC-Bridge Diego Core Dependencies

route-emitter

Routing-BridgeSSH

diego-sshdssh-proxy

SSH

auctioneer



bbs

etcd

garden

consul

converger

rep

CC-Bridge Diego CoreCC

Apps

Gorouter

SSH

auctioneer

Routing-Bridge



bbs

etcd

garden

consul

converger

rep


Apps

Gorouter

SSH

containerplacement &

lifecycle

auctioneer

Routing-Bridge



bbs

etcd

garden

consul

converger

rep


Apps

Gorouter

SSH


lifecycle

auctioneer

Routing-Bridge

consistentdatastore



bbs

etcd

garden

consul

converger

rep


Apps

Gorouter

SSH


lifecycle

auctioneer

Routing-Bridge

consistentdatastore

containerexecution



bbs

etcd

garden

consul

converger

rep


Apps

Gorouter

SSH


lifecycle

auctioneer

Routing-Bridge

consistentdatastore

containerexecution

component discovery& coordination



bbs

etcd

garden

consul

converger

rep


Apps

Gorouter

SSH


lifecycle

auctioneer

Routing-Bridge

consistentdatastore

containerexecution


CF Apps



bbs

etcd

garden

consul

converger

rep


Apps

Gorouter

SSH

CF Apps


lifecycle


consistentdatastore

containerexecution

auctioneer

Routing

Routing-Bridge


Diego DesignNotes

https://github.com/cloudfoundry/diego-design-notes

http://htmlpreview.github.io/?https://raw.githubusercontent.com/cloudfoundry/diego-design-notes/master/clickable-diego-overview/clickable-diego-overview.html

https://github.com/cloudfoundry/diego-design-notes

http://htmlpreview.github.io/?https://raw.githubusercontent.com/cloudfoundry/diego-design-notes/master/clickable-diego-overview/clickable-diego-overview.html

6.What

… else can I do?

Features✘ Files

Features✘ Files✘SSH

Features✘ Files✘SSH✘Tasks

Features✘ Files✘SSH✘Tasks✘Isolation segments

https://docs.google.com/document/d/1JPjKxo0JT5rqKZV1XtpydL6ehlyKIk7ipNWMCz2xyKE

https://docs.google.com/document/d/1JPjKxo0JT5rqKZV1XtpydL6ehlyKIk7ipNWMCz2xyKE/edit


Compute




ComputeLogging




ComputeLoggingNetworking



7.VersusImpress me part

250 000Containers

https://www.cloudfoundry.org/250k-containers-in-production-a-real-test-for-the-real-world/


1250Diego Cells

28.5 TBmemory

305Infra nodes



thanks!Any questions?

You can find me [email protected]

CreditsSpecial thanks to all the people who made and released these awesome resources for free:✘ Presentation template by SlidesCarnival✘ Photographs by Unsplash

http://www.slidescarnival.com/

http://unsplash.com/

SlidesCarnival icons are editable shapes.

This means that you can:● Resize them without losing

quality.● Change fill color and opacity.

Isn’t that nice? :)

Examples:

Now you can use any emoji as an icon!And of course it resizes without losing quality and you can change the color.

How? Follow Google instructions https://twitter.com/googledocs/status/730087240156643328

✋👆👉👍👤👦👧👨👩👪💃❤😂😉😋😒😭 😸🏃💑 👶 🐟🍒

🍔💣📌📖🔨🎃🎈🎨🏈🏰🌏🔌🔑 and many more...

��

https://twitter.com/googledocs/status/730087240156643328

Extra graphics

diego container scheduler

Software