beyond static configuration

44
Beyond static configuration management An overview of current developments Dr. Stefan Schimanski

Upload: sttts

Post on 19-Aug-2014

318 views

Category:

Engineering


0 download

DESCRIPTION

Current development around Docker orchestration, service discovery and cluster operation systems

TRANSCRIPT

Page 1: Beyond static configuration

Beyond static configuration managementAn overview of current developments !Dr. Stefan Schimanski

Page 2: Beyond static configuration

About meOpenSource developer since mid of 90ies

Freelance Developer & Consultant

1st DevOpsDays

Ex-head of Linux Competence Center @ German Air Traffic Controlpushing Puppet, DevOps, CI, OpenStack & agile

Before spent 5 years @ LMU Munich to come up with essentially this:

!

!

@the1stein, github.com/sttts, [email protected], schimmi@freenode

Page 3: Beyond static configuration

Long long time ago….

Page 4: Beyond static configuration

Long long time ago, in 2011:

The DevOps world was simple. !

Essentially, choose between: !

Cfengine/Puppet/Chef Ubuntu/Debian/Red Hat/Suse

implement monitoring, metrics, orchestration, log-management and deploy apps, preferably from

some CI/CD pipeline, usually on bare metal or IaaS

Page 5: Beyond static configuration

– In some OpenSpace @ DevOpsDays Berlin May 2013

„Who thinks that Docker changes everything?“

Page 6: Beyond static configuration

16 months later

Page 7: Beyond static configuration
Page 8: Beyond static configuration

Claim: DevOps tool chains are being disrupted

Page 9: Beyond static configuration

Claim: System engineering is being disrupted

Page 10: Beyond static configuration

Claim: Linux/Unix userland is being disrupted

Page 11: Beyond static configuration

Exciting times

Page 12: Beyond static configuration

Development Linux Distributions change,

become very special-purpose oriented. !A container doesn’t need - multi-user/tenant support - an init system - initrd, kernel, boot loader, installer,

hardware detection - application separation - a strict file system hierarchy standard - a lot of daemons.

Page 13: Beyond static configuration

Development Linux Distributions change,

become very special-purpose oriented. !A host for containers doesn’t need - multi-user/tenant support - a lot of user customizations - an extensive packaging system - an in-place&time update system - a strict file system hierarchy standard - a lot of traditional daemons - a desktop environment.

Page 14: Beyond static configuration

CoreOS an OS for cluster nodes running containers

based on systemd

fault-tolerant distributed key-value store: etcd

cluster layer above systemd: fleet and fleedctl

clear separation of state and OS

update = reboot

under heavy development

Page 15: Beyond static configuration

Container Orchestration

Page 16: Beyond static configuration

figFast, isolated development environments using Docker

like a vagrant for docker

single-host

fig.yml describing multiple containers, with links, environment variables, etc.

Page 17: Beyond static configuration

maestro-ng Container orchestration for Docker environments

multi-host supportwith static host assignment

powerful container dependencies and linking features

service discovery via env variables and integration with zookeeper

Page 18: Beyond static configuration

fleet in CoreOS

Normal systemd primitives for dependencies

Page 19: Beyond static configuration

libswarm by Docker A minimalist toolkit to compose network services

libswarm, libchan

Page 20: Beyond static configuration
Page 21: Beyond static configuration

Development static to dynamic system configuration

Page 22: Beyond static configuration

zookeeperserf

consul

etcd

SkyDock

Page 23: Beyond static configuration

ZookeeperDemo

Page 24: Beyond static configuration

etcdInitialization • app1: etcd -peer-addr 192.168.0.1:7001 -addr 192.168.0.1:4001 -data-dir /tmp/etcd -name app1 -bind-addr 0.0.0.0 • app2: etcd -peer-addr 192.168.0.2:7001 -addr 192.168.0.2:4001 -data-dir /tmp/etcd -name app2 -bind-addr 0.0.0.0

-peers 192.168.0.1:7001, 192.168.0.3:7001 • app3: etcd -peer-addr 192.168.0.3:7001 -addr 192.168.0.3:4001 -data-dir /tmp/etcd -name app3 -bind-addr 0.0.0.0

-pears 192.168.0.1:7001, 192.168.0.2:7001 !Distributed Key/Value store • etcdctl set /message Hello • etcdctl get /message

Hello • curl -L http://127.0.0.1:4001/v2/keys/message

{"action":"get","node":{"key":"/message","value":"Hello","modifiedIndex":4,"createdIndex":4}} !Watches • app1: etcdctl watch /foo-service —recursive • app2: etcdctl set /foo-service/container2 localhost:2222

app1: localhost:2222 !TTLs • etcdctl set /foo "Expiring Soon" --ttl 20 # gone after 20 seconds

Page 25: Beyond static configuration

confd

/etc/confd/conf.d/myapp-nginx.toml /etc/confd/templates/nginx.tmpl

Key/Values in etcd (or consul):

confd -verbose -onetime -node 'http://127.0.0.1:4001' -confdir /etc/confd/conf.d

Page 26: Beyond static configuration

Serfserf agent & serf join app2:7946 app3:7946 serf event user:deploy !

!

!

!

!

!

serf members -tag role=load-balancer app1.devops-meetup.com 10.131.251.53:7946 alive role=db

deploy.sh

Page 27: Beyond static configuration

Consulapp1:# consul agent -server -data-dir /tmp/consul -bootstrap-expect 1

app2:# consul agent -server -data-dir /tmp/consul

app3:# consul agent -server -data-dir /tmp/consul

consul join app2 app3

consul members

Page 28: Beyond static configuration

Consul – service discoveryapp1: /root/consul.d/web.json

consul agent -server -data-dir /tmp/consul -config-dir /root/consul.d

dig @127.0.0.1 -p 8600 web.service.consul

apt-get install dnsmasq echo server=/consul/127.0.0.1#8600 >> /etc/dnsmasq reload dnsmasqdocker run —dns 172.17.42.1 ubuntu ping -c 1 web.service.consul

Page 29: Beyond static configuration

Consul – dynamic service announcement

Early integration of Mesos/Marathon and Consul – not todayhttps://github.com/keenlabs/marathon/commit/290036e34337dcd6483550b7ab7d723bc4378d5f

curl -i localhost:8500/v1/agent/services

!

curl -i -X PUT localhost:8500/v1/agent/service/register -d ‚{"ID":"web2","Name":"web2","Service":"web","Tags":["prod"],"Port":31774}'

curl -i -X DELETE localhost:8500/v1/agent/service/deregister/web2

Page 30: Beyond static configuration

Consul – service announcement from dockerconsul agent -server -data-dir /tmp/consul -config-dir /root/consul.d -client 0.0.0.0 # so docker can access consul API

Dockerfilewith run_consul_service.sh code:

Page 31: Beyond static configuration

Consul – discovery in action

app1: docker run -e CONSUL_SERVICE='{"Name":"web2","Service":"web","Tags":["prod"],"Port":1080}' -e RUN="nodejs /webserver.js" -p 1080:80 -i -t consul-ubuntu

app3: curl web2.service.consul:1080

!

!

What about dynamic ports assigned by Docker?

Page 32: Beyond static configuration

Vulcand – web proxy with native etcd supportChanges to configuration take effect immediately without restarting the service

Page 33: Beyond static configuration

An operation system for a computer - manages hardware resources - offers services to programs - isolates running programs

for a cluster of machines

Development

Page 34: Beyond static configuration
Page 35: Beyond static configuration

Mesos a cluster meta operation system

„Apache Mesos is a cluster manager that simplifies the complexity of running applications on a shared pool of servers.“

Fault-tolerant replicated master using ZooKeeper

Scalability to 10,000s of nodes

Isolation between tasks with Linux Containers

Quite OS agnostic

developed since 2010, OpenSource

Marathon = init daemon on Mesos

Chronos = cron daemon on Mesos

Deimos = Docker integration

Page 36: Beyond static configuration

– Abdur Chowdhury Former Chief Scientist, Twitter Inc. Claim:

Mesosphere brings Google-scale compute to everybody.

Page 37: Beyond static configuration

Scaling

Fault Tolerance

Page 38: Beyond static configuration

Mesos/MarathonDemo

Page 39: Beyond static configuration

Google Kubernetes also a cluster operation system„Kubernetes is a system for managing containerized applications across multiple hosts, providing basic mechanisms for deployment, maintenance, and scaling of applications.“

released into OpenSource in June

very early stage

IBM, Mesosphere, Red Hat promise to support it

Mesos promise support as Framework ontop of Mesos

„Main contribution: abstractions for system engineering which are proven within Google“

runs on CoreOS with etcd

runs Docker containers

Page 40: Beyond static configuration

Development App operation will split into - platform operation - app operation

Platform will be more&more off-the-shelf or PaaS

Page 41: Beyond static configuration

Deis – Heroku-style PaaS on top of CoreOS, prototype on Mesos

• Still in (quite) early development • depends a lot on CoreOS,

e.g. on Fleet’s non-existing scheduling • similar project: Flynn

Page 42: Beyond static configuration

Where are !- classical configuration management, - IaaS solutions like OpenStack, - monolithic PaaS like OpenShift and CloudFoundry, - classical Linux distributions?

Page 43: Beyond static configuration
Page 44: Beyond static configuration

Exciting times