build applications like google · kubernetes glossary (part 1) master: the managing machine, which...
TRANSCRIPT
Build applications like Google: Using containers, Kubernetes and Mesos
CloudOpen 2014, DüsseldorfBernd Mathiske, MesosphereNat Welch, Google
1. Keep programs simple
2. Prefer smaller, stateless programs
Images by Connie Zhou
A 2000-machine cluster will have >10 machine crashes per day
DRAM errors (1% AFR)Disk failures (2-10% AFR)Machine crashes (~2/year)OS upgrades (2-6/year)
Agenda● Hardware => GCE
● Containers => Docker
● Cluster management => Mesos
● Container management => Kubernetes
● Platform stacks for large distributed apps
● How to get started
Google Compute Engine
● VMs as service
● Europe, Asia, US
● Live migration
Problems with raw VMs
● Expensive to turn up (time)
● "Hard" to make repeatable
● Difficult to replicate locally for testing
● High possibility of failure
Container
● Lightweight Linux execution environment○ libcontainer, LXC, LMCTFY
● Static application composition○ Reliable deployment
● Unit of resource isolation○ Multi-tenancy without heavyweight VMs
Docker● Open source
● Configurable layers
● Reproducible
● Version-controlled
● Lots and lots of other peoples containers○ https://registry.hub.docker.com/
Run everything in containers!
Mesos facts● Supports Docker
● Scales to 10,000s of nodes
● Top-level Apache project
● Twitter, Airbnb are major users and contributors
● APIs for C++, Python, JVM-languages, Go, ...
● Pluggable CPU, memory, IO isolation
● Highly configurable, with easy-to-use defaults
● Packages and support through Mesosphere
How Mesos worksApplication
Framework
Mesos Slave
Executor
Task
Task
TaskExecutor
Scheduler
Mesos Master
Mesos MasterMesos
Master
Zookeeper
Mesos Master
Kubernetes● Employs and promotes Docker
● Simple, portable, extensible modular
framework for container-based service
app management
● Patterned after internal systems in Google that
manage internet-scale workloads
● Written in Go
github.com/GoogleCloudPlatform/kubernetes
Kubernetes glossary (part 1)
● Master: the managing machine, which oversees one or more minions.
● Minion: a slave that runs tasks as delegated by the user and Kubernetes master.
● Pod: an application that runs on a minion.
Kubernetes glossary (part 2)● Replication controller: Verifies everything
that should be running is running.● Label: an arbitrary key/value pair that the
Replication Controller uses for service discovery
● kubecfg: the config tool● Service: an amalgamation of similar pods
https://raw.githubusercontent.com/GoogleCloudPlatform/kubernetes/master/docs/architecture.png
Kubernetesoverview
Connection points
● Kubernetes Pod → Mesos TaskGroup● Pod Labels → Mesos TaskGroup Labels, Task Labels● LabelSelectors ↝ Mesos Label Service
● ReplicationController Marathon● Kubernetes Service ↝ Marathon service discovery
Kubernetes-Mesos framework overview
Kubernetes-Mesos master and scheduler
Kubernetes-Mesos executors and tasks
What Mesos contributes● Multi-framework: weighted fair sharing, roles, preemption*, node drain*
● Run pods alongside other popular frameworks
(e.g. Spark, Rails, Hadoop, ...)
● Run services and batch apps in the same cluster
● Advanced scheduling: resources, constraints, global view
● High resource availability, cluster self-healing
● Proven at scale, battle-tested in production
● GUI/CLI cluster management console
(* in development)
What Kubernetes contributes● Pods: tasks to co-locate/deploy/isolate/replicate
● Label Service: arbitrary metadata for tasks/pods
● Tightly integrated service discovery solution
● Ability to run Kubernetes API workloads
+ + +
HardwareFault tolerance
PackagingUpdatesIsolation High availability
Service discoveryService aggregation
ScalabilityElasticity
Multi-tenancyBatch jobsUtilization
Google’s Hardware + Linux
Omega Mesos
Kubernetes
Google-like computing for everybody
Batch
Service Batch Service
GCE
Some possible stack variations
Google’s Hardware + Linux
GCE
Mesos
Marathon
Service
BatchService
Kubernetes
More Possible Stack Variations
Other Hardware or Cloud + Linux or CoreOS
Kubernetes Mesos
Service
Service
Portability => no vendor lock-in
Kubernetes
Mesos
Marathon
Service
Batch
google.mesosphere.com
Summary● Complete stack for large distributed apps
● Multi-tenancy
● Resource optimization
● Easy to deploy
● No vendor lock-in
Useful Links● GCE
○ cloud.google.com/compute
● Kubernetes○ github.com/GoogleCloudPlatform/kubernetes
● Mesosphere○ mesosphere.com
● Kubernetes - Mesos○ github.com/mesosphere/kubernetes-mesos
● Docker○ docker.com
Photo Credits● https://secure.flickr.com/photos/icco/7990117774/ ● https://secure.flickr.com/photos/icco/9956577145/ ● http://www.google.co.uk/about/datacenters/gallery/#/all/20● http://www.google.co.uk/about/datacenters/gallery/#/all/12 ● https://secure.flickr.com/photos/icco/14349351930/