why should i care about stateful containers?
TRANSCRIPT
Why should I care about stateful containers?
Luke MarsdenCTO, ClusterHQ
@lmarsden @clusterhq
Microservices are smashing up monolithic databases
Many more database instances, many more flavours
prod
staging in cloud 2
DR in cloud 3
hosted CIdev
laptop
Aren’t containers stateless?
12 factor apps
One platform
Dev/prod parity
Cloud services are effectively closed source
Portability
OK, who’s in?
Cattle vs. petsDeep dive
compute
local storage
infrastructure evolution1990s
servers with disks in
Servers Volumes
Pets Pets
Cattle Pets
Cattle Cattle
SAN storage
VM VM VM
infrastructure evolution2000s
vol vol vol
VMware hosts
virtualisation and shared storage
Servers Volumes
Pets Pets
Cattle Pets
Cattle Cattle
EBS
VM VM VM
infrastructure evolution2010s
cloudVMs no longer HA
vol vol vol
EC2Servers Volumes
Pets Pets
Cattle Pets
Cattle Cattle
infrastructure evolution2010s
computelocal storage
on-prem “hyperconverged” or “SDS”eg OpenStack + Ceph or ScaleIO
VM
vol vol
Servers Volumes
Pets Pets
Cattle Pets
Cattle Cattle
infrastructure evolution2010s
computelocal storage
distributed databasesbare metal / cloud VMs + local storage
db
Servers Volumes
Pets Pets
Cattle Pets
Cattle Cattle
db
infrastructure evolution2010s
computelocal storage
distributed databases on SDS
vol vol
Servers Volumes
Pets Pets
Cattle Pets
Cattle Cattle
db db
infrastructure evolution2015
computelocal storage
distributed databases on SDSin container clusters
vol vol
Servers Volumes
Pets Pets
Cattle Pets
Cattle Cattle
db db
container scheduler
Servers Volumes What is it?
Pets Pets just crap
Cattle Pets storage orchestration
Cattle Cattleephemeral
stateful containers
Pet ~= HA provided by infrastructure/platformCattle ~= HA provided by application
Pros & consStorage orchestration• Run any database/stateful app• Storage layer responsible for
data resilience• Ops independent of database
types• Works within a storage zone
Ephemeral stateful containers• Requires distributed database(s)• Database-defined data resilience• Ops requires understanding each
data service• Can span storage zones
Ephemeral stateful
Storage orchestration
Why should data be pets when servers are cattle?
Relocatable
EBS/SDS/SAN…
Stateless things scale outStateful things scale up
Container can be a shard
You can mix and match stateful + stateless in a
container cluster
Use case: rebalance cluster
storagev1
compute
containerhost
(VM orbare metal)
c1 c2
container composer + scheduler + SD
v2
Use case: rebalance cluster
storagev1
compute
containerhost
(VM orbare metal)
c1 c2
v2
container composer + scheduler + SD
Use case: drain a node
storagev1
compute
containerhost
(VM orbare metal)
c1 c2
v2 v3
c3 c4
v4
container composer + scheduler + SD
Use case: drain a node
storage
compute
containerhost
(VM orbare metal)
v3
c3 c4
v4
v1
c1 c2
v2
container composer + scheduler + SD
Use case: drain a node
storage
compute
containerhost
(VM orbare metal)
v3
c3 c4
v4
v1
c1 c2
v2
container composer + scheduler + SD
Use case: HA when node fails
storagev1
compute
containerhost
(VM orbare metal)
c1
container composer + scheduler + SD
Use case: HA when node fails
storagev1
compute
containerhost
(VM orbare metal)
container composer + scheduler + SD
Use case: HA when node fails
storagev1
compute
containerhost
(VM orbare metal)
c1
container composer + scheduler + SD
Use case: speed up distributed database recovery
storagev1
compute
containerhost
(VM orbare metal)
c1
container composer + scheduler + SD
c2
container overlay network
v2
10.0.0.1 10.0.0.2
c3
v3
10.0.0.3
Use case: speed up distributed database recovery
storagev1
compute
containerhost
(VM orbare metal)
c1
container composer + scheduler + SD
c2
container overlay network
v2
10.0.0.1 10.0.0.2
c3
v3
10.0.0.3
Use case: speed up distributed database recovery
storagev1
compute
containerhost
(VM orbare metal)
container composer + scheduler + SD
c2
container overlay network
v2
10.0.0.2
c3
v3
10.0.0.3
Use case: speed up distributed database recovery
storagev1
compute
containerhost
(VM orbare metal)
container composer + scheduler + SD
c2
container overlay network
v2
10.0.0.2
c3
v3
10.0.0.3
c110.0.0.1
Connecting storage to containers
Try Flocker todayFor stateful microservices
github.com/clusterhq/flocker
We are hiring in Bristol (1 hour hop from Schiphol)+ SF Bay Area!
clusterhq.com/careers