dockercon eu 2015: the missing piece: when docker networking unleashing soft architecture 2.0
TRANSCRIPT
The missing piece: when Docker networking unleashes software architecture 2.0
A. BlindDevOps coach
Société Générale
@adrienblind
L. GrangeauSolutions architect
Finaxys
@laurentgrangeau
Agenda
2 - Starters
Docker networking
& volume features
discovered
3 - Dessert
Taste-an-app
1 - Apetizer
Back on current
Docker paradigms
3 - Main course
Application
architecture shifts
Back on current
docker paradigms
Back on Docker paradigms
‘’A universal, self-sufficient and standard artifact embedding an app
module, and its subsequent infrastructure configuration’’
Immutable
Versionned
Light
Portable
Disposable
Programatic
Social
Incremental
It’s mainly focused on enclosing computingcapabilities: what about storage ? Network ?
Networking & volume
features discovered
Docker networking
Docker networking
Docker networking
The Container Network Model (CNM)
A docker container
Endpoint
A docker container
Endpoint
A docker container
EndpointEndpoint
Network sandbox Network sandbox Network sandbox
Front network Back network
Docker networking
$ docker network create mynetwork
5000dec7c180a63d87031de7e6bfcf2b25cf1e5daef6338f16fbd4
451210a938
$ docker network create –d overlay multihostnetwork
e6537b859359843bc02392245ab226070f79dbf87be2d492969c84
3f89fb6de6
Docker networking
$ docker network inspect mynetwork
[
{
"Name": "mynetwork",
"Id":
"5000dec7c180a63d87031de7e6bfcf2b25cf1e5daef6338f16fbd4451
210a938",
"Scope": "local",
"Driver": "bridge",
"IPAM": {
"Driver": "default",
"Config": [
{}
]
},
"Containers": {},
"Options": {}
}
]
Docker networking
Host Host Host Host
SD
Ns
SDN 1
SDN 2
SDN 3
Docker networking
Docker networking
Docker Compose evolved to embrace
new networking features
$ docker-compose --x-networking
--x-network-driver=overlay up
$ docker-compose up
Docker volumes
Docker volumes
aufs/overlayfs/btrfs
Docker volumes
Docker volumes
$ docker volume ls
DRIVER VOLUME NAME
local cf872ca21d27843f6b6319ac1a34390…
local f19f50251f48c64a6b33a5c637c2330…
$ docker volume inspect
cf872ca21d27843f6b6319ac1a34390dd38d94…
[
{
"Name": "cf872ca21d27843f6b6319ac1a34390dd38d94…",
"Driver": "local",
"Mountpoint":
"/mnt/sda1/var/lib/docker/volumes/[…]/_data"
}
]
Docker volumes
$ docker volume create –d volplugin --name pool/name
Cf872ca21d27843f6b6319ac1a34390dd38d94ed4649cd985456d5
23fb05d4cc
$ docker run –d –p 8080:8080 –v
pool/name:/var/jenkins_home jenkins
96aec6f4e45e050dfb4f75a1009e7f105bced5b406752e62d47061
5d07348b07
Take-away
Application
Compute
(Run containers)
Docker building blocks
Application
Compute
(Run containers)
Storage
(Volumes)
‘’Immutability of containers led to data
Externalization in separate objects’’
‘’Fine-grained granularity of containers led
to closely interconnect them’’
Application
Compute
(Run containers)
Storage
(Volumes)
Transport
(Network)
‘’The whole topology can now
be described’’
Application
Compute
(Run containers)
Storage
(Volumes)
Transport
(Network)
Topology
(Compose)
‘’Docker finally shifted to
object-oriented infra. architecture’’
Application
Compute
(Run containers)
Storage
(Volumes)
Transport
(Network)
Topology
(Compose)
CaaS platform (Swarm, Machine...)
Application
architecture shifts
Security paradigms shifts
Security paradigms shifts
Your IT opens up• Externalization
• Cloud (IaaS/PaaS/SaaS)
Open up your IS• B2B, services exposition
• Multi tenancy
More & more breaches appears in your Great Wall of China!
Security is an app feature, not just an outer infra. concern
Onboard security guys in your feature team SecDevOps
Security paradigms shifts
The necessary porosity of your IS requires to stick
security closer to each application: sandbox your appsand expose protected interfaces!
Network is part of application topology
Authentication & authorization is the key
Network paradigms shifts
VM
VM
VM
VM
VM
VM
VM
VM
VM
Internet
Internet
DMZ
Physical overview
Logical overview
Tenant
#1
Tenant
#2
LAN
LAN
DMZ1
DMZ2
Traditional networks
relies a lot on low layers
(L2, etc.)
Application topologies
are quite different from
physical ones
Network paradigms shifts
SDNs proposes network solutions embracing
cloud paradigms
Massively multi-tenant
Thousands tenants, massively scalable
Easy & fast (de)provisioning
Infra as code, API centric
Infrastructure agnostic
L3, does not stick with lower levels (physical designs, vlans & co)
Decouple infrastructure & tenants lifecycles
Cross technology, vendor agnostic
From Enterprise Services buses
to full-mesh topologies
ESB
Service Service Service
Service Service
>ServiceService
Service
Service
Service
Micro services
Fine-grained, highly decoupled and
atomic purpose centric services
Designed
for failure
Multi-versioned
Scalable
Micro services
Stateless
Share-nothing
Immutable
Continuously
delivered
Distributed
Service
consumer
Service
provider
Registry
2. Find 1. Publish
3. Bind
Leverage on a Service registry to discover
where are services located
Micro services
Resilience & scalability: apps problem now!
Vertical > horizontal
Dumber infrastructure
Apps designed for failure & scalability
Data to be externalized Structured: MongoDB, Hadoop, Cassandra, Elastic Search...
Binaries: object storage with Ceph, OpenStack Swift...
Helpful patterns: stateless, share nothing, loose coupling...
Infrastructure rationalization
Low-cost, poor-SLA commodity
« Organizations which design systems... are constrained to
produce designs which are copies of the communication structures
of these organizations ». - M. Conway, 1968
Consider shifting your organization if you
wish to shift your architecture
Forget about the central architects myth of
organizing, integrating everything
Consider changing your organization to expect
changing the architecture! promote feature teams
Organization
Docker suits perfectly new applications
challenges
Create docker networks to isolate applications
Docker container properties fits micro-services challenges
Resilience & scalability is mostly about multiplying containers
Expect to discuss roles shift in organization
Taste-an-app
Execution platform
Registry
Docker-machineDocker-swarm
The registrator
discovers new
containers and
feeds the registry
Application design
Provider micro serviceConsumers
The python app module exposes a REST service searching
information in the MongoDB
The NGINX reverse proxy forward app. requests on one of the
python instance registered in Consul
Find
Application topology & runtime
The whole application topology is stored as:
docker-compose yaml file
docker-compose args (aka --x-networking & --x-network-driver)
You can scale up or down the python instances of the micro-
service using traditionnal docker-compose scale command
Network view
Only the load balancer VIP is exposed externally
A WAF instance could secure this entrypoint
SDN « myapp »
Host network
Provider micro serviceConsumers
Network view - advancedProvider micro service
ConsumersSDN « front »
SDN « back »
Host network
Back
Middle
Front
‘’To enhance security
you may decoupleeach application tier’’
Zoom on the registry usages
At infrastructure level, the registry is used by swarm
(internally) to be aware of the cluster’s participants
At container level, the registrator enable to registers
any container instances, grouped per type
At application level, the consumers asks the registry
where the micro-service (the NGINX front-end) is located
Noticed the three different usages of the registry ?
You may consider using different registries for each usage : for example an
internal registry for the micro service internal topology
Conclusions
Docker shifted from universal containers to
object-oriented infrastructure
Security is an app concern
Software is eating the world: application
architecture is the key, infrastructure is commodity
