dockercon - the missing piece : when docker networking unleashes software archtecture 2.0
TRANSCRIPT
The missing piece: when Docker networking unleashes software architecture 2.0A. Blind
DevOps coachSociete Generale@adrienblind
L. GrangeauSolutions architect
Finaxys@laurentgrangeau
Agenda
2 - StartersDocker networking & volume features discovered
4 - DessertTaste-an-app
1 - ApetizerBack on current Docker paradigms
3 - Main courseApplication architecture shifts
Back on current docker paradigms
Back on Docker paradigms‘’A universal, self-sufficient and standard artifact embedding an app
module, and its subsequent infrastructure configuration’’
Immutable
Versionned
Light
Portable
Disposable
Programatic
Social
Incremental
It’s mainly focused on enclosing computingcapabilities: what about storage ? Network ?
Networking & volume features discovered
Docker networking
Docker networking
Docker networkingThe Container Network Model (CNM)
A docker contai-ner
Endpoint
A docker contai-ner
Endpoint
A docker contai-ner
EndpointEndpoint
Network sandbox Network sandbox Network sandbox
Front net-work
Back net-work
Docker networking
$ docker network create mynetwork
5000dec7c180a63d87031de7e6bfcf2b25cf1e5daef6338f16fbd4451210a938
$ docker network create –d overlay multihostnetwork
e6537b859359843bc02392245ab226070f79dbf87be2d492969c843f89fb6de6
Docker networking$ docker network inspect mynetwork[ { "Name": "mynetwork", "Id": "5000dec7c180a63d87031de7e6bfcf2b25cf1e5daef6338f16fbd4451210a938", "Scope": "local", "Driver": "bridge", "IPAM": { "Driver": "default", "Config": [ {} ] }, "Containers": {}, "Options": {} }]
Docker networking
Host Host Host Host
SDN
s
SDN 1
SDN 2
SDN 3
Docker networking
Docker networking
Docker Compose evolved to embrace new networking features
$ docker-compose --x-networking --x-network-driver=overlay up
$ docker-compose up
Docker volumes
Docker volumes
Host file system Host file system
‘’Former data management locked in a host’’
Docker volumes
Host file system
Container
Volume
‘’Containers mount a volume which may be backed externaly’’
Docker volumes
$ docker volume create –d volplugin --name pool/name
Cf872ca21d27843f6b6319ac1a34390dd38d94ed4649cd985456d523fb05d4cc
$ docker run –d –p 8080:8080 –v pool/name:/var/jenkins_home jenkins
96aec6f4e45e050dfb4f75a1009e7f105bced5b406752e62d470615d07348b07
Docker volumes$ docker volume lsDRIVER VOLUME NAMElocal cf872ca21d27843f6b6319ac1a34390…local f19f50251f48c64a6b33a5c637c2330…
$ docker volume inspect cf872ca21d27843f6b6319ac1a34390dd38d94…[ { "Name": "cf872ca21d27843f6b6319ac1a34390dd38d94…", "Driver": "local", "Mountpoint": "/mnt/sda1/var/lib/docker/volumes/[…]/_data" }]
Take-away
Application
Compute(Run containers)
Docker building blocks
Application
Compute(Run containers)
Storage(Volumes)
‘’Immutability of containers, resiliency & scalability led to data externalization in separate objects’’
‘’Fine-granularity of containers led to closely interconnect them’’
Application
Compute(Run containers)
Storage(Volumes)
Transport(Network)
‘’The whole topology can nowbe described’’
Application
Compute(Run containers)
Storage(Volumes)
Transport(Network)
Topology(Compose)
‘’Docker finally shifted toobject-oriented infra. architecture’’
Application
Compute(Run containers)
Storage(Volumes)
Transport(Network)
Topology(Compose)
CaaS platform (Swarm, Machine...)
Application architecture shifts
Security paradigms shifts
Security paradigms shifts
Your IT opens up• Externalization (housing, hosting)• Cloud (IaaS/PaaS/SaaS)
Open up your IS• B2B, services exposition• Multi tenancy
More & more breaches appears in your Great Wall of China!
Security paradigms shiftsThe necessary porosity of your IS requires to stick
security closer to each application: sandbox your appsand expose protected interfaces!
Network is part of application topology Security is an app topic, not just infra.
concern Onboard security in feature
teamSecDevOps
Network paradigms shifts
VMVMVM
VMVMVM
VMVMVM
Internet
InternetDMZ
Physical overviewLogical overview
Tenant#1
Tenant#2
LAN
LAN
DMZ1
DMZ2
Traditional networks relies a lot on low layers (L2, etc.)
Application topologies are quite different from physical ones
Network paradigms shiftsSDNs propose network solutions
embracing cloud paradigmsMassively multi-tenant
Thousands tenants, massively scalable
Easy & fast (de)provisioningInfra as code, API centric
Infrastructure agnosticL3, does not stick with lower levels (physical designs, vlans & co)Decouple infrastructure & tenants lifecyclesCross technology, vendor agnostic
From Enterprise Services busesto full-mesh topologies
ESB
Ser-vice
Ser-vice
Ser-vice
Ser-vice
Ser-vice
>Ser-vice
Ser-vice
Ser-vice
Ser-vice
Ser-vice
Micro services
Fine-grained, highly decoupled and atomic purpose centric services
Designedfor failure
Multi-versioned
Scalable
Micro services
Stateless
Share-nothingImmutable
Continuouslydelivered
Distributed
Serviceconsumer
Serviceprovider
Regis-try2. Find 1. Publish
3. Bind
Leverage on a Service registry to discoverwhere are services located
Micro services
Resilience & scalability: apps problem now!
Vertical >horizontal
Apps designed for failure & scalability
Data to be externalized
Dumber infrastructure
Structured: MongoDB, Hadoop, Cassandra, Elastic Search... Binaries: object storage with Ceph, OpenStack Swift...
Helpful patterns: stateless, multi-versioning, loose coupling...
Infrastructure rationalization Low-cost, poor-SLA commodity
« Organizations which design systems... are constrained to produce designs which are copies of the communication structures of these organizations ». - M. Conway, 1968
Consider shifting your organization if you wish to shift your architec-
ture
Forget about the central architects myth of organizing, integrating everything
Consider changing your organization to ex-pect changing the architecture! promote fea-ture teams
Organization
Docker suits perfectly new applications challenges
Create docker networks to isolate applicationsDocker container properties fits micro-services
challengesResilience & scalability is mostly about multiplying
containers
Expect to discuss roles shift in organization
Taste-an-app
Execution platform
Registry
Docker-machine Docker-swarm
The registratordiscovers newcontainers and
feeds the registry
Application design
Provider micro serviceConsumersThe python app module exposes a REST service
searching information in the MongoDBThe NGINX reverse proxy forward app. requests on
one of the python instance registered in Consul
Find
Application topology & runtime
The whole application topology is stored as:docker-compose yaml filedocker-compose args (aka --x-networking & --x-
network-driver)
You can scale up or down the python instances of the micro-service using traditionnal docker-compose scale command
Network view
Only the load balancer VIP is exposed externallyA WAF instance could secure this entrypoint
SDN « dockerconeu15 »
Host network
Provider micro serviceConsumers
Network view - advancedProvider micro service
ConsumersSDN « front »
SDN « back »
Host network
Back
Middle
Front
‘’To enhance securityyou may decoupleeach application tier’’
Zoom on the registry usages
Between micro-services, consumers asks the registry where a desired micro-service is located
Inside a micro-service, NGINX is made aware of the backend API instances available, via the registry
At container level, the registrator enable to registers any container instances, grouped per type
At infrastructure level, the registry is used by swarm (internally) to be aware of the cluster’s participants
Noticed the different usages of a registry ?
You may consider using different registries for each usage : for example an internal registry for the micro service internal topology
Conclusions
Docker shifted from universal containers to object-oriented infrastructure
Security is an app concern
Software is eating the world: application architecture is the key, infrastructure is commodity
