when docker engine 1.12 features unleashes software architecture
TRANSCRIPT
voxxeddays.com/luxembourg/ #voxxeddaysLU
When Docker Engine 1.12 features unleashes software architecture
[Update of Dockercon EU’ 15 presentartion initially built w/ L. Grangeau]
Adrien Blind@adrienblind
voxxeddays.com/luxembourg/ #voxxeddaysLU
Back on Docker paradigms‘’A universal, self-sufficient and standard artifact embedding an app module,
and its subsequent infrastructure configuration’’
It’s mainly focused on enclosing computingaspects of the app: what about persistence? communication? Topologies?
Immutable
Portable
Lightweight
Incremental
Versionned
Disposable
voxxeddays.com/luxembourg/ #voxxeddaysLU
1 - StartersApplication architecture shifts
3 - DessertTaste-an-app
2 - Main courseDocker networking, service & volume features discovered
voxxeddays.com/luxembourg/ #voxxeddaysLU
Application architecture shifts
voxxeddays.com/luxembourg/ #voxxeddaysLU
Security paradigms shifts
voxxeddays.com/luxembourg/ #voxxeddaysLU
Security paradigms shifts
Your IT opens up• Externalization (housing, hosting)• Cloud (IaaS/PaaS/SaaS)
Open up your IS• B2B, services exposition• Multi tenancy
More & more breaches appears in your Great Wall of China!
voxxeddays.com/luxembourg/ #voxxeddaysLU
Security paradigms shiftsThe necessary porosity of your IS requires to stick security closer to each application:
sandbox your apps and expose protected interfaces (ciphered/auth/authz)!
• Network is now part of application topology • Security is an app topic, not just infra. Concern• Onboard security in feature teamSecDevOps
voxxeddays.com/luxembourg/ #voxxeddaysLU
Fine-grained, highly decoupled and atomic purpose centric services
Multi-versioned
ScalableStateless
Share-nothing
Immutable
Continuouslydelivered
Distributed
Micro services
voxxeddays.com/luxembourg/ #voxxeddaysLU
From Enterprise Services buses to full-mesh topologies
ESB
Ser-vice
Ser-vice
Ser-vice
Ser-vice
Ser-vice
>Ser-vice
Ser-vice
Ser-vice
Ser-vice
Ser-vice
Micro services
voxxeddays.com/luxembourg/ #voxxeddaysLU
Serviceconsumer
Serviceprovider
Regis-try2. Find 1. Publish
3. Bind
Leverage on a Service registry to discoverwhere services are located
Micro services
voxxeddays.com/luxembourg/ #voxxeddaysLU
Resilience & scalability: apps problem now!
Vertical > horizontal• Apps designed for failure & scalability
• Data to be externalized
• Dumber infrastructure
Structured: MongoDB, Hadoop, Cassandra, Elastic Search... Binaries: object storage with Ceph, OpenStack Swift...
Helpful patterns: stateless, multi-versioning, loose coupling...
Infrastructure rationalization Low-cost, poor-SLA commodity
voxxeddays.com/luxembourg/ #voxxeddaysLU
« Organizations which design systems... are constrained to produce designs which are copies of the communication structures of these organizations ». - M. Conway, 1968
Consider shifting your organization if you wish to shift your architec-
ture!
•Forget about the central architects myth of organizing, integrating everything
•Promote feature teams
Organization
voxxeddays.com/luxembourg/ #voxxeddaysLU
Networking, Service & Volume features discovered#Networking
voxxeddays.com/luxembourg/ #voxxeddaysLU
Docker networkingThe Container Network Model (CNM)
A docker contai-ner
Endpoint
A docker contai-ner
Endpoint
A docker contai-ner
EndpointEndpoint
Network sandbox Network sandbox Network sandbox
Front net-work
Back net-work
voxxeddays.com/luxembourg/ #voxxeddaysLU
Docker networking
Host Host Host Host
SDN 1 / App 1
SDN 2 / App 2
SDN 3 / App 3
voxxeddays.com/luxembourg/ #voxxeddaysLU
Docker networking
$docker network create --driver overlay appBo51qvm381rrf6obchh6uw4w7
$docker network lsNETWORK ID NAME DRIVER SCOPEbo51qvm381rr app overlay swarm
voxxeddays.com/luxembourg/ #voxxeddaysLU
Docker networking$ dockerdocker network inspect app[ { "Name": "app", "Id": "bo51qvm381rrf6obchh6uw4w7", "Scope": "swarm", "Driver": "overlay", "EnableIPv6": false, "IPAM": { "Driver": "default", "Options": null, "Config": [] }, "Internal": false, "Containers": null, "Options": { "com.docker.network.driver.overlay.vxlanid_list": "258" }, "Labels": null }]
voxxeddays.com/luxembourg/ #voxxeddaysLU
docker-compose evolved to embrace new networking features:version: '2'services: wordpressapp: image: wordpress: latest networks: - wordpressnet wordpressdb: image: mysql:latest networks: - wordpressnetnetworks: wordpressnet: driver: overlay
Docker networking
voxxeddays.com/luxembourg/ #voxxeddaysLU
Networking, Service & Volume features discovered#Service/Task/Stack
voxxeddays.com/luxembourg/ #voxxeddaysLU
Docker service, tasks, stackDocker engine 1.12 introduces a complete new swarm orchestration framework
• Built in the engine• Decentralized• More secured• More resilient
$docker node lsID NAME MEMBERSHIP STATUS AVAILABILITY MANAGER STATUS0cdxzmgi1a[...] m1 Accepted Ready Active Leader4wz1zlur5c[...] * m3 Accepted Ready Active Reachablea4v6da1yre[...] m2 Accepted Ready Active Reachable
NEWEngine 1.12
voxxeddays.com/luxembourg/ #voxxeddaysLU
Docker serviceDepicts the desired runtime behavior of a given image : networking, resiliency, quotas... shift to state-machine paradigms
$docker service create --name front -–network app –replicas 3 -p 80:80/tcp nginx:latest
$docker service lsID NAME REPLICAS IMAGE COMMAND9gxxdqpauq08 front 3/3 nginx:latest
$docker service scale front=10Front scaled to 10
Attach the containers to a given networkDefine the desired amount of instances for this service (named « tasks »)Attach each instance to a transversal L4 loadbalancer instance, reachable on each node of the cluster
voxxeddays.com/luxembourg/ #voxxeddaysLU
Docker tasksThe Swarm cluster schedules creation/deletion of tasks (aka containers) to meet the desired state described in the service
$docker service tasks front
ID NAME SERVICE IMAGE LAST STATE DESIRED NODE
7yu4rgc23[...] front.1 front nginx:latest Running 8 hours Running m3
dj4trimu4[...] front.2 front nginx:latest Running 8 hours Running m2
7rdiv2r2e[...] front.3 front nginx:latest Running 7 hours Running m1
Should a task or a cluster node fails, swarm will fire new tasks to meet the desired state
voxxeddays.com/luxembourg/ #voxxeddaysLU
Docker stack
docker-compose enabled to trigger a complete containerized app topology and its network conf., piloted from the client side
docker-compose bundle enables to create an artifact depicting this topology, which can be consumed server-side
docker stack deploy enables to pop up the desired topology at runtime. The state-machine behavior of swarm then ensures this topology to be maintained
Experimental
voxxeddays.com/luxembourg/ #voxxeddaysLU
Networking, Service & volume features discovered#Volume
voxxeddays.com/luxembourg/ #voxxeddaysLU
Docker volumes
Host file system Host file system
‘’Former data management locked in a host’’
• No persistent data inside app containers
• Object storage: OpenStack Swift, Ceph, Amazon S3…
• Consider use of Docker Volumes if you need to manage binaries on app container filesystem
• Handle configuration at run time
voxxeddays.com/luxembourg/ #voxxeddaysLU
Docker volumes
Host file system
Container
Volume
‘’Containers mount a volume which may be backed externally’’
voxxeddays.com/luxembourg/ #voxxeddaysLU
Docker volumes$ docker volume create –d volplugin --name pool/nameCf872ca21d27843f6b6319ac1a34390dd38d94ed4649cd985456d523fb05d4cc
$ docker run –d –p 8080:8080 –v pool/name:/var/jenkins_home jenkins96aec6f4e45e050dfb4f75a1009e7f105bced5b406752e62d470615d07348b07
voxxeddays.com/luxembourg/ #voxxeddaysLU
Docker volumes$ docker volume lsDRIVER VOLUME NAMElocal cf872ca21d27843f6b6319ac1a34390…local f19f50251f48c64a6b33a5c637c2330…
$ docker volume inspect cf872ca21d27843f6b6319ac1a34390dd38d94…[ { "Name": "cf872ca21d27843f6b6319ac1a34390dd38d94…", "Driver": "local", "Mountpoint": "/mnt/sda1/var/lib/docker/volumes/[…]/_data" }]
voxxeddays.com/luxembourg/ #voxxeddaysLU
Networking, Service & Volume features discoveredFrom infrastructure containers to a complete object-oriented app
voxxeddays.com/luxembourg/ #voxxeddaysLU
Distributed applicationCompute (service/task)
Storage (volume) Transport (network)
Topology(compose, bundle,
deploy, stack)
Docker shifted from container infra. to object-oriented app. topologies
CaaS platformClustering (swarm)
Image mgmt(registry) Hosting (node)
Provisioning (machine)
... relying on an CaaS platform
voxxeddays.com/luxembourg/ #voxxeddaysLU
Taste-an-app
voxxeddays.com/luxembourg/ #voxxeddaysLU
AppConsumers
•The NGINX reverse proxy forward app. requests on one of the python instance registered in Consul
Find
Application design in nov’ 15Registrator agents makes Consul aware of popped up containers
voxxeddays.com/luxembourg/ #voxxeddaysLU
Application design with Engine 1.12
AppConsumers
Routing layer
Any swarm node IP address
loadbalances to all WP instances
•Load balancing is now directly achieved at engine/swarm level
voxxeddays.com/luxembourg/ #voxxeddaysLU
3 command lines to heavendocker network create --driver overlay wordpressnet
docker service create --env MYSQL_ROOT_PASSWORD=plop --env MYSQL_DATABASE=wordpress --network wordpressnet --replicas 1 --name wordpressdb mysql:latest
docker service create --env WORDPRESS_DB_HOST=wordpressdb --env WORDPRESS_DB_PASSWORD=plop --network wordpressnet --replicas 4 --name wordpressapp --publish 80:80/tcp wordpress:latest
... Or even, even better: use docker-compose bundle & deploy through CI
voxxeddays.com/luxembourg/ #voxxeddaysLU
Between apps, consumers may asks a service discovery where a desired micro-service is located
Docker now exposes multi-instanciated services, leveraging on IPVS load balancing and internal service discovery
The app. may use internally its own service service discovery to get more higher control (Java ex.: Spring Cloud stack with Zuul/Eureka)
At infrastructure level, an internal service discovery is used by swarm
Noticed the different usages of a service discovery & name resolution mechanism?
Zoom on various service discovery usages
voxxeddays.com/luxembourg/ #voxxeddaysLU
Conclusion
voxxeddays.com/luxembourg/ #voxxeddaysLU
Conclusion• Software is eating the world: application
architecture is the key, infrastructure is commodity
• Security is an app concern
• Docker shifted from universal containers to object-oriented app. architecture