flyingcloud: building docker containers with saltstack

Copyright © MetaBrite, Inc. 2016

FlyingCloudBuilding Docker Containers

with SaltStack

Adam Feuer and George V. Reilly


The Story of Lectern

• Lectern Web Service• Receives uploaded images of receipts from CookBrite mobile app• OCRs images using Tesseract + OpenCV• Extracts data about purchases

• Deployed on AWS ElasticBeanstalk

• Could not get app to build on Amazon Linux


Lectern v1

• ElasticBeanstalk also supports Docker

• We built a Docker app on top of an Ubuntu image

• Solved the build problem. Sort of.

• Our build was a twisted, unmaintainable, monolithic mess of shell scripts.

• Took an hour to build. Any failure reset the clock.


Lectern's old Docker image setup scripts


Lectern v2: with FlyingCloud


One Big Idea

• Use Configuration Management to build Docker images• Configure rather than write code• More understandable• Reusable and repeatable


What is Docker?

• Lightweight Virtualization

• Run multiple containers on one host system• Isolated process group for each container• Shared kernel• Run a different Linux OS in each container

• Image = filesystem snapshot

• Container = running image


What is SaltStack?

• Configuration management system• "Automated system administration"

• Configuration, not scripts

• Declarative, not imperative

• Salt is implemented in Python• Though you usually write Salt States in YAML

• We use Salt Masterless mode• Configure system at build time; no runtime dependency on Salt


Two Ways to Build Docker Images

• Traditional• Dockerfile: static list of commands• Run commands inside container, then commit

• FlyingCloud • Use SaltStack to update state of Docker image• Easy to make layers• Easy to run in Continuous Integration


Example App - layers overview


Example App - project and base layers


Example App - app layer


Example App - testrunner layer


FlyingCloud Demo

See https://github.com/cookbrite/flyingcloud/tree/master/examples/flask-web-app

FlyingCloud

● Use Configuration Management to build layered Docker images

● An FC Layer can be made from any of these:○ A Dockerfile, Salt states, or custom Python code○ Do not confuse with Docker’s layer filesystem

● Layers are buildable and/or runnable● Can be pushed to or pulled from a Docker repository


Kubernetes

• Google’s open source platform that automates deployment, scaling, and operations of application containers built on Docker images across clusters of hosts

• Our Kubernetes workflow for new services:• Build Docker images with FlyingCloud• Configure Kubernetes cluster• Test cluster in Continuous Integration• Deploy cluster to AWS


FlyingCloud is Open Source

• Source: https://github.com/cookbrite/flyingcloud• Docs: http://flyingcloud.readthedocs.org • PyPI: https://pypi.python.org/pypi/flyingcloud/ • Mail: https://groups.google.com/group/flyingcloud-users • Apache License

Contributors welcome! Especially:

• SaltStack users• Ansible users

https://github.com/cookbrite/flyingcloud

http://flyingcloud.readthedocs.org

https://pypi.python.org/pypi/flyingcloud/

https://groups.google.com/group/flyingcloud-users


We are organizing the world's product information

We’re Hiring! Located in Pioneer Square.• Back-end devs. We're a Python shop.• Android devs• iOS devs• Data scientists

Contact

• [email protected] or [email protected]

flyingcloud: building docker containers with saltstack

Software