3 hours to docker fundamentals - jumpstart your docker knowledge

3 hours to Docker fundamentals: Jumpstart your Docker knowledge

Presented by Aater Suleman

Jay Patel Oct 3, 2014

Today’s Presenters:Aater Suleman Co-founder & CEO Flux7

Faculty, University of Texas at Austin

Flux7: Cloud and DevOps SolutionsCloud and Devops for Web teams

Enterprise DevOps management

AWS Certified Team

Clients:Partners:

Today’s Presenters:Jay Patel Cloud Engineer at Flux7

MS, University of Texas at Austin

Flux7: Cloud and DevOps SolutionsCloud and Devops for Web teams

Enterprise DevOps management

AWS Certified Team

Clients:Partners:

Why Learn Docker?

Fastest growing IT Technology I have seen

in my career

50K+ downloads in less than 30 days of public

introduction

Over 100 Fortune 500s are toying with Docker

already

Docker is impacting the way we code, the way we test, and the way we deliver

Docker is Attracting Investors!

Today’s Agenda

Basic concepts and Docker terminology

Docker Commands

Must know Docker features

Docker in real world

(implemented by

Flux7)

Docker Hands-on Tutorial

Lightweight Portable Cross Cloud Infrastructure

Docker’s attraction lies in its

But, Why Docker?

Sounds familiar?

Ineffective code pipeline management

Inconsistency across environments

Mismatches in dev and prod environments

Resource provisioning takes ages

Increasing bills

Build Once, Configure Once & Run Anywhere

Docker in Docker terms...

Docker Components & Elements

Docker Client is the user interface that allows communication between the user

and the Docker daemon

Docker Daemon sits on the host machine answering requests for services

Docker Index is a centralized registry allowing backup of Docker container

images with public and private access permissions

Docker Components & Elements

Docker Containers are the actual containers running the applications and includes the operating system, user added files, and meta-data

Docker Images are all inclusive images that help launch Docker containers

DockerFile is a file containing instructions that help automate image creation

Layer: Each file system that is stacked when Docker mounts rootfs

Today’s Agenda


Docker Commands



(implemented by

Flux7)


Installing Docker

Let’s discuss how to install Docker in Amazon Linux AMI

Install Docker using a single command:

sudo rpm install docker

Basic Commands

docker pull Pull a pre-built image from the public repos

docker run Run the container in one of 3 modes: Background, Foreground, Interactive

docker logs View the logs of the running job

docker commit Save the container state as an image

docker images Obtain a list of all images

More Docker Commands

docker diff List of changes in files and directories (one of the powerful commands Docker provides)

docker build Build docker images from Dockerfiles

docker inspect Low-level information about containers and images

docker attach Interact with running containers

docker kill Kill the main process of the container

Today’s Agenda


Docker Commands



(implemented by

Flux7)


DockerFile

Automates Image creation process

Set of instructions to create an image

General DockerFile commands’ syntax:

INSTRUCTION argument

DockerFile Commands

MAINTAINER <author name> Set an author field for the image

RUN <command> Execute a command in a shell or exec form

ADD <src> <destination> Copy files from one location to another

CMD["executable","param1","param2"] Provides defaults for an executing container

EXPOSE <port>; Port on which container listens to

DockerFile Commands

ENTRYPOINT [‘executable’,

‘param1’,’param2’]

Configure a container as an executable

WORKDIR /path/to/workdir Set Working Directory

ENV <key> <value> Set environment variables

USER <uid> Set UID for use when running an image

VOLUME [‘/data’] Enable access to a directory from a working container

Docker Hub

Centralized Management of

User Accounts

Image Checksums

Public and Private Docker Repositories

Today’s Agenda


Docker Commands



(implemented by

Flux7)


Docker Hands-on

Installation

Install VMWare from USB stick

Install the Linux image from USB stick

Install Docker

The BusyBox Test

Pull busybox container

Run ls inside of Docker

Docker run

Run with only the main process

Run interactively

Run in daemon mode

Docker diff

Docker commit

Docker create

Using Dockerfile

Tips on Dockerfile

DockerHub

Create a new repo

Docker push

Pushing a container to Dockerhub

Volumes

Create container with 2 volumes

Mount Data volumes into application container

Port Forwarding

Auto-map all exposed ports to host

Binding a port to a host interface

DockerFile

# sshd## VERSION 0.0.1

FROM ubuntuMAINTAINER Thatcher R. Peskens "[email protected]"

# make sure the package repository is up to dateRUN echo "deb http://archive.ubuntu.com/ubuntu precise main universe" > /etc/apt/sources.listRUN apt-get update

RUN apt-get install -y openssh-serverRUN mkdir /var/run/sshd RUN echo 'root:screencast' |chpasswd

EXPOSE 22CMD /usr/sbin/sshd -D

Dockerfile to set up an sshd service in a container

Today’s Agenda


Docker Commands



(implemented by

Flux7)


Docker in Real-World

Simplifying Configuration

Developer Productivity

Multi-Tenancy

Server Consolidation

App Isolation

Code Pipeline Management

Debugging Capabilities

Rapid Deployment

Australia’s 5th largest solar panel installer

Application: Solar Panel Monitoring

Case Study 1: Quick and Easy Multi-tenancy using Docker

Business Problem

Low customer engagement ⇒ limited upsell opportunities

Unhappy customers stuck by large electricity bills

Low collection rates on leased units

Measure power output at solar panels Store data

Create reports

Status Indicator

Monitoring

Mail

Customer 1 Customer 2

Customer n

Controlling

Customer 2Customer 1

Measure power output at solar panels

Customer n

Status Indicator

Monitoring

Mail

Store dataCreate reports

Controlling

Mail

Status Indicator

Monitoring

Mail

Store dataCreate reports

Customer 2Customer 1

Measure power output at solar panels

Customer n

Controlling

Multi-tenancy

Scenario:

The Dev Solution: Add Tenant ID in Code and Database

Tedious Error prone Code

changes

Downsides

The Ops Solution:

New set of servers per tenant

Low Utilization High Cost

Downsides

High Maintenance

Page load time

Why not Leaner Servers?

> 5 seconds

The DevOps Solution:

New Docker containers for every tenant

Inexpensive

Up to 30 tenants per large AWS

instance

Page loads under 1 second

Benefits

Fast

Implemented in under 2 weeks with NO code changes

DevOps

Take Aways

Dev or Ops alone >

Case Study 2: Improved Developer Workflow using Docker

Application: National & International Automotive Listing

Auto.com’s “Big Plan” for Cloud

Push-Button deployment of infrastructure:

Inclusive of everything from

Security

App and Database servers

Load Balancing

SOA

Queues

Data Processing

Local Dev Environment That Matches Prod

The “Holy Grail” of dev environment management

Now possible in all its complexity due to Docker.

Why Developer Workflow?

Web Developer Workflow

Developer Local Test QA Production

Least expensive but most executed

Most expensive but least executed

Development Environment


Least expensive but most executed

Most expensive but least executed

95-99% of developer time is spent in

the development environment

Developer environment must be ...

Quick Production-like Repeatable

Production-like

Each tier runs same OS & software versions

Tiers are isolated from each other as in production

Tiers are isolated from the developers device

[A client] never realized they needed to run gem install bundle on a tier until we put the two services on different servers.

Production-like

Our URL: http://xxxx.xxxxxx.xxx is not loading in IE and Safari but is in

Chrome/Firefox. This is only in AWS QA and not in AWS DEV or PROD. We are

wondering if there is something in the infra that is causing this behavior.

-Thx

http://xxxx.xxxxxx.xxx/

Quick

Waiting Frustration Distraction

“When I fire a local QA run in a mature project, it takes 8 mins. I check cricinfo.com and facebook while it runs”

Developer at a Fortune 100 company

Repeatable

Here is a Bash script to run on

your device

Follow these docs and ask away if any questions

Here is a Vagrant file

Install software PHP and MySQL on your device

Prone to human errors Prone to 3rd party changes

Repeatable

Here is a Bash script to run on

your device

Follow these docs and ask away if any questions

Here is a Vagrant file

Install software PHP and MySQL on your device

Prone to human errors Prone to 3rd party changes

[A client] supported MySQL Percona v5.53x. Vendors updated the version to 5.54 and broke our software.

Our Solution


One Docker container for each application tier

Run a VM with folder sync and port forwarding

Installs Docker Downloads containers (if not available)

Used to bootstrap Docker containers in dev and AWS instances in prod

AppWeb

DB

LogStashMem

$

HTTP

SSH

Virtualbox VM

App Code and Logs

Laptop

devenv.sh for interacting with

the setup

Docker Containers

Redis

Result: We no longer hear ...

“works on my machine but not on yours”

“I have been helping the [new hire]

setup her environment all day”

“IT guys must do something differently in staging”

“I can’t try it because IT has not

updated the dev environment”

Who sees the benefit?

Backend Developers

● Homogenous environment● Quick feedback● Agile: break it fearlessly

Front-end HTML Devs

● Quick visual feedback with populated data

● Avoid installations

Ops

Only one standardized dev environment to support

Management

● Less conflict● Faster time to market● Happier engineers● Easier hiring

Why we used Docker?

Minimal Performance

Overhead

Fast Boot and Shutdown

Smaller files sizes and ability to diff

containers

Container will run in the

Cloud

Variation: Today


Variation


Use Docker alone with boot2docker and Dockerfiles

Requires docker in entire workflow including prod

Docker will release version 1.0 soon

These Scripts Make a Working System

bootstrap_container

bootstrap_containers

build_base_image

build_from_scratch

clean_for_docker

copy_etc_hosts

create_etc_hosts

create_etc_hosts_for

deploy_app

help

plumb_container

plumb_containers

prep_vm

provision_container

push_etc_hosts

restart_container

restart_unicorn

retry_thrice

rm_containers

run_container

run_container_raw

run_containers

setup_ssh_keys

stop_container

stop_containers

A Full Production Database on a Laptop?

Multi-GB Production database won’t run

on a laptop

Strategically sampled data set

The Solution

Rule: Development uses fresh production data

SQS Runs In Cloud … Not A Laptop

Production uses Amazon SQS

Rule #2: No AWS Dependencies in Dev

Modified app to use RabbitMQ

Rule #1: Environment is Identical in Dev to Prod

The Solution

Service Addressing In Dev

Production uses static IP addresses

dev uses /etc/hosts,

prod uses Route53

Rule: Service addressing transparent between Dev and

Prod for developers

The Solution

Automation Benefits

Innovation: Easier to Innovate when cost of being wrong is low

Continuity: Organizational ownership of operations knowledge - very valuable for business continuity.

Faster Iteration: High fidelity and confidence in Dev and QA means means faster iteration of product

Talent: Willingness to experiment with advanced technology like docker attracts talented contributors to our team

Application: e-invoicing, ERP, Dynamic Discounting, AP automation, Workflow, supplier self service, Treasury Strategy, vendor portal,

supply chain finance, e-invoicing network, and Oracle based solutions

Case Study 3: QA using Docker

Problems

QA jobs take long

Tests need to run in parallel, but conflict with each other

Run different instances of test in different Docker container

Setup

Jenkins runs QA jobs

Jenkins slaves are Docker hosts

Dockerfile is used to create a generic image

Process

Jenkins checks out latest code from Git

Jenkins starts a new container on one of

the slaves

A container is built using Dockerfile

Jenkins workspace is attached to

container as volume

Code is compiled and tests are run

Test results are put in the Jenkins

workspace

Jenkins copies results back as

Artifacts

Application: 3D Printing

Case Study 4: Efficient and Easy Code Deployment using Docker

Problems

Deploy code conveniently to production from laptop

Setup

Use Dockerhub as code delivery mechanism

A staging environment, and a Production environment in AWS

Process

Build containers in Staging, test, and

push to DockerHub

Push triggers a pull of containers on instances

Containers are killed and new containers are

started

Other Docker Projects Flux7 has impacted

Djed Studios

Trucking Office

Docker Austin Meetup is participating in the Global Docker

Hackday

Thank You!

Twitter: @Flux7Labs

Blog: blog.flux7.com

https://twitter.com/Flux7Labs

http://blog.flux7.com/

BACKUP

Demo

How to start the environment?

How to stop the environment?

How to edit code and refresh in the

browser?

Docker on EB

10 Step Deployment Process

Elastic BeanStalk

AWS Elastic BeanStalk

✓ PaaS

✓ Application deployment and scaling

Supports: PHP, Node.js, Ruby, Python, Java, .NET, and now Docker!

EB Working in 3 steps

Choose Software stack

✓ Provisioning env✓ Deploying Code ✓ Load Balancing✓ Autoscaling ✓ Health Monitoring

URL to access the application

Pre-Docker

Limited stack options

Forced to choose predefined versions

Limited AMI customization

Post-Docker

Docker provides flexibility

Ability to choose any stack (Go, C++???)

3 Methods

Create a Dockerfile and

upload it to EB.Create a Dockerrun.aws.

json file to deploy an

existing Docker image.

Create a zip file with

Dockerfile, Dockerrun.

aws.json, and any

application file, then

upload to EB.

1 3 2

no ADD commands in Dockerfile AND first exposed PORT is what I need AND base image is public

How to Choose your method?

Image is in a repo and no changes are needed, use Dockerrun.aws.json

Zip Dockerfile, Dockerrun.aws.json, and other files and upload

Use dockerfileUse Dockerrun.aws.json

Default

Deploy a pre-baked Container to AWS EB

Step 1: Create S3 bucket

Step 2: Commit Docker image

Create and commit a Docker image with all required packages

Step 3: Create Docker Auth Config File

Login to Docker repo

For each login, a .dockerconfg file will be created

Upload this file to S3

EB gains access to Docker repo through S3

Step 4: Create Dockerrun.aws.json

Configuration file to deploy a container in EB: {"AWSEBDockerrunVersion" : "1", "Authentication": { "Bucket": "flux7.com", "Key": "dockercfg"}, "Ports": [ { "ContainerPort": "80" }]}

Step 5: Create Dockerfile

Sample Dockerfile

# VERSION 0.0.1

FROM flux7/wp-site # This is the location of our docker container.

RUN apt-get install supervisor

RUN mkdir -p /var/log/supervisor

ADD supervisord.conf /etc/supervisor/conf.d/supervisord.conf

EXPOSE 80

CMD supervisord -c /etc/supervisor/conf.d/supervisord.conf

Step 7: Create IAM role

Create IAM Role

Set user access policy with S3 bucket access

Step 8: Compress the files into a ZIP

Note: Do not copy into directory and ZIP

Step 9: Create EB environment

Create a new application

Set appropriate configuration details and environment tags

Choose the appropriate VPC and VPC security group

Review configuration and launch the environment

Step 10: Update Docker Containers

To deploy new code to EB

Create a new Docker image

Push to Docker repo