flocker documentation - read the docs · 2019-04-02 · chapter 1 introduction 1.1motivation for...

Flocker DocumentationRelease 0.1.0

ClusterHQ

August 12, 2014

Contents

1 Introduction 31.1 Motivation for Building Flocker . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.2 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.3 Initial Implementation Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.4 User Experience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2 Getting Started 72.1 Installing Flocker . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.2 Tutorial: Deploying and Migrating MongoDB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92.3 Example: Running PostgreSQL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

3 Advanced Documentation 213.1 What’s New in v0.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213.2 Using Flocker . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213.3 Configuring Flocker . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223.4 Volume Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233.5 Setting up External Routing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253.6 Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

4 Getting Involved 274.1 Contributing to Flocker . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274.2 Infrastructure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

5 Areas of Potential Future Development 355.1 Flocker Volume Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

6 Authors 39

i

Flocker Documentation, Release 0.1.0

Flocker is a data volume manager and multi-host Docker cluster management tool. With it you can control your datausing the same tools you use for your stateless applications. This means that you can run your databases, queues andkey-value stores in Docker and move them around as easily as the rest of your app.

With Flocker’s command line tools and a simple configuration language, you can deploy your Docker-based applica-tions onto one or more hosts. Once deployed, your applications will have access to the volumes you have configuredfor them. Those volumes will follow your containers when you use Flocker to move them between different hosts inyour Flocker cluster.

Contents:

Contents 1


2 Contents

CHAPTER 1

Introduction

1.1 Motivation for Building Flocker

Flocker lets you move your Docker containers and their data together between hosts. This means that you can runyour databases, queues and key-value stores in Docker and move them around as easily as the rest of your app. Evenstateless apps depend on many stateful services and currently running these services in Docker containers in productionis nearly impossible. Flocker aims to solve this problem by providing an orchestration framework that allows you toport both your stateful and stateless containers between environments.

Docker allows for multiple isolated, reproducible application environments on a single node: “containers”. Applica-tion state can be stored on a local disk in “volumes” attached to containers. And containers can talk to each other andthe external world via specified ports.

But what happens if you have more than one node?

• Where do containers run?

• How do you talk to the container you care about?

• How do containers across multiple nodes talk to each other?

• How does application state work if you move containers around?

The diagram below provides a high level representation of how Flocker addresses these questions.

3


1.2 Architecture

Below is a high-level overview of Flocker’s architecture. For more information, you can follow along with a tutorialthat walks you through deploying and migrating MongoDB or read more in our advanced documentation.

1.2.1 Flocker - Orchestration

• Flocker can run multiple containers on multiple nodes.

• Flocker offers a configuration language to specify what to run and where to run it.

1.2.2 Flocker - Routing

• Container configuration includes externally visible TCP port numbers.

• Connect to any node on a Flocker cluster and traffic is routed to the node hosting the appropriate container(based on port).

• Your external domain (www.example.com) configured to point at all nodes in the Flocker cluster(192.0.2.0, 192.0.2.1)

1.2.3 Flocker - Application State

• Flocker manages ZFS filesystems as Docker volumes. It attaches them to your containers.

• Flocker provides tools for copying those volumes between nodes.

• If an application container is moved from one node to another, Flocker automatically moves the volume with it.

1.2.4 Application Configuration

• Application configuration describes what you want to run in a container.

– it identifies a Docker image

– an optional volume mountpoint

– externally “routed” ports

• This configuration is expected to be shared between development, staging, production, etc environments.

• Flocker 0.1 does not support automatic re-deployment of application configuration changes.

1.2.5 Deployment Configuration

• Deployment configuration describes how you want your containers deployed.

– which nodes run which containers.

• This configuration can vary between development, staging, production, etc environments.

– Developer might want to deploy all of the containers on their laptop.

– Production might put database on one node, web server on another node, etc.

• Reacting to changes to this configuration is the primary focus of Flocker 0.1.

4 Chapter 1. Introduction


1.3 Initial Implementation Strategy

• This is the 0.1 approach.

• Future approaches will be very different; feedback is welcome.

• All functionality is provided as short-lived, manually invoked processes.

• flocker-deploy connects to each node over SSH and runs flocker-reportstate to gather the clusterstate.

• flocker-deploy then connects to each node over SSH and runs flocker-changestate to make thenecessary deployment changes.

• Nodes might connect to each other over SSH to copy volume data to the necessary place.

1.3.1 flocker-changestate

• This is installed on nodes participating in the Flocker cluster.

• Accepts the desired global configuration and current global state.

• Also looks at local state - running containers, configured network proxies, etc.

• Makes changes to local state so that it complies with the desired global configuration.

– Start or stop containers.

– Push volume data to other nodes.

– Add or remove routing configuration.

1.3.2 Managing Containers

• Geard is used to start, stop, and enumerate containers.

• Geard works by creating systemd units.

• Systemd units are a good way to provide admin tools for:

– Logging and state inspection.

– Starting/stopping (including at boot).

– Inter-unit dependency management.

1.3.3 Managing Volumes

• Volumes are ZFS filesystems.

• Volumes are attached to a Docker “data” container.

• Geard automatically associates the “data” container’s volumes with the actual container.

– Association is done based on container names by Geard.

• Data model * Volumes are owned by a specific node.

– Node A can push a copy to node B but node A still owns the volume. Node B may not modify its copy.

– Volumes can be “handed off” to another node, i.e. ownership is changed. Node A can hand off the volumeto node B. Then node B is now the owner and can modify the volume and node A no longer can.

1.3. Initial Implementation Strategy 5

https://github.com/openshift/geard


• Volumes are pushed and handed off so as to follow the containers they are associated with.

– This happens automatically when flocker-deploy runs with a new deployment configuration.

1.3.4 Managing Routes

• Containers claim TCP port numbers with the application configuration that defines them.

• Connections to that TCP port on the node that is running the container are proxied (NAT’d) into the containerfor whatever software is listening for them there.

• Connections to that TCP port on any other node in the Flocker cluster are proxied (NAT’d) to the node that isrunning the container.

• Proxying is done using iptables.

1.4 User Experience

• Flocker provides a command-line interface for manually deploying or re-deploying containers across nodes.

• The tool operates on two distinct pieces of configuration:

– Application

– Deployment

• Your sysadmin runs a command like flocker-deploy deployment-config.ymlapplication-config.yml on their laptop.

6 Chapter 1. Introduction

CHAPTER 2

Getting Started

2.1 Installing Flocker

As a user of Flocker you will need to install the flocker-cli package which provides command line tools tocontrol the cluster. This should be installed on a machine with SSH credentials to control the cluster nodes (e.g., ifyou use our Vagrant setup then the machine which is running Vagrant).

There is also a flocker-node package which is installed on each node in the cluster. It contains theflocker-changestate, flocker-reportstate, and flocker-volume utilities. These utilities arecalled by flocker-deploy (via SSH) to install and migrate Docker containers and their data volumes.

Note: For now, the flocker-node package is pre-installed by the Vagrant configuration in the tutorial. In the nextrelease it will be distributed as a standalone package which you will be able to install on an existing Fedora 20 server.

Note: If you’re interested in developing Flocker (as opposed to simply using it) see Contributing to Flocker.

2.1.1 Installing flocker-cli

Linux

Before you install flocker-cli you will need a compiler, Python 2.7 and the virtualenv Python utility in-stalled. On Fedora 20 you can do so by running:

alice@mercury:~$ sudo yum install @buildsys-build python python-devel python-virtualenv

On Ubuntu or Debian you can run:

alice@mercury:~$ sudo apt-get install gcc python2.7 python-virtualenv python2.7-dev

Then run the following script to do the actual install:

linux-install.sh

#!/bin/sh

# Create a virtualenv, an isolated Python environment, in a new directory called# "flocker-tutorial":virtualenv --python=/usr/bin/python2.7 flocker-tutorial

# Upgrade the pip Python package manager to its latest version inside the

7


# virtualenv. Some older versions of pip have issues installing Python wheel# packages.flocker-tutorial/bin/pip install --upgrade pip

# Install flocker-cli and dependencies inside the virtualenv:echo "Installing Flocker and dependencies, this may take a few minutes with no output to the terminal..."flocker-tutorial/bin/pip install --quiet https://storage.googleapis.com/archive.clusterhq.com/downloads/flocker/Flocker-0.1.0-py2-none-any.whlecho "Done!"

Save the script to a file and then run it:

alice@mercury:~$ sh linux-install.sh...alice@mercury:~$

The flocker-deploy command line program will now be available in flocker-tutorial/bin/:

alice@mercury:~$ cd flocker-tutorialalice@mercury:~/flocker-tutorial$ bin/flocker-deploy --version0.1.0alice@mercury:~/flocker-tutorial$

If you want to omit the prefix path you can e.g. add the appropriate directory to your $PATH. You’ll need to do thisevery time you start a new shell.

alice@mercury:~/flocker-tutorial$ export PATH="${PATH:+${PATH}:}${PWD}/bin"alice@mercury:~/flocker-tutorial$ flocker-deploy --version0.1.0alice@mercury:~/flocker-tutorial$

OS X

To install flocker-cli on OS X you can install virtualenv and then run the flocker-cli install script:

Installing virtualenv

Install the Homebrew package manager.

Make sure Homebrew has no issues:

alice@mercury:~/flocker-tutorial$ brew doctor...alice@mercury:~/flocker-tutorial$

Fix anything which brew doctor recommends that you fix by following the instructions it outputs.

Install Python, pip and virtualenv:

alice@mercury:~/flocker-tutorial$ brew updatealice@mercury:~/flocker-tutorial$ brew install python...alice@mercury:~/flocker-tutorial$ pip install virtualenv...alice@mercury:~/flocker-tutorial$

8 Chapter 2. Getting Started

http://brew.sh


Running the Install Script

osx-install.sh

#!/bin/sh

# Create a virtualenv, an isolated Python environment, in a new directory called# "flocker-tutorial":virtualenv --python=/usr/local/bin/python2.7 flocker-tutorial

# Upgrade the pip Python package manager to its latest version inside the# virtualenv. Some older versions of pip have issues installing Python wheel# packages.flocker-tutorial/bin/pip install --upgrade pip

# Install flocker-cli and dependencies inside the virtualenv:flocker-tutorial/bin/pip install https://storage.googleapis.com/archive.clusterhq.com/downloads/flocker/Flocker-0.1.0-py2-none-any.whl

Save the script to a file and then run it:

alice@mercury:~$ sh osx-install.sh...alice@mercury:~$

The flocker-deploy command line program will now be available in flocker-tutorial/bin/:

alice@mercury:~$ cd flocker-tutorialalice@mercury:~/flocker-tutorial$ bin/flocker-deploy --version0.1.0alice@mercury:~/flocker-tutorial$

If you want to omit the prefix path you can e.g. add the appropriate directory to your $PATH. You’ll need to do thisevery time you start a new shell.

alice@mercury:~/flocker-tutorial$ export PATH="${PATH:+${PATH}:}${PWD}/bin"alice@mercury:~/flocker-tutorial$ flocker-deploy --version0.1.0alice@mercury:~/flocker-tutorial$

2.2 Tutorial: Deploying and Migrating MongoDB

The goal of this tutorial is to teach you to use Flocker’s container, network, and volume orchestration functionality.By the time you reach the end of the tutorial you will know how to use Flocker to create an application. You will alsoknow how to expose that application to the network and how to move it from one host to another. Finally you willknow how to configure a persistent data volume for that application.

This tutorial is based around the setup of a MongoDB service. Flocker is a generic container manager. MongoDB isused only as an example here. Any application you can deploy into Docker you can manage with Flocker.

If you have any feedback or problems, you can Talk to Us.

2.2.1 Before You Begin

Requirements

To replicate the steps demonstrated in this tutorial, you will need:

2.2. Tutorial: Deploying and Migrating MongoDB 9


• Linux, FreeBSD, or OS X

• Vagrant (1.6.2 or newer)

• VirtualBox

• At least 10GB disk space available for the two virtual machines

• The OpenSSH client (the ssh, ssh-agent, and ssh-add command-line programs)

• bash

• The mongo MongoDB interactive shell (see below for installation instructions)

You will also need flocker-cli installed (providing the flocker-deploy command). See Installing flocker-cli.

Setup

Installing MongoDB

The MongoDB client can be installed through the various package managers for Linux, FreeBSD and OS X. If you donot already have the client on your machine, you can install it by running the appropriate command for your system.

Ubuntualice@mercury:~$ sudo apt-get install mongodb-clients...alice@mercury:~$

Red Hat / Fedoraalice@mercury:~$ sudo yum install mongodb...alice@mercury:~$

OS X Install Homebrew

alice@mercury:~$ brew update...alice@mercury:~$ brew install mongodb...alice@mercury:~$

Other Systems See the official MongoDB installation guide for your system.

Creating Vagrant VMs Needed for Flocker

Before you can deploy anything with Flocker you’ll need a node onto which to deploy it. To make this easier, thistutorial uses Vagrant to create two VirtualBox VMs.

These VMs serve as hosts on which Flocker can run Docker. Flocker does not require Vagrant or VirtualBox. You canrun it on other virtualization technology (e.g., VMware), on clouds (e.g., EC2), or directly on physical hardware.

For your convenience, this tutorial includes Vagrantfile which will boot the necessary VMs. Flocker and itsdependencies will be installed on these VMs the first time you start them. One important thing to note is that theseVMs are statically assigned the IPs 172.16.255.250 (node1) and 172.16.255.251 (node2). These two IP


https://docs.vagrantup.com/

https://www.virtualbox.org/

http://brew.sh/

http://docs.mongodb.org/manual/installation/

https://docs.vagrantup.com/


addresses will be used throughout the tutorial and configuration files. If these addresses conflict with your localnetwork configuration you can edit the Vagrantfile to use different values. Note that you will need to make thesame substitution in commands used throughout the tutorial.

Note: The two virtual machines are each assigned a 10GB virtual disk. The underlying disk files grow to about 5GB.So you will need at least 10GB of free disk space on your workstation.

First create a tutorial directory:

alice@mercury:~/$ mkdir flocker-tutorialalice@mercury:~/$ cd flocker-tutorialalice@mercury:~/flocker-tutorial$

Next download the Vagrant configuration file by right clicking on the link below. Save it in the flocker-tutorial directoryand preserve its filename.

• Vagrant configuration

alice@mercury:~/flocker-tutorial$ lsVagrantfilealice@mercury:~/flocker-tutorial$

Then use vagrant up to start and provision the VMs:

alice@mercury:~/flocker-tutorial$ vagrant upBringing machine ’node1’ up with ’virtualbox’ provider...==> node1: Importing base box ’clusterhq/flocker-dev’...... lots of output ...==> node2: ln -s ’/usr/lib/systemd/system/docker.service’ ’/etc/systemd/system/multi-user.target.wants/docker.service’==> node2: ln -s ’/usr/lib/systemd/system/geard.service’ ’/etc/systemd/system/multi-user.target.wants/geard.service’alice@mercury:~/flocker-tutorial$

This step may take several minutes or more as it downloads the Vagrant image, boots up two nodes and downloads theDocker image necessary to run the tutorial. Your network connectivity and CPU speed will affect how long this takes.Fortunately this extra work is only necessary the first time you bring up a node (until you destroy it).

After vagrant up completes you may want to verify that the two VMs are really running and accepting SSHconnections:

alice@mercury:~/flocker-tutorial$ vagrant statusCurrent machine states:

node1 running (virtualbox)node2 running (virtualbox)...alice@mercury:~/flocker-tutorial$ vagrant ssh -c hostname node1node1Connection to 127.0.0.1 closed.alice@mercury:~/flocker-tutorial$ vagrant ssh -c hostname node2node2Connection to 127.0.0.1 closed.alice@mercury:~/flocker-tutorial$

If all goes well, the next step is to configure your SSH agent. This will allow Flocker to authenticate itself to the VM.If you’re not sure whether you already have an SSH agent running, ssh-add can tell you. If you don’t, you’ll see anerror:

alice@mercury:~/flocker-tutorial$ ssh-addCould not open a connection to your authentication agent.alice@mercury:~/flocker-tutorial$



If you do, you’ll see no output:

alice@mercury:~/flocker-tutorial$ ssh-addalice@mercury:~/flocker-tutorial$

If you don’t have an SSH agent running, start one:

alice@mercury:~/flocker-tutorial$ eval $(ssh-agent)Agent pid 27233alice@mercury:~/flocker-tutorial$

Finally, add the Vagrant key to your agent:

alice@mercury:~/flocker-tutorial$ ssh-add ~/.vagrant.d/insecure_private_keyalice@mercury:~/flocker-tutorial$

You now have two VMs running and easy SSH access to them. This completes the Vagrant-related setup.

2.2.2 Moving Applications

Note: If you haven’t already, make sure to install the flocker-cli package before continuing with this tutorial.

Starting an Application

Let’s look at an extremely simple Flocker configuration for one node running a container containing a MongoDBserver.

minimal-application.yml

"version": 1"applications":

"mongodb-example":"image": "clusterhq/mongodb"

minimal-deployment.yml

"version": 1"nodes":

"172.16.255.250": ["mongodb-example"]"172.16.255.251": []

Notice that we mention the node that has no applications deployed on it to ensure that flocker-deploy knows thatit exists. If we hadn’t done that certain actions that might need to be taken on that node will not happen, e.g. stoppingcurrently running applications.

Next take a look at what containers Docker is running on the VM you just created. The node IPs are those which werespecified earlier in the Vagrantfile:

alice@mercury:~/flocker-tutorial$ ssh [email protected] docker psCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMESalice@mercury:~/flocker-tutorial$

From this you can see that there are no running containers. To fix this, use flocker-deploy with the simpleconfiguration files given above and then check again:



alice@mercury:~/flocker-tutorial$ flocker-deploy minimal-deployment.yml minimal-application.ymlalice@mercury:~/flocker-tutorial$ ssh [email protected] docker psCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES4d117c7e653e clusterhq/mongodb:latest mongod 2 seconds ago Up 1 seconds 27017/tcp, 28017/tcp mongodb-examplealice@mercury:~/flocker-tutorial$

flocker-deploy has made the necessary changes to make your node match the state described in the configurationfiles you supplied.

Moving an Application

Let’s see how flocker-deploy can move this application to a different VM. Recall that the Vagrant configurationsupplied in the setup portion of the tutorial started two VMs. Copy the deployment configuration file and edit it so thatit indicates the application should run on the second VM instead of the first. The only change necessary to indicatethis is to change the original IP address, 172.16.255.250, to the address of the other node, 172.16.255.251.The new file should be named minimal-deployment-moved.yml.

minimal-deployment-moved.yml


"172.16.255.250": []"172.16.255.251": ["mongodb-example"]

Note that nothing in the application configuration file needs to change. Moving the application only involves updatingthe deployment configuration.

Use flocker-deploy again to enact the change:

alice@mercury:~/flocker-tutorial$ flocker-deploy minimal-deployment-moved.yml minimal-application.ymlalice@mercury:~/flocker-tutorial$

docker ps shows that no containers are running on 172.16.255.250:

alice@mercury:~/flocker-tutorial$ ssh [email protected] docker psCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMESalice@mercury:~/flocker-tutorial$

and that MongoDB has been successfully moved to 172.16.255.251:

alice@mercury:~/flocker-tutorial$ ssh [email protected] docker psCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES4d117c7e653e clusterhq/mongodb:latest mongod 3 seconds ago Up 2 seconds 27017/tcp, 28017/tcp mongodb-examplealice@mercury:~/flocker-tutorial$

At this point you have successfully deployed a MongoDB server in a container on your VM. You’ve also seen howFlocker can move an existing container between hosts. There’s no way to interact with it apart from looking at thedocker ps output yet. In the next section of the tutorial you’ll see how to expose container services on the host’snetwork interface.

2.2.3 Exposing Ports

Each application running in a Docker container has its own isolated networking stack. To communicate with anapplication running inside the container we need to forward traffic from a network port in the node where the containeris located to the appropriate port within the container. Flocker takes this one step further: an application is reachableon all nodes in the cluster, no matter where it is currently located.



Let’s start a MongoDB container that exposes the database to the external world.

port-application.yml


"mongodb-port-example":"image": "clusterhq/mongodb""ports":- "internal": 27017

"external": 27017

port-deployment.yml


"172.16.255.250": ["mongodb-port-example"]"172.16.255.251": []

We will once again run these configuration files with flocker-deploy:

alice@mercury:~/flocker-tutorial$ flocker-deploy port-deployment.yml port-application.ymlalice@mercury:~/flocker-tutorial$ ssh [email protected] docker psCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES4d117c7e653e clusterhq/mongodb:latest mongod 2 seconds ago Up 1 seconds 27017/tcp, 28017/tcp mongodb-port-examplealice@mercury:~/flocker-tutorial$

This time we can communicate with the MongoDB application by connecting to the node where it is running. Usingthe mongo command line tool we will insert an item into a database and check that it can be found. You should try tofollow along and do these database inserts as well.

Note: To keep your download for the tutorial as speedy as possible, we’ve bundled the latest dev version of MongoDBin to a micro-sized Docker image. You should not use this image for production.

If you get a connection refused error try again after a few seconds; the application might take some time to fully startup.

alice@mercury:~/flocker-tutorial$ $ mongo 172.16.255.250MongoDB shell version: 2.4.9connecting to: 172.16.255.250/test> use example;switched to db example> db.records.insert({"flocker": "tested"})> db.records.find({}){ "_id" : ObjectId("53c958e8e571d2046d9b9df9"), "flocker" : "tested" }

We can also connect to the other node where it isn’t running and the traffic will get routed to the correct node:

alice@mercury:~/flocker-tutorial$ mongo 172.16.255.251MongoDB shell version: 2.4.9connecting to: 172.16.255.251/test> use example;switched to db example> db.records.find({}){ "_id" : ObjectId("53c958e8e571d2046d9b9df9"), "flocker" : "tested" }

Since the application is transparently accessible from both nodes you can configure a DNS record that points at bothIPs and access the application regardless of its location. See Setting up External Routing for more details.

At this point you have successfully deployed a MongoDB server and communicated with it. You’ve also seen how



external users don’t need to worry about applications’ location within the cluster. In the next section of the tutorialyou’ll learn how to ensure that the application’s data moves along with it, the final step to running stateful applicationson a cluster.

2.2.4 Data Volumes

The Problem

By default moving an application from one node to another does not move its data along with it. Before proceedinglet’s see in more detail what the problem is by continuing the Exposing Ports example.

Recall that we inserted some data into the database. Next we’ll use a new configuration file that moves the applicationto a different node.

port-deployment-moved.yml


"172.16.255.250": []"172.16.255.251": ["mongodb-port-example"]

alice@mercury:~/flocker-tutorial$ flocker-deploy port-deployment-moved.yml port-application.ymlalice@mercury:~/flocker-tutorial$ ssh [email protected] docker psCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES4d117c7e653e clusterhq/mongodb:latest mongod 2 seconds ago Up 1 seconds 27017/tcp, 28017/tcp mongodb-port-examplealice@mercury:~/flocker-tutorial$

If we query the database the records we’ve previously inserted have disappeared! The application has moved but thedata has been left behind.

alice@mercury:~/flocker-tutorial$ mongo 172.16.255.251MongoDB shell version: 2.4.9connecting to: 172.16.255.251/test> use example;switched to db example> db.records.find({})>

The Solution

Unlike many other Docker frameworks Flocker has a solution for this problem, a ZFS-based volume manager. Anapplication with a Flocker volume configured will move the data along with the application, transparently and with noadditional intervention on your part.

We’ll create a new configuration for the cluster, this time adding a volume to the MongoDB container.

volume-application.yml


"mongodb-volume-example":"image": "clusterhq/mongodb""ports":- "internal": 27017

"external": 27017"volume":

# The location within the container where the data volume will be



# mounted:"mountpoint": "/data/db"

volume-deployment.yml


"172.16.255.250": ["mongodb-volume-example"]"172.16.255.251": []

Then we’ll run these configuration files with flocker-deploy:

alice@mercury:~/flocker-tutorial$ flocker-deploy volume-deployment.yml volume-application.ymlalice@mercury:~/flocker-tutorial$ ssh [email protected] docker psCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES4d117c7e653e clusterhq/mongodb:latest mongod 2 seconds ago Up 1 seconds 27017/tcp, 28017/tcp mongodb-volume-examplealice@mercury:~/flocker-tutorial$

Once again we’ll insert some data into the database:

alice@mercury:~/flocker-tutorial$ $ mongo 172.16.255.250MongoDB shell version: 2.4.9connecting to: 172.16.255.250/test> use example;switched to db example> db.records.insert({"the data": "it moves"})> db.records.find({}){ "_id" : ObjectId("53d80b08a3ad4df94a2a72d6"), "the data" : "it moves" }

Next we’ll move the application to the other node.

volume-deployment-moved.yml


"172.16.255.250": []"172.16.255.251": ["mongodb-volume-example"]

alice@mercury:~/flocker-tutorial$ flocker-deploy volume-deployment-moved.yml volume-application.ymlalice@mercury:~/flocker-tutorial$ ssh [email protected] docker psCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES4d117c7e653e clusterhq/mongodb:latest mongod 2 seconds ago Up 1 seconds 27017/tcp, 28017/tcp mongodb-volume-examplealice@mercury:~/flocker-tutorial$

This time however the data has moved with the application:

alice@mercury:~/flocker-tutorial$ mongo 172.16.255.251MongoDB shell version: 2.4.9connecting to: 172.16.255.251/test> use example;switched to db example> db.records.find({}){ "_id" : ObjectId("53d80b08a3ad4df94a2a72d6"), "the data" : "it moves" }

At this point you have successfully deployed a MongoDB server and communicated with it. You’ve also seen howFlocker allows you to move an application’s data to different locations in a cluster as the application is moved. Younow know how to run stateful applications in a Docker cluster using Flocker.

The virtual machines you are running will be useful for testing Flocker and running other examples in the documenta-tion. If you would like to shut them down temporarily you can run vagrant halt in the tutorial directory. You can



then restart them by running vagrant up. If you would like to completely remove the virtual machines you can runvagrant destroy.

2.3 Example: Running PostgreSQL

Once you’ve successfully followed through Tutorial: Deploying and Migrating MongoDB this example will show youhow to deploy a PostgreSQL container with Flocker. We’ll insert some data, then use flocker-deploy to movethe PostgreSQL server container to another virtual machine. The data in the database will be moved along with theapplication.

The requirements are the same as the MongoDB tutorial, except that we will be using PostgreSQL instead of mongoto connect to the server.

2.3.1 Create the Virtual Machines

We’ll be using the same Vagrant configuration as the MongoDB tutorial. If you haven’t already started up the Vagrantvirtual machines follow the setup instructions.

2.3.2 Download the Docker Image

The official postgres Docker image is quite big, so you may wish to pre-fetch it to your nodes so you don’t have towait for downloads half-way through this example. (We use the -t option to ssh so that progress is displayed; if youomit it the pull will still work but you may not get any output for a long time.)

alice@mercury:~/flocker-postgres$ ssh -t [email protected] docker pull postgres...alice@mercury:~/flocker-postgres$ ssh -t [email protected] docker pull postgres...alice@mercury:~/flocker-postgres$

2.3.3 Launch PostgreSQL

Download and save the following configuration files to your flocker-postgres directory:

postgres-application.yml


"postgres-volume-example":"image": "postgres""ports":- "internal": 5432


# The location within the container where the data volume will be# mounted; see https://github.com/docker-library/postgres/blob/docker/Dockerfile.template"mountpoint": "/var/lib/postgresql/data"

postgres-deployment.yml

2.3. Example: Running PostgreSQL 17

https://www.postgresql.org/download/



"172.16.255.250": ["postgres-volume-example"]"172.16.255.251": []

As you can see, we will be pulling the postgres image and deploying to one of our virtual nodes. Runflocker-deploy to download the image and get the container running:

alice@mercury:~/flocker-postgres$ flocker-deploy postgres-deployment.yml postgres-application.ymlalice@mercury:~/flocker-postgres$

You can keep running ssh [email protected] docker ps until you see the container running:

alice@mercury:~/flocker-postgres$ ssh [email protected] docker psCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMESf6ee0fbd0446 postgres:latest /bin/sh -c /init 7 seconds ago Up 6 seconds 0.0.0.0:5432->5432/tcp postgres-volume-examplealice@mercury:~/flocker-postgres$

2.3.4 Connect to PostgreSQL

We can now use the psql client on our host machine to connect to the PostgreSQL server running inside the container.Connect using the client to the IP address of our virtual machine, using the port number we exposed in our applicationconfiguration.

alice@mercury:~/flocker-postgres$ psql postgres --host 172.16.255.250 --port 5432 --username postgrespsql (9.3.5)Type "help" for help.

postgres=#

2.3.5 Insert a Row in the Database

Now we have a connection, we’ll create a new database with the simplest possible structure; a single table with oneinteger column. We’ll then switch our connection to use the new database and insert a row. From the running psqlclient shell:

postgres=# \listList of databases

Name | Owner | Encoding | Collate | Ctype | Access privileges-----------+----------+-----------+---------+-------+-----------------------postgres | postgres | SQL_ASCII | C | C |template0 | postgres | SQL_ASCII | C | C | =c/postgres +

| | | | | postgres=CTc/postgrestemplate1 | postgres | SQL_ASCII | C | C | =c/postgres +

| | | | | postgres=CTc/postgres(3 rows)

postgres=# CREATE DATABASE flockertest;CREATE DATABASEpostgres=# \list

List of databasesName | Owner | Encoding | Collate | Ctype | Access privileges

-------------+----------+-----------+---------+-------+-----------------------flockertest | postgres | SQL_ASCII | C | C |postgres | postgres | SQL_ASCII | C | C |template0 | postgres | SQL_ASCII | C | C | =c/postgres +



| | | | | postgres=CTc/postgrestemplate1 | postgres | SQL_ASCII | C | C | =c/postgres +

| | | | | postgres=CTc/postgres(4 rows)

postgres=# \connect flockertest;psql (9.3.5)You are now connected to database "flockertest" as user "postgres".flockertest=# CREATE TABLE testtable (testcolumn int);CREATE TABLEflockertest=# INSERT INTO testtable (testcolumn) VALUES (3);INSERT 0 1flockertest=# SELECT * FROM testtable;testcolumn

------------3

(1 row)

flockertest=# \quit

2.3.6 Create a New Deployment Configuration and Move the Application

Download the new deployment configuration and save to your flocker-postgres directory.

postgres-deployment-moved.yml


"172.16.255.250": []"172.16.255.251": ["postgres-volume-example"]

Now run flocker-deploy on the new configuration:

alice@mercury:~/flocker-postgres$ flocker-deploy postgres-deployment-moved.yml postgres-application.ymlalice@mercury:~/flocker-postgres$

Now we’ll verify that our application has moved to the other VM:

alice@mercury:~/flocker-postgres$ ssh [email protected] docker psCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES51b5b09a46bb clusterhq/postgres:latest /bin/sh -c /init 7 seconds ago Up 6 seconds 0.0.0.0:5432->5432/tcp postgres-volume-examplealice@mercury:~/flocker-postgres$

And is no longer running on the original host:

alice@mercury:~/flocker-postgres$ ssh [email protected] docker psCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMESalice@mercury:~/flocker-postgres$

2.3.7 Verify Our Data Has Moved with the Application

Query the flockertest database for the data we previously inserted. You will find that Flocker has moved ourvolume with the container and our data has been preserved.

alice@mercury:~/flocker-postgres$ psql postgres --host 172.16.255.251 --port 5432 --username postgrespsql (9.3.5)

2.3. Example: Running PostgreSQL 19


Type "help" for help.

postgres=# \connect flockertest;psql (9.3.5)You are now connected to database "flockertest" as user "postgres".flockertest=# select * from testtable;testcolumn

------------3

(1 row)

This concludes our example for using Flocker with PostgreSQL. Now you’ve successfully followed through bothour tutorial and a further working example of what you can do with flocker, you may now wish to read through theAdvanced Documentation.


CHAPTER 3

Advanced Documentation

3.1 What’s New in v0.1

Everything is new since this is our first release.

3.1.1 Known Limitations

• This release is not ready for production and should not be used on publicly accessible servers or to store datayou care about. Backwards compatibility is not a goal yet.

• Changes to the application configuration file will often not be noticed by flocker-deploy, and there is noway to delete applications or volumes. Choose new names for your applications if you are making changes tothe application configuration.

You can learn more about where we might be going with future releases by:

• Stopping by the #clusterhq channel on irc.freenode.net.

• Visiting our GitHub repository at https://github.com/ClusterHQ/flocker.

• Reading Areas of Potential Future Development.

3.2 Using Flocker

Flocker manages which containers are running and on what hosts. It also manages network configuration for thesecontainers (between them and between containers and the world). And Flocker also creates and replicates volumes.All of this functionality is available via a simple invocation of the flocker-deploy program. This program isincluded in the flocker-cli package. If you haven’t installed that package yet, you may want to do so now.

3.2.1 Command Line Arguments

flocker-deploy takes just two arguments. The first of these is the path to a deployment configuration file. Thesecond is the path to an application configuration file.

$ flocker-deploy clusterhq_deployment.yml clusterhq_app.yml

The contents of these two configuration files determine what actions Flocker actually takes. The configuration filescompletely control this; there are no other command line arguments or options. See Configuring Flocker for detailsabout these two files.

21

https://github.com/ClusterHQ/flocker


You can run flocker-deploy anywhere you have it installed. The containers you are managing do not need to berunning on the same host as flocker-deploy.

3.2.2 Authentication

Setup

flocker-deploy lets you manage containers on one or more hosts. Before flocker-deploy can do this itneeds to be able to authenticate itself to these hosts. Flocker uses SSH to communicate with the hosts you specifyin the deployment configuration file. It requires that you configure SSH access to the root user in advance. Therecommended configuration is to generate an SSH key (if you don’t already have one):

$ ssh-keygen

Then add it to your SSH key agent:

$ ssh-add <path to key file>

Finally add it to the authorized_keys file of each host you want to manage:

$ ssh-copy-id -i <path to key file> root@<hostname>

This will allow flocker-deploy to connect to these hosts (as long as the key is still available in your key agent).

If you have a different preferred SSH authentication configuration which allows non-interactive SSH authenticationyou may use this instead.

Other Keys

flocker-deploy will generate an additional SSH key. This key is deployed to each host you manage with Flockerand allows the hosts to authenticate to each other.

3.3 Configuring Flocker

Flocker operates on two configuration files: application and deployment. Together these configurations define a de-ployment.

The configuration is represented using yaml syntax.

3.3.1 Application Configuration

The application configuration consists of a version and short, human-meaningful application names and the parametersnecessary to run those applications.

The required parameters are version and applications. For now the version must be 1.

The parameters required to define an application are:

• image

This is the name of the Docker image which will be used to start the container which will run the application.Optionally, this may include a tag using the <name>:<tag> syntax.

For example, an application which is meant to use version 1.0 of ClusterHQ’s flocker-dev Docker image isconfigured like this:

22 Chapter 3. Advanced Documentation

https://en.wikipedia.org/wiki/Ssh-keygen

https://en.wikipedia.org/wiki/Ssh-agent


"image": "clusterhq/flocker-dev:v1.0"

The following parameters are optional when defining an application:

• ports

This is an optional list of port mappings to expose to the outside world. Connections to the external port on thehost machine are forwarded to the internal port in the container.

"ports":- "internal": 80"external": 8080

• volume

This specifies that the application container requires a volume. It also allows you to specify where in thecontainer the volume will be mounted via the mountpoint key. The value for this key must be a string givingan absolute path.

Here’s an example of a simple but complete configuration defining one application:


"site-clusterhq.com":"image": "clusterhq/clusterhq-website""ports":- "internal": 80


"mountpoint": "/var/mysql/data"

3.3.2 Deployment Configuration

The deployment configuration specifies which applications are run on what nodes. It consists of a version and amapping from node names to application names.

The required parameters are version and applications. For now the version must be 1.

Here’s an example of a simple but complete configuration defining a deployment of one application on one host:


"node017.example.com":"site-clusterhq.com"

3.4 Volume Manager

Flocker comes with a volume manager, a tool to manage volumes that can be attached to Docker containers. Ofparticular note is the ability to push volumes to different machines.

3.4.1 Configuration

Each host in a Flocker cluster has a universally unique identifier (UUID) for its volume manager. By default the UUIDis stored in /etc/flocker/volume.json.

The volume manager stores volumes inside a ZFS pool called flocker.

3.4. Volume Manager 23


3.4.2 Volume Ownership

Each volume is owned by a specific volume manager and only that volume manager can write to it. To begin witha volume is owned by the volume manager that created it. A volume manager can push volumes it owns to anothermachine, copying the volume’s data to a remote volume manager. The copied volume on that remote volume managerwill continue to be owned by the local volume manager, and therefore the remote volume manager will not be able towrite to it.

A volume manager can also handoff a volume to a remote volume manager, i.e. transfer ownership. The remotevolume manager becomes the owner of the volume and subsequently it is able to write to the volume. The volumemanager that did the handoff ceases to own the volume and subsequently is not allowed to write to the volume.

3.4.3 Implementation Details

Each volume is a ZFS dataset. Volumes are created with three parameters:

• The UUID of the volume manager that owns the volume. The creating volume manager’s UUID (see above) isused to supply a value for this parameter.

• The logical name; this must be the same as the name of the container it will be mounted in. For example, for acontainer named "myapp-mongodb" a volume called "myapp-mongodb" will be created.

• A mount path, indicating where within a container the volume will be mounted. For example, for a Mongoserver this would be "/var/lib/mongodb" since that is where Mongo stores its data.

The ZFS dataset name is a combination of the UUID and the logical name, e.g. 1234.myapp-mongodb.

Docker Integration

Volumes are exposed to Docker by creating a container with a "-data" suffix that mounts the volume inthe appropriate location. For example, if you create a volume called "myapp-mongodb" with mountpoint"/var/lib/mongodb" then a container called "myapp-mongodb-data" will be created that has the volumemounted at that path.

You can then use this volume manually using --volumes-from:

$ docker run --volumes-from myapp-mongodb-data --name myapp-mongodb openshift/centos-mongodb

The myapp-mongodb container will now have a volume mounted at /var/lib/mongodb pointing at the ZFSdataset managed by Flocker.

Even easier, geard and therefore the Flocker orchestration system will automatically mount volumes from"myapp-mongodb-data" if you create a unit called "myapp-mongodb".

Push and Handoff

Push and handoffs are currently done over SSH between nodes, with ad-hoc calls to the flocker-volumecommand-line tool. In future releases this will be switched to a real protocol and later on to communication betweenlong-running daemons rather than short-lived scripts. (See #154.)

When a volume is pushed a zfs send is used to serialize its data for transmission to the remote machine, whichdoes a zfs receive to decode the data and create or update the corresponding ZFS dataset.

Handoff involves renaming the ZFS dataset to change the owner UUID encoded in the dataset name. For example,imagine two volume managers with UUIDs 1234 and 5678 and a dataset called mydata.


https://github.com/ClusterHQ/flocker/issues/154


Action Volume Manager 1234 Volume Manager 5678

1. Create mydata on 12341234.mydata (owner)

2. Push mydata to 56781234.mydata (owner) 1234.mydata

3. Handoff mydata to 56785678.mydata 5678.mydata (owner)

3.5 Setting up External Routing

Flocker allows you to expose public ports on your applications. For example, you can export port 8443 on an HTTPSserver running inside a container as an externally visible port 443 on the host machine. Because Flocker runs on acluster of nodes your web application might run on different nodes at different times. You could update the DNSrecord every time a container moves. However, updating DNS records can take anywhere from a minute to a fewhours to take effect for all clients so this will impact your application’s availability. This is where Flocker’s routingfunctionality comes in handy.

When an external route is configured (e.g. on port 443) Flocker routes that port on all nodes to the node where yourapplication is running. You can therefore move a node and then change your DNS configuration appropriately withoutincurring any downtime.

3.5.1 No-Change DNS Configuration

What’s more it is also possible to configure your DNS records in such a way that no DNS changes are necessary whenapplications move to different nodes. Specifically, the DNS record for your application should be configured to pointat all IPs in the cluster.

For example, consider the following setup:

www.example.com has a DNS record pointing at two different nodes’ IP. Every time you connect towww.example.com your browser will choose one of the two IPs at random.

• If you connect to port 80 on the node2 — which is not hosting the container — the traffic will be routed on tonode1.

• If you connect to port 80 on node1 you will reach the web server that is listening on port 8080 within acontainer.

Note that if nodes are in different data centers and you pay for bandwidth this configuration will require you to pay forforwarded traffic between nodes.

3.6 Debugging

3.6.1 Logging

The Flocker processes running on the nodes will write their logs to /var/log/flocker/. The log files are named<processname>-<pid>.log, e.g. flocker-volume-1234.log.

3.5. Setting up External Routing 25


Logs from the Docker containers are written to systemd’s journal with a unit name constructed with a ctr- prefix. Forexample if you’ve started an application called mymongodb you can view its logs by running the following commandon the node where the application was started:

$ journalctl -u ctr-mymongodb


http://www.freedesktop.org/software/systemd/man/journalctl.html

CHAPTER 4

Getting Involved

4.1 Contributing to Flocker

4.1.1 Introduction

ClusterHQ develops software using a variation of the Ultimate Quality Development System.

• Each unit of work is defined in an issue in the issue tracker and developed on a branch.

• Code is written using test-driven development.

• The issue is closed by merging the branch (via a GitHub pull request).

• Before a branch is merged it must pass code review.

• The code reviewer ensures that the pull request:

– Follows the coding standard (Python’s PEP 8).

– Includes appropriate documentation.

– Has full test coverage (unit tests and functional tests).

– The tests pass in the continuous integration system (Buildbot).

– Resolves the issue.

• The code reviewer can approve the pull request for merging as is, with some changes, or request changes and anadditional review.

4.1.2 Talk to Us

Have questions or need help? Besides filing a GitHub issue with feature requests or bug reports you can also join uson the #clusterhq channel on the irc.freenode.net IRC network or on the flocker-users Google Group.

4.1.3 Development Environment

• To run the complete test suite you will need ZFS, geard and docker installed. geard requires an operatingsystem with systemd. The recommended way to get an environment with these installed is to use the includedVagrantfilewhich will create a pre-configured Fedora 20 virtual machine. Vagrant 1.6.2 or later is required.Once you have Vagrant installed (see the Vagrant documentation) you can run the following to get going:

27

https://twistedmatrix.com/trac/wiki/UltimateQualityDevelopmentSystem

http://legacy.python.org/dev/peps/pep-0008/

http://build.clusterhq.com/

https://github.com/ClusterHQ/flocker/issues

https://groups.google.com/forum/?hl=en#!forum/flocker-users

http://zfsonlinux.org

https://openshift.github.io/geard/

https://www.docker.com/

http://docs.vagrantup.com/


$ vagrant up$ vagrant ssh

• You will need Python 2.7 and a recent version of PyPy installed on your development machine.

• If you don’t already have tox on your development machine, you can install it and other development depen-dencies (ideally in a virtualenv) by doing:

$ python setup.py install .[doc,dev]

4.1.4 Running Tests

You can run all unit tests by doing:

$ tox

Functional tests require ZFS, geard and docker to be installed and in the case of the latter two running as well. Inaddition, tox needs to be run as root:

$ sudo tox

Since these tests involve global state on your machine (filesystems, iptables, docker containers, etc.) we recommendrunning them in the development Vagrant image.

4.1.5 Documentation

Documentation is generated using Sphinx and stored in the docs/ directory. You can build it individually by running:

$ tox -e sphinx

You can view the result by opening docs/_build/html/index.html in your browser.

4.1.6 Requirements for Contributions

1. All code must have unit test coverage and to the extent possible functional test coverage.

Use the coverage.py tool with the --branch option to generate line and branch coverage reports. This reportcan tell you if you missed anything. It does not necessarily catch everything though. Treat it as a helper but notthe definitive indicator of success. You can also see coverage output in the Buildbot details link of your pullrequest. Practice test-driven development to ensure all code has test coverage.

2. All code must have documentation.

Modules, functions, classes, and methods must be documented (even if they are private). Function parametersand object attributes must be documented (even if they are private).

3. All user-facing tools must have documentation.

Document tool usage as part of big-picture documentation. Identify useful goals the user may want to accom-plish and document tools within the context of accomplishing those goals.

4. Add your name (in alphabetical order) to the AUTHORS.rst file.

28 Chapter 4. Getting Involved

http://sphinx-doc.org/


4.1.7 Project Development Process

The core development team uses GitHub issues to track planned work. Issues are organized by release milestones, andthen by subcategories:

Backlog Issues we don’t expect to do in the release. These issues don’t have any particular category label. All issuesstart in the backlog when they are filed. The requirements for an issue must be completely specified before itcan move out of the backlog.

Design Issues that we expect to work on soon. This is indicated by a design label. A general plan for accomplishingthe requirements must be specified on the issue before it can move to the Ready state. The issue is assigned tothe developer working on the plan. When there is a proposed plan the review label is added to the issue (sothat it has both design and review).

Ready Issues that are ready to be worked on. This is indicated by a ready label. Issues can only be Ready after theyhave been in Design so they include an implementation plan. When someone starts work on an issue it is movedto the In Progress category (the ready keyword is removed and the in progress label is added).

In Progress Such issues are assigned to the developer who is currently working on them. This is indicated by an inprogress label. When the code is ready for review a new pull request is opened. The pull request is added tothe Review category.

Ready for Review An issue or pull request that includes work that is ready to be reviewed. This is indicated by areview label. Issues can either be in design review (design and review) or final review (just review). Areviewer can move a design review issue to Ready (to indicate the design is acceptable) or back to Design (toindicate it needs more work). A reviewer can move a final review issue to Approved (to indicate the work isacceptable) or back to In Progress (to indicate more work is needed).

Passed Review A pull request that has some minor problems that need addressing, and can be merged once those aredealt with and all tests pass. This is indicated by an accepted label.

Done Closed issues and pull requests.

Blocked Issues that can’t be worked on because they are waiting on some other work to be completed. This isindicated by a blocked label.

You can see the current status of all issues and pull requests by visiting https://waffle.io/clusterhq/flocker. In generalissues will move from Backlog to Design to Ready to In Progress. An in-progress issue will have a branch with theissue number in its name. When the branch is ready for review a pull request will be created in the Review category.When the branch is merged the corresponding pull requests and issues will be closed.

Steps to Contribute Code

GitHub collaborators can participate in the development workflow by changing the labels on an issue. GitHub lets non-collaborators create new issues and pull requests but it does not let them change labels. If you are not a collaboratoryou may seek out assistances from a collaborator to set issue labels to reflect the issue’s stage.

1. Pick the next issue in the Ready category. Drag it to the In Progress column in Waffle (or change the label fromready to in progress in GitHub).

2. Create a branch from master with a name including a few descriptive words and ending with the issue number,e.g. add-thingie-123.

3. Resolve the issue by making changes in the branch.

4. Submit the issue/branch for review. Create a pull request on GitHub for the branch. The pull request shouldinclude a Fixes #123 line referring to the issue that it resolves (to automatically close the issue when thebranch is merged). Make sure Buildbot indicates all tests pass.

4.1. Contributing to Flocker 29

https://waffle.io/clusterhq/flocker


5. Address any points raised by the reviewer. If a re-submission for review has been requested, change the labelfrom in progress to review in GitHub (or drag it to the Ready for Review column in Waffle) and go backto step 4.

6. Once it is approved, merge the branch into master by clicking the Merge button.

7. As a small thank you for contributing to Flocker, we’d like to send you some ClusterHQ swag. Once your pullrequest has been merged, just send an email to [email protected] with your t-shirt size, mailing addressand a phone number to be used only for filling out the shipping form. We’ll get something in the mail to you.

Steps to Contribute Reviews

1. Pick a pull request in GitHub/Waffle that is ready for review (review label/Review category).

2. Use the continuous integration information in the PR to verify the test suite is passing.

3. Verify the code satisfies the Requirements for Contribution (see above).

4. Verify the change satisfies the requirements specified on the issue.

5. Think hard about whether the code is good or bad.

6. Leave comments on the GitHub PR page about any of these areas where you find problems.

7. Leave a comment on the GitHub PR page explicitly approving or rejecting the change. If you accept the PRand no final changes are required then use the GitHub merge button to merge the branch. If you accept the PRbut changes are needed move it to the Review Passed column in Waffle or change its label from review toapproved. If you do not accept the PR move it to the In Progress column in Waffle or change its label fromreview to in progress.

4.2 Infrastructure

Contents:

4.2.1 Vagrant

There is a Vagrantfile in the base of the repository, that is preinstalled with all of the dependencies required torun flocker.

See the vagrant documentation for more details.

Base Image

The box the above Vagrantfile is based on is generated from vagrant/base/Vagrantfile. The box is ini-tialized with the yum repositories for zfs and for dependencies not available in fedora and installs all the dependencies.

To build the box, run the following commands in the vagrant/base directory:

vagrant upvagrant package --output flocker-dev-$(python ../../setup.py --version).boxvagrant destroy

This will generate a package-<version>.box.

Upload this file to google cloud storage, using gsutil:


mailto:[email protected]

http://docs.vagrantup.com/v2/

https://console.developers.google.com/project/apps~hybridcluster-docker/storage/clusterhq-vagrant/

https://developers.google.com/storage/docs/gsutil?csw=1


gsutil cp -a public_read flocker-dev-$(python ../../setup.py --version).box gs://clusterhq-vagrant/

Then add a version on vagrantcloud. The version on vagrantcloud should be the version with - replaced with ..

Testing

It is possible to test this image locally before uploading. First add the box locally:

vagrant box add --name clusterhq/flocker-dev flocker-dev-$(python ../../setup.py --version).box

This adds the box with version 0. Then change config.vm.box_version to = 0 in the Vagrantfile in thebase of the repository, and then destroy and re-up that vagrant image.

4.2.2 Building RPMs

To build flocker RPMs, run the following commands:

python setup.py sdistpython setup.py generate_speccp dist/Flocker-$(python setup.py --version).tar.gz ~/rpmbuild/SOURCESsudo yum-builddep flocker.specrpmbuild -ba flocker.spec

The above commands require the rpmdevtools and yum-utils packages installed.

Flocker depends on a number of packages which aren’t available in fedora, or newer versions than are availablethere. These packages are available from our copr repository. To enable yum to find them, put the repo file in/etc/yum.repos.d/.

4.2.3 Release Process

Outcomes

By the end of the release process we will have:

• a tag in version control

• a Python wheel in the ClusterHQ package index

• Fedora 20 RPMs for software on the node and client

• documentation on docs.clusterhq.com

• announcement on mailing list, blog, IRC (others?)

• download links on clusterhq.com

Prerequisites

Software

• Fedora 20 (rpmbuild, createrepo, yumdownloader) - might be possible to install these on Ubuntu though

You are advised to perform the release from a flocker development machine, which will have all the requisitesoftware pre-installed.

4.2. Infrastructure 31

https://vagrantcloud.com/clusterhq/flocker-dev

https://copr.fedoraproject.org/coprs/tomprince/hybridlogic/

https://copr.fedoraproject.org/coprs/tomprince/hybridlogic/repo/fedora-20-x86_64/tomprince-hybridlogic-fedora-20-x86_64.repo

http://archive.clusterhq.com

https://docs.clusterhq.com


• a web browser

• an IRC client

• an up-to-date clone of the Flocker repository

Access

• A readthedocs account (registration), with maintainer access to the Flocker project.

• Ability to change topic in #clusterhq. Ensure that you have +t next to your nickname in the output of:

/msg ChanServ access list #clusterhq

Somebody with +f can grant access by running:

/msg ChanServ access add #clusterhq <nickname> +t

• Access to Google cloud storage using gsutil.

Preliminary Step: Pre-populating RPM Repository

This only needs to be done if the dependency packages for Flocker (i.e. geard and Python libraries) change; it shouldnot be done every release. If you do run this you need to do it before running the release process above as it removesthe flocker-cli etc. packages from the repository!

These steps must be performed from a machine with the ClusterHQ copr repo installed. You caneither use the Flocker development enviroment or install the copr repo locally by running curlhttps://copr.fedoraproject.org/coprs/tomprince/hybridlogic/repo/fedora-20-x86_64/tomprince-hybridlogic-fedora-20-x86_64.repo>/etc/yum.repos.d/hybridlogic.repo

mkdir repoyumdownloader --destdir=repo geard python-characteristic python-eliot python-idna python-netifaces python-service-identity python-treq python-twistedcreaterepo repogsutil cp -a public-read -R repo gs://archive.clusterhq.com/fedora/20/x86_64

mkdir srpmyumdownloader --destdir=srpm --source geard python-characteristic python-eliot python-idna python-netifaces python-service-identity python-treq python-twistedcreaterepo srpmgsutil cp -a public-read -R srpm gs://archive.clusterhq.com/fedora/20/SRPMS

Preparing for a release

1. Choose a version number:

• Release numbers should be of the form x.y.z eg:

export VERSION=0.0.3

• Sanity check the proposed version number by checking the last version. Check the ClusterHQ websitefor the last released version. You might also double check the current version by running the followingcommands:

$ python setup.py --version0.0.1-576-ge15c6be

$ git tag


https://github.com/ClusterHQ/flocker.git

https://readthedocs.org/accounts/signup/

https://readthedocs.org/dashboard/flocker/users/

https://console.developers.google.com/project/apps~hybridcluster-docker/storage/archive.clusterhq.com/

https://developers.google.com/storage/docs/gsutil


...0.0.6

2. In a clean, local working copy of Flocker with no modifications, checkout the branch for the release:

Note: All releases of the x.y series will be made from the releases/flocker-x.y branch.

• If this is a major or minor release then create the branch for the minor version:

git checkout -b release/flocker-${VERSION%.*} origin/mastergit push origin --set-upstream release/flocker-${VERSION%.*}

• If this is a patch release then there will already be a branch:

$ git checkout -b release/flocker-${VERSION%.*} origin/release/flocker-"${VERSION%.*}"

3. Update the version number in the downloads in docs/gettingstarted/linux-install.shand docs/gettingstarted/osx-install.sh, as well as the two RPMs indocs/gettingstarted/tutorial/Vagrantfile (a total of 4 locations).

4. Commit the changes:

git commit -am"Bumped version number in installers and Vagrantfiles"git push

5. Ensure the release notes in NEWS are up-to-date.

XXX: Process to be decided. See https://github.com/ClusterHQ/flocker/issues/523

6. Ensure copyright dates in LICENSE are up-to-date.

XXX: Process to be decided. If we modify the copyright in the release branch, then we’ll needto merge that back to master. It should probably just be updated routinely each year. Seehttps://github.com/ClusterHQ/flocker/issues/525

7. Ensure all the tests pass on BuildBot. Go to the BuildBot web status and force a build on the just-created branch.

8. Do the acceptance tests. (https://github.com/ClusterHQ/flocker/issues/315)

Release

1. Change your working directory to be the Flocker release branch checkout.

2. Create (if necessary) and activate the Flocker release virtual environment:

virtualenv ~/Environments/flocker-release. ~/Environments/flocker-release/bin/activatepip install --editable .[release]

3. Tag the version being released:

git tag --annotate "${VERSION}" release/flocker-"${VERSION%.*}" -m "Tag version ${VERSION}"git push origin "${VERSION}"

4. Go to the BuildBot web status and force a build on the tag.

Note: We force a build on the tag as well as the branch because the RPMs built before pushing the tag won’thave the right version. Also, the RPM upload script currently expects the RPMs to be built from the tag, ratherthan the branch.

You force a build on a tag by putting the tag name into the branch box (without any prefix).

4.2. Infrastructure 33



http://build.clusterhq.com/boxes-flocker


http://build.clusterhq.com/boxes-flocker


5. Set up gsutil authentication by following the instructions from the following command:

$ gsutil config

6. Build python packages for upload, and upload them to archive.clusterhq.com, as well as uploading the RPMs:

python setup.py bdist_wheelgsutil cp -a public-read dist/Flocker-"${VERSION}"-py2-none-any.whl gs://archive.clusterhq.com/downloads/flocker/admin/upload-rpms "${VERSION}"

7. Build tagged docs at readthedocs.org:

(a) Go to the readthedocs dashboard.

(b) Enable the version being released.

(c) Set the default version to that version.

(d) Force readthedocs.org to reload the repository, in case the GitHub webhook fails, by running:

curl -X POST http://readthedocs.org/build/flocker

8. Make a Pull Request on GitHub for the release branch against master, with a Fixes #123 line in thedescription referring to the release issue that it resolves.

Announcing Releases

• Announcement

– on the mailing list - https://groups.google.com/forum/#!forum/flocker-users

– on the blog - https://clusterhq.com/blog/

– on the IRC channel - #clusterhq on freenode

• Update download links on clusterhq.com

XXX Arrange to have download links on a page on clusterhq.com somewhere. Seehttps://github.com/ClusterHQ/flocker/issues/359 and https://github.com/ClusterHQ/flocker/issues/488


https://readthedocs.org/dashboard/flocker/versions/

https://groups.google.com/forum/#!forum/flocker-users

https://clusterhq.com/blog/



CHAPTER 5

Areas of Potential Future Development

Flocker is an ongoing project whose direction will be guided in large part by the community. The list below includessome potential areas for future development. It is not all encompassing or indicative of what definitely will be built.Feedback welcome and encouraged.

• Support for atomic updates.

• Scale-out for stateless containers.

• API to support managing Flocker volumes programmatically.

• Statically configured continuous replication and manual failover.

• No-downtime migrations between containers.

• Automatically configured continuous replication and failover.

• Multi-data center support.

• Automatically balance load across cluster.

• Roll-back a container to a snapshot.

Detailed plans have been made in some areas:

5.1 Flocker Volume Manager

The Flocker Volume Manager (FVM) provides snapshotting and replication of Flocker volumes. It has the ability topush volumes to remote nodes, track changes to those volumes, and roll them back to earlier states.

Although initially built on top of ZFS, FVM should eventually be capable of being backed on a number of filesystems.As such a generic data model is required.

5.1.1 Data Model

Motivation:

• ZFS has some peculiarities in its model when it comes to clones, e.g. promoting a clone moves snapshots fromoriginal dataset to the clone.

• Having clones be top-level constructs on the same level as originating dataset is a problem, since they are closelytied to each other both in terms of usage and in administrative “cleaning up old data” way.

• We don’t want to be too tied to the ZFS model (or terminology!) in case we want to switch to btrfs or someother system. Especially given conflicting terminology - Btrfs “snapshots” are the same as ZFS “clones”.

35


• When it comes to replication, it is probably useful to differentiate between “data which is a copy of what theremote host has” and “local version”, in particular when divergence is a potential issue (e.g. can be caused byerroneous failover). In git you have “origin/branchname” vs. the local “branchname”, for example.

We are therefore going to be using the following model for CLI examples below:

• A “volume” is a tree of “branches”.

• A “tag” is a named read-only pointer to the contents of a branch at a given point in time; it is attached to thevolume, and is not mounted on the filesystem.

• Given volume called “mydata”, “mydata/trunk” is (by convention) is the main branch from which other branchesoriginate, “mydata/branchname” is some other branch, and “mytag@mydata” is a tag.

• Branches’ full name includes the Flocker instance they came from (by default let’s say using itshostname), e.g. “somehost/myvolume/trunk”. “dataset/branch” is shorthand for the current host,e.g. “thecurrenthost.example.com/dataset/branch”. In a replication scenario we could have “re-mote.example.com/datavolume/trunk” and “thecurrenthost.example.com/datavolume/trunk” (aka “datavol-ume/trunk”) as a branch off of that.

• Local branches are mounted on the filesystem, and then exposed to Docker, e.g. “myvolume/trunk” is exportedvia a docker container called “flocker:myvolume/trunk” (“flocker:” prefix isn’t a Docker feature, just a proposedconvention for naming our containers).

• Remote branches are not mounted, but a local branch can be created off of them and then that is auto-mounted.

Implementation Notes - ZFS

The names of volumes, branches and tags do not map directly onto the ZFS naming system.

Each flocker instance has a UUID, with a matching (unique across a Flocker cluster) human readable name, typicallythe hostname. We can imagine having two flocker instances on same machine (with different pools) for testing, sodon’t want to require hostname. This is the first part of the <flocker instance uuid>/<volume uuid>/<branch name>triplet of branch names - in human-exposed CLI we probably want to use human names though, not UUIDs. Branchesare known to be local if branch’s specified flocker instance matches the UUID of flocker process that is managing it.

Volumes have UUIDs, and a matching (cluster unique?) human readable name. Tags are indicated by having asnapshot with a user attributes indicating it is a tag, the tag name and the volume name. However, not all ZFSsnapshots will be exposed as tags. E.g. the fact that a snapshot is necessary for cloning (and therefore branch creation)is an implementation detail; sometimes you want to branch off a tag, but if you want to branch off of latest version thefact that a snapshot is created needn’t be exposed.

A remote branch exists if there is a non-tag ZFS snapshot naming it, i.e. the snapshot has a user attribute indicatingwhich branch it’s on (e.g. “thathost/somevolume/abranch”).

In either case the ZFS-level snapshot name is the flocker instance UUID + the timestamp when it was generated.

A local branch exists due to local existence ZFS dataset, one of:

1. A root dataset (“trunk”), if this is the primary host (whatever that means).

2. A clone of a remote branch snapshot.

3. A clone of a local branch snapshot.

The branch name is stored as a user attribute on the ZFS dataset. Dataset names can be the branch human readablenames, since only one Flocker instance will ever be setting them.

In cases where we can’t use attributes the data will be in a local database of some sort. E.g. ZFS properties areinherited automatically (not the behavior we want), which might lead to some corrupt state in crashes if the low-levelAPIs don’t allow bypassing this. . .

36 Chapter 5. Areas of Potential Future Development

mailto:mytag@mydata


Implementation Notes - Btrfs

Btrfs does not have a concept of clones - it just has snapshots, and they are mounted and writeable. As such theproposed model should also work with btrfs. Btrfs appears to lack promotion, but that can be emulated via renames.It’s not clear if btrfs has the “can’t delete parent if it has children” restriction, though it may just keep around extradisk storage in that case.

5.1. Flocker Volume Manager 37


38 Chapter 5. Areas of Potential Future Development

CHAPTER 6

Authors

Flocker is maintained by ClusterHQ and is licensed under the Apache Software License 2.0.

The following people and organizations contributed to its development; please add your name in alphabetical orderwith your first pull request:

• ClusterHQ (formerly Hybrid Logic Ltd.)

• Scotch Media (Base for documentation theme)

39

flocker documentation - read the docs · 2019-04-02 · chapter 1 introduction 1.1motivation for...

Documents