raspberry pi swarm cluster

Sample Cases - RPIs

Docker SwarmEueung Mulyana

http://eueung.github.io/docker-stuff/clusterCodeLabs | Attribution-ShareAlike CC BY-SA

1 / 30

Outline

Swarm on RPIs - A 3-Node Cluster

Docker Machine - Generic Driver

2 / 30

Docker Swarm on RPIs

A 3-Node Cluster

3 / 30

$ flash --hostname node1 hypriot-rpi-20160306-192317.img $ flash --hostname node2 hypriot-rpi-20160306-192317.img $ flash --hostname node3 hypriot-rpi-20160306-192317.img

# -d /dev/mmcblk0 doesn't work

4 / 30

Step #1

Prepare & Flash

5 / 30

Step #2

ConnectEverythingPlug & Power-Up

$ ping -c 3 node1.localPING node1.local (192.168.1.102) 56(84) bytes of data.64 bytes from 192.168.1.102: icmp_seq=1 ttl=64 time=3.51 ms...--- node1.local ping statistics ---3 packets transmitted, 3 received, 0% packet loss, time 6026msrtt min/avg/max/mdev = 1.673/2.859/3.511/0.840 ms

$ ping -c 3 node2.localPING node2.local (192.168.1.101) 56(84) bytes of data.64 bytes from 192.168.1.101: icmp_seq=1 ttl=64 time=3.60 ms...--- node2.local ping statistics ---3 packets transmitted, 3 received, 0% packet loss, time 6025msrtt min/avg/max/mdev = 1.806/2.491/3.608/0.796 ms

$ ping -c 3 node3.localPING node3.local (192.168.1.100) 56(84) bytes of data.64 bytes from 192.168.1.100: icmp_seq=1 ttl=64 time=3.09 ms...--- node3.local ping statistics ---3 packets transmitted, 3 received, 0% packet loss, time 2003msrtt min/avg/max/mdev = 1.684/2.186/3.090/0.640 ms

6 / 30

Step #3

TestConnectivity

$ ssh root@node1.local$ ssh root@node2.local$ ssh root@node3.local# root/hypriot

HypriotOS: root@node1 in ~

HypriotOS: root@node1 in ~$ docker version

Client: Version: 1.10.2 API version: 1.22 Go version: go1.4.3 Git commit: c3959b1 Built: Wed Feb 24 09:51:38 2016 OS/Arch: linux/arm

Server: Version: 1.10.2 API version: 1.22 Go version: go1.4.3 Git commit: c3959b1 Built: Wed Feb 24 09:51:38 2016 OS/Arch: linux/arm

$ docker-machine --versiondocker-machine version 0.4.1 (549d016)

7 / 30

Step #4

CheckDocker

HypriotOS: root@node1 in ~$ docker info

Containers: 0 Running: 0 Paused: 0 Stopped: 0Images: 25Server Version: 1.10.2Storage Driver: overlay Backing Filesystem: extfsExecution Driver: native-0.2Logging Driver: json-filePlugins: Volume: local Network: null host bridgeKernel Version: 4.1.17-hypriotos-v7+Operating System: Raspbian GNU/Linux 8 (jessie)OSType: linuxArchitecture: armv7lCPUs: 4Total Memory: 925.5 MiBName: node1ID: HX6D:ZAUS:RCVC:OA6Y:NTU6:KJ2P:KUIF:BE6Q:ANIX:U3NE:CMDO:TCHX

8 / 30

Step #4

CheckDocker

HypriotOS: root@node1 in ~

$ docker pull hypriot/rpi-swarmUsing default tag: latestlatest: Pulling from hypriot/rpi-swarm

a3ed95caeb02: Pull complete f12f883ff802: Pull complete Digest: sha256:058810659b2cfa26e68aabd9499fe5eb9b70b2d0bc753cdcafcff00359049609Status: Downloaded newer image for hypriot/rpi-swarm:latest

$ docker imagesREPOSITORY TAG IMAGE ID CREATED SIZEhypriot/rpi-swarm latest 07fafcb46ec6 31 hours ago 13.97<none> <none> c298de062190 8 weeks ago 13.27

9 / 30

Step #5

UpdateSwarm

$ ssh root@node1.local$ ssh-keygen -t rsa -C "root@node1"Generating public/private rsa key pair.Enter file in which to save the key (/root/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /root/.ssh/id_rsa.Your public key has been saved in /root/.ssh/id_rsa.pub.The key fingerprint is:d8:76:66:1a:45:5f:9e:9f:7d:cd:21:ca:55:c7:1d:be root@node1The key's randomart image is:+---[RSA 2048]----+| . .++|| . . oo.+|| . .ooo || o .. o .o*|| . S +o E*|| . * .|| . || || |+-----------------+

# no passphrase# if necessary# ssh-keygen -f "~/.ssh/known_hosts" -R node1.local

$ ssh-copy-id -i ~/.ssh/id_rsa.pub root@node1.local$ ssh-copy-id -i ~/.ssh/id_rsa.pub root@node2.local$ ssh-copy-id -i ~/.ssh/id_rsa.pub root@node3.local

10 / 30

Step #6

Prepare forMachine

Operation

$ sudo apt-get install bc

# generate-token.sh# ------#!/bin/bash

export TOKEN=$(for i in $(seq 1 32); do echo -n $(echo "obase=16; $(($RANDOM % 16))" | bc); echo $TOKEN

# ------# cluster.token# ------export TOKEN=405C5E6893E28BE726F89C08BB1666CF

# ------$ source cluster.token && echo $TOKEN

11 / 30

Step #7

GenerateToken

Using Docker Hub DiscoveryBackend Service

HypriotOS: root@node1 in ~$ docker-machine create -d hypriot --swarm --swarm-master --swarm-discovery token://$TOKENImporting SSH key...To see how to connect Docker to this machine, run: docker-machine env node1

HypriotOS: root@node1 in ~$ docker-machine create -d hypriot --swarm --swarm-discovery token://$TOKEN --hypriot-ip-address Importing SSH key...To see how to connect Docker to this machine, run: docker-machine env node2

HypriotOS: root@node1 in ~$ docker-machine create -d hypriot --swarm --swarm-discovery token://$TOKEN --hypriot-ip-address Importing SSH key...To see how to connect Docker to this machine, run: docker-machine env node3

HypriotOS: root@node1 in ~$ docker-machine lsNAME ACTIVE DRIVER STATE URL SWARMnode1 hypriot Running tcp://192.168.1.102:2376 node1 (master)node2 hypriot Running tcp://192.168.1.101:2376 node1node3 hypriot Running tcp://192.168.1.100:2376 node1

12 / 30

Step #8

ExecuteMachine

HypriotOS: root@node1 in ~$ eval $(docker-machine env --swarm node1)

HypriotOS: root@node1 in ~$ docker psCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMESaf2f6f431141 hypriot/rpi-swarm:latest "/swarm join --advert" 5 minutes ago Up 36a7229cad5a hypriot/rpi-swarm:latest "/swarm join --advert" About an hour ago Up About an hour fbc861a660de hypriot/rpi-swarm:latest "/swarm join --advert" About an hour ago Up About an hour 9e849976b10b hypriot/rpi-swarm:latest "/swarm manage --tlsv" About an hour ago Up About an hour

13 / 30

Step #9

Ready forTesting

HypriotOS: root@node1 in ~$ docker run -d -p 8080:80 hypriot/rpi-nano-httpd01be89a41fdc7c3caadb4f985e9cf43161a15f991e06ef2d47fe7e6c6c79e32b

$ docker run -d -p 8080:80 hypriot/rpi-nano-httpd628ba526621af33ca289303f6f54a80c4fe139bcece38e202ff4ea5bd4ec8e5a

$ docker psCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES628ba526621a hypriot/rpi-nano-httpd "/httpd 80" 14 seconds ago Up 01be89a41fdc hypriot/rpi-nano-httpd "/httpd 80" About a minute ago Up About a minute af2f6f431141 hypriot/rpi-swarm:latest "/swarm join --advert" 10 minutes ago Up 36a7229cad5a hypriot/rpi-swarm:latest "/swarm join --advert" About an hour ago Up About an hour fbc861a660de hypriot/rpi-swarm:latest "/swarm join --advert" About an hour ago Up About an hour 9e849976b10b hypriot/rpi-swarm:latest "/swarm manage --tlsv" About an hour ago Up About an hour

14 / 30

Step #9

Ready forTesting

HypriotOS: root@node1 in ~$ docker info

Containers: 6 Running: 6 - Paused: 0 - Stopped: 0Images: 5Server Version: swarm/1.2.1 - Role: primaryStrategy: spreadFilters: health, port, containerslots, dependency, affinity, constraintNodes: 3 node1: 192.168.1.102:2376 - ID: MHER:4BEC:J5ER:3TJL:IH55:UFWY:ONGH:2S4I:R4VP:UNS5:NTG4:FJ3J - Status: Healthy - Containers: 2 - Reserved CPUs: 0 / 4 - Reserved Memory: 0 B / 971.8 MiB - Labels: executiondriver=native-0.2, kernelversion=4.1.17-hypriotos-v7+, operatingsystem=Raspbian GNU/Linux - Error: (none) - UpdatedAt: 2016-05-08T01:42:11Z - ServerVersion: 1.10.2 node2: 192.168.1.101:2376 - Containers: 2 - Reserved CPUs: 0 / 4 - Reserved Memory: 0 B / 971.8 MiB node3: 192.168.1.100:2376 - Containers: 2 - Reserved CPUs: 0 / 4 - Reserved Memory: 0 B / 971.8 MiB...Kernel Version: 4.1.17-hypriotos-v7+ - Operating System: linux - Architecture: armCPUs: 12Total Memory: 2.847 GiBName: 9e849976b10b

15 / 30

Step #9

Ready forTesting

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES628ba526621a hypriot/rpi-nano-httpd "/httpd 80" 14 seconds ago Up 01be89a41fdc hypriot/rpi-nano-httpd "/httpd 80" About a minute ago Up About a minute

16 / 30

Step #9

Ready forTesting

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES628ba526621a hypriot/rpi-nano-httpd "/httpd 80" 14 seconds ago Up 01be89a41fdc hypriot/rpi-nano-httpd "/httpd 80" About a minute ago Up About a minute

17 / 30

Step #9

Ready forTesting

Generic Driver

More with Docker Machine

18 / 30

$ docker-machine versiondocker-machine version 0.7.0, build a650a40

$ source cluster.token && echo $TOKEN

$ ./prepare-node.sh node1.local$ ./prepare-node.sh node2.local$ ./prepare-node.sh node3.local

#!/bin/bash# prepare-node.sh

function getip() { (traceroute $1 2>&1 | head -n 1 | cut -d\( -f 2 | cut -d\) -f 1) }

IP_ADDRESS=$(getip $1)#echo $IP_ADDRESS

ssh-keygen -R $IP_ADDRESSssh-copy-id -oStrictHostKeyChecking=no -oCheckHostIP=no root@$IP_ADDRESS

ssh root@$IP_ADDRESS sed -i \'s/ID=raspbian/ID=debian/g\' /etc/os-release

# -----# cluster.tokenexport TOKEN=405C5E6893E28BE726F89C08BB1666FE

19 / 30

Prepare theNodes

#!/bin/bash# activate-master.sh

function getip() { (traceroute $1 2>&1 | head -n 1 | cut -d\( -f 2 | cut -d\) -f 1) }

IP_ADDRESS=$(getip $1)echo $IP_ADDRESSecho $TOKEN

docker-machine -D create -d generic \ --engine-storage-driver=overlay --swarm --swarm-master \ --swarm-image hypriot/rpi-swarm:1.2.1 \ --swarm-discovery="token://$TOKEN" \ --generic-ip-address=$(getip $1) \ $1

20 / 30

ActivateMaster Node

$ ./activate-master.sh node1.local

Running pre-create checks...Creating machine...(node1.local) No SSH key specified. Connecting to this machine now and in the future will require the ssh agent to contain the appropriate key.Waiting for machine to be running, this may take a few minutes...Detecting operating system of created instance...Waiting for SSH to be available...Detecting the provisioner...Provisioning with debian...Copying certs to the local machine directory...Copying certs to the remote machine...Setting Docker configuration on the remote daemon...Configuring swarm......

21 / 30

ActivateMaster Node

ssh root@node1.local docker rm ..

docker rmi ..

docker pull hypriot/rpi-swarm:1.2.1

$ ./activate-agent.sh node2.local$ ./activate-agent.sh node3.local...

#!/bin/bash# activate-agent.sh

function getip() { (traceroute $1 2>&1 | head -n 1 | cut -d\( -f 2 | cut -d\) -f 1) }

IP_ADDRESS=$(getip $1)echo $IP_ADDRESSecho $TOKEN

docker-machine -D create -d generic \ --engine-storage-driver=overlay --swarm \ --swarm-image hypriot/rpi-swarm:1.2.1 \ --swarm-discovery="token://$TOKEN" \ --generic-ip-address=$(getip $1) \ $1

22 / 30

ActivateAgents

ssh root@node1.local docker rm ..

docker rmi ..

docker pull hypriot/rpi-swarm:1.2.1

$ docker-machine lsNAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORSagent1 - virtualbox Stopped Unknown agent2 - virtualbox Stopped Unknown default - virtualbox Stopped Unknown manager - virtualbox Stopped Unknown node1.local * (swarm) generic Running tcp://192.168.1.102:2376 node1.local (master) v1.node2.local - generic Running tcp://192.168.1.101:2376 node1.local v1.

23 / 30

Check

$ curl -sL https://download.getcarina.com/dvm/latest/install.sh | sh

Downloading dvm.sh...######################################################################## 100,0%Downloading bash_completion######################################################################## 100,0%Downloading dvm-helper...######################################################################## 100,0%

Docker Version Manager (dvm) has been installed to /home/em/.dvmRun the following command to start using dvm. Then add it to your bash profile (e.g. ~/.bashrc or ~/.bash_profile) to complete the installation.

source /home/em/.dvm/dvm.sh

$ source /home/em/.dvm/dvm.sh$ dvm install 1.10.2Installing 1.10.2...Now using Docker 1.10.2

24 / 30

Install DVMDocker (Client) Version Manager

$ docker version

Client: Version: 1.10.2 API version: 1.22 Go version: go1.5.3 Git commit: c3959b1 Built: Mon Feb 22 22:37:33 2016 OS/Arch: linux/amd64

Server: Version: 1.11.1 API version: 1.23 Go version: go1.5.4 Git commit: 5604cbe Built: Tue Apr 26 23:43:49 2016 OS/Arch: linux/amd64

$ dvm ls-> 1.10.2 system (1.11.1)$ dvm use 1.10.2

25 / 30

Install DVMDocker (Client) Version Manager

$ eval $(docker-machine env --swarm node1.local)

$ docker infoContainers: 3 Running: 3 - Paused: 0 - Stopped: 0Images: 6Server Version: swarm/1.2.1Role: primaryStrategy: spreadFilters: health, port, containerslots, dependency, affinity, constraintNodes: 2 node1.local: 192.168.1.102:2376 - Containers: 2 - Reserved CPUs: 0 / 4 - Reserved Memory: 0 B / 971.8 MiB - ServerVersion: 1.10.2 node2.local: 192.168.1.101:2376 - Containers: 1 - Reserved CPUs: 0 / 4Kernel Version: 4.1.17-hypriotos-v7+ - Operating System: linux - Architecture: armCPUs: 8 - Total Memory: 1.898 GiB

26 / 30

Ready forTesting

$ docker run -d -p 8080:80 hypriot/rpi-nano-httpd95458ecf5c3fe3577f92cbbb408bc02facef7de4618f266717805e0422f8e30b

$ docker psCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES95458ecf5c3f hypriot/rpi-nano-httpd "/httpd 80" 8 seconds ago Up df3a453ac694 hypriot/rpi-swarm:1.2.1 "/swarm join --advert" 17 minutes ago Up c0d26b7f6d6f hypriot/rpi-swarm:1.2.1 "/swarm join --advert" 45 minutes ago Up c796d9b32052 202851b321f9 "/swarm manage --tlsv" 46 minutes ago Up

27 / 30

Ready forTesting

Refs

28 / 30

Refs1. Installing Docker Swarm on HypriotOS2. Let Docker Swarm all over your Raspberry Pi Cluster3. How to setup a Docker Swarm cluster with Raspberry Pi's4. getcarina/dvm: Docker Version Manager5. Manage Docker clients with the Docker Version Manager | Carina by

Rackspace

29 / 30

ENDEueung Mulyana

http://eueung.github.io/docker-stuff/clusterCodeLabs | Attribution-ShareAlike CC BY-SA

30 / 30