raspberry pi swarm cluster
TRANSCRIPT
Sample Cases - RPIs
Docker SwarmEueung Mulyana
http://eueung.github.io/docker-stuff/clusterCodeLabs | Attribution-ShareAlike CC BY-SA
1 / 30
Outline
Swarm on RPIs - A 3-Node Cluster
Docker Machine - Generic Driver
2 / 30
Docker Swarm on RPIs
A 3-Node Cluster
3 / 30
$ flash --hostname node1 hypriot-rpi-20160306-192317.img $ flash --hostname node2 hypriot-rpi-20160306-192317.img $ flash --hostname node3 hypriot-rpi-20160306-192317.img
# -d /dev/mmcblk0 doesn't work
4 / 30
Step #1
Prepare & Flash
5 / 30
Step #2
ConnectEverythingPlug & Power-Up
$ ping -c 3 node1.localPING node1.local (192.168.1.102) 56(84) bytes of data.64 bytes from 192.168.1.102: icmp_seq=1 ttl=64 time=3.51 ms...--- node1.local ping statistics ---3 packets transmitted, 3 received, 0% packet loss, time 6026msrtt min/avg/max/mdev = 1.673/2.859/3.511/0.840 ms
$ ping -c 3 node2.localPING node2.local (192.168.1.101) 56(84) bytes of data.64 bytes from 192.168.1.101: icmp_seq=1 ttl=64 time=3.60 ms...--- node2.local ping statistics ---3 packets transmitted, 3 received, 0% packet loss, time 6025msrtt min/avg/max/mdev = 1.806/2.491/3.608/0.796 ms
$ ping -c 3 node3.localPING node3.local (192.168.1.100) 56(84) bytes of data.64 bytes from 192.168.1.100: icmp_seq=1 ttl=64 time=3.09 ms...--- node3.local ping statistics ---3 packets transmitted, 3 received, 0% packet loss, time 2003msrtt min/avg/max/mdev = 1.684/2.186/3.090/0.640 ms
6 / 30
Step #3
TestConnectivity
$ ssh [email protected]$ ssh [email protected]$ ssh [email protected]# root/hypriot
HypriotOS: root@node1 in ~
HypriotOS: root@node1 in ~$ docker version
Client: Version: 1.10.2 API version: 1.22 Go version: go1.4.3 Git commit: c3959b1 Built: Wed Feb 24 09:51:38 2016 OS/Arch: linux/arm
Server: Version: 1.10.2 API version: 1.22 Go version: go1.4.3 Git commit: c3959b1 Built: Wed Feb 24 09:51:38 2016 OS/Arch: linux/arm
$ docker-machine --versiondocker-machine version 0.4.1 (549d016)
7 / 30
Step #4
CheckDocker
HypriotOS: root@node1 in ~$ docker info
Containers: 0 Running: 0 Paused: 0 Stopped: 0Images: 25Server Version: 1.10.2Storage Driver: overlay Backing Filesystem: extfsExecution Driver: native-0.2Logging Driver: json-filePlugins: Volume: local Network: null host bridgeKernel Version: 4.1.17-hypriotos-v7+Operating System: Raspbian GNU/Linux 8 (jessie)OSType: linuxArchitecture: armv7lCPUs: 4Total Memory: 925.5 MiBName: node1ID: HX6D:ZAUS:RCVC:OA6Y:NTU6:KJ2P:KUIF:BE6Q:ANIX:U3NE:CMDO:TCHX
8 / 30
Step #4
CheckDocker
HypriotOS: root@node1 in ~
$ docker pull hypriot/rpi-swarmUsing default tag: latestlatest: Pulling from hypriot/rpi-swarm
a3ed95caeb02: Pull complete f12f883ff802: Pull complete Digest: sha256:058810659b2cfa26e68aabd9499fe5eb9b70b2d0bc753cdcafcff00359049609Status: Downloaded newer image for hypriot/rpi-swarm:latest
$ docker imagesREPOSITORY TAG IMAGE ID CREATED SIZEhypriot/rpi-swarm latest 07fafcb46ec6 31 hours ago 13.97<none> <none> c298de062190 8 weeks ago 13.27
9 / 30
Step #5
UpdateSwarm
$ ssh [email protected]$ ssh-keygen -t rsa -C "root@node1"Generating public/private rsa key pair.Enter file in which to save the key (/root/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /root/.ssh/id_rsa.Your public key has been saved in /root/.ssh/id_rsa.pub.The key fingerprint is:d8:76:66:1a:45:5f:9e:9f:7d:cd:21:ca:55:c7:1d:be root@node1The key's randomart image is:+---[RSA 2048]----+| . .++|| . . oo.+|| . .ooo || o .. o .o*|| . S +o E*|| . * .|| . || || |+-----------------+
# no passphrase# if necessary# ssh-keygen -f "~/.ssh/known_hosts" -R node1.local
$ ssh-copy-id -i ~/.ssh/id_rsa.pub [email protected]$ ssh-copy-id -i ~/.ssh/id_rsa.pub [email protected]$ ssh-copy-id -i ~/.ssh/id_rsa.pub [email protected]
10 / 30
Step #6
Prepare forMachine
Operation
$ sudo apt-get install bc
# generate-token.sh# ------#!/bin/bash
export TOKEN=$(for i in $(seq 1 32); do echo -n $(echo "obase=16; $(($RANDOM % 16))" | bc); echo $TOKEN
# ------# cluster.token# ------export TOKEN=405C5E6893E28BE726F89C08BB1666CF
# ------$ source cluster.token && echo $TOKEN
11 / 30
Step #7
GenerateToken
Using Docker Hub DiscoveryBackend Service
HypriotOS: root@node1 in ~$ docker-machine create -d hypriot --swarm --swarm-master --swarm-discovery token://$TOKENImporting SSH key...To see how to connect Docker to this machine, run: docker-machine env node1
HypriotOS: root@node1 in ~$ docker-machine create -d hypriot --swarm --swarm-discovery token://$TOKEN --hypriot-ip-address Importing SSH key...To see how to connect Docker to this machine, run: docker-machine env node2
HypriotOS: root@node1 in ~$ docker-machine create -d hypriot --swarm --swarm-discovery token://$TOKEN --hypriot-ip-address Importing SSH key...To see how to connect Docker to this machine, run: docker-machine env node3
HypriotOS: root@node1 in ~$ docker-machine lsNAME ACTIVE DRIVER STATE URL SWARMnode1 hypriot Running tcp://192.168.1.102:2376 node1 (master)node2 hypriot Running tcp://192.168.1.101:2376 node1node3 hypriot Running tcp://192.168.1.100:2376 node1
12 / 30
Step #8
ExecuteMachine
HypriotOS: root@node1 in ~$ eval $(docker-machine env --swarm node1)
HypriotOS: root@node1 in ~$ docker psCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMESaf2f6f431141 hypriot/rpi-swarm:latest "/swarm join --advert" 5 minutes ago Up 36a7229cad5a hypriot/rpi-swarm:latest "/swarm join --advert" About an hour ago Up About an hour fbc861a660de hypriot/rpi-swarm:latest "/swarm join --advert" About an hour ago Up About an hour 9e849976b10b hypriot/rpi-swarm:latest "/swarm manage --tlsv" About an hour ago Up About an hour
13 / 30
Step #9
Ready forTesting
HypriotOS: root@node1 in ~$ docker run -d -p 8080:80 hypriot/rpi-nano-httpd01be89a41fdc7c3caadb4f985e9cf43161a15f991e06ef2d47fe7e6c6c79e32b
$ docker run -d -p 8080:80 hypriot/rpi-nano-httpd628ba526621af33ca289303f6f54a80c4fe139bcece38e202ff4ea5bd4ec8e5a
$ docker psCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES628ba526621a hypriot/rpi-nano-httpd "/httpd 80" 14 seconds ago Up 01be89a41fdc hypriot/rpi-nano-httpd "/httpd 80" About a minute ago Up About a minute af2f6f431141 hypriot/rpi-swarm:latest "/swarm join --advert" 10 minutes ago Up 36a7229cad5a hypriot/rpi-swarm:latest "/swarm join --advert" About an hour ago Up About an hour fbc861a660de hypriot/rpi-swarm:latest "/swarm join --advert" About an hour ago Up About an hour 9e849976b10b hypriot/rpi-swarm:latest "/swarm manage --tlsv" About an hour ago Up About an hour
14 / 30
Step #9
Ready forTesting
HypriotOS: root@node1 in ~$ docker info
Containers: 6 Running: 6 - Paused: 0 - Stopped: 0Images: 5Server Version: swarm/1.2.1 - Role: primaryStrategy: spreadFilters: health, port, containerslots, dependency, affinity, constraintNodes: 3 node1: 192.168.1.102:2376 - ID: MHER:4BEC:J5ER:3TJL:IH55:UFWY:ONGH:2S4I:R4VP:UNS5:NTG4:FJ3J - Status: Healthy - Containers: 2 - Reserved CPUs: 0 / 4 - Reserved Memory: 0 B / 971.8 MiB - Labels: executiondriver=native-0.2, kernelversion=4.1.17-hypriotos-v7+, operatingsystem=Raspbian GNU/Linux - Error: (none) - UpdatedAt: 2016-05-08T01:42:11Z - ServerVersion: 1.10.2 node2: 192.168.1.101:2376 - Containers: 2 - Reserved CPUs: 0 / 4 - Reserved Memory: 0 B / 971.8 MiB node3: 192.168.1.100:2376 - Containers: 2 - Reserved CPUs: 0 / 4 - Reserved Memory: 0 B / 971.8 MiB...Kernel Version: 4.1.17-hypriotos-v7+ - Operating System: linux - Architecture: armCPUs: 12Total Memory: 2.847 GiBName: 9e849976b10b
15 / 30
Step #9
Ready forTesting
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES628ba526621a hypriot/rpi-nano-httpd "/httpd 80" 14 seconds ago Up 01be89a41fdc hypriot/rpi-nano-httpd "/httpd 80" About a minute ago Up About a minute
16 / 30
Step #9
Ready forTesting
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES628ba526621a hypriot/rpi-nano-httpd "/httpd 80" 14 seconds ago Up 01be89a41fdc hypriot/rpi-nano-httpd "/httpd 80" About a minute ago Up About a minute
17 / 30
Step #9
Ready forTesting
Generic Driver
More with Docker Machine
18 / 30
$ docker-machine versiondocker-machine version 0.7.0, build a650a40
$ source cluster.token && echo $TOKEN
$ ./prepare-node.sh node1.local$ ./prepare-node.sh node2.local$ ./prepare-node.sh node3.local
#!/bin/bash# prepare-node.sh
function getip() { (traceroute $1 2>&1 | head -n 1 | cut -d\( -f 2 | cut -d\) -f 1) }
IP_ADDRESS=$(getip $1)#echo $IP_ADDRESS
ssh-keygen -R $IP_ADDRESSssh-copy-id -oStrictHostKeyChecking=no -oCheckHostIP=no root@$IP_ADDRESS
ssh root@$IP_ADDRESS sed -i \'s/ID=raspbian/ID=debian/g\' /etc/os-release
# -----# cluster.tokenexport TOKEN=405C5E6893E28BE726F89C08BB1666FE
19 / 30
Prepare theNodes
#!/bin/bash# activate-master.sh
function getip() { (traceroute $1 2>&1 | head -n 1 | cut -d\( -f 2 | cut -d\) -f 1) }
IP_ADDRESS=$(getip $1)echo $IP_ADDRESSecho $TOKEN
docker-machine -D create -d generic \ --engine-storage-driver=overlay --swarm --swarm-master \ --swarm-image hypriot/rpi-swarm:1.2.1 \ --swarm-discovery="token://$TOKEN" \ --generic-ip-address=$(getip $1) \ $1
20 / 30
ActivateMaster Node
$ ./activate-master.sh node1.local
Running pre-create checks...Creating machine...(node1.local) No SSH key specified. Connecting to this machine now and in the future will require the ssh agent to contain the appropriate key.Waiting for machine to be running, this may take a few minutes...Detecting operating system of created instance...Waiting for SSH to be available...Detecting the provisioner...Provisioning with debian...Copying certs to the local machine directory...Copying certs to the remote machine...Setting Docker configuration on the remote daemon...Configuring swarm......
21 / 30
ActivateMaster Node
ssh [email protected] docker rm ..
docker rmi ..
docker pull hypriot/rpi-swarm:1.2.1
$ ./activate-agent.sh node2.local$ ./activate-agent.sh node3.local...
#!/bin/bash# activate-agent.sh
function getip() { (traceroute $1 2>&1 | head -n 1 | cut -d\( -f 2 | cut -d\) -f 1) }
IP_ADDRESS=$(getip $1)echo $IP_ADDRESSecho $TOKEN
docker-machine -D create -d generic \ --engine-storage-driver=overlay --swarm \ --swarm-image hypriot/rpi-swarm:1.2.1 \ --swarm-discovery="token://$TOKEN" \ --generic-ip-address=$(getip $1) \ $1
22 / 30
ActivateAgents
ssh [email protected] docker rm ..
docker rmi ..
docker pull hypriot/rpi-swarm:1.2.1
$ docker-machine lsNAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORSagent1 - virtualbox Stopped Unknown agent2 - virtualbox Stopped Unknown default - virtualbox Stopped Unknown manager - virtualbox Stopped Unknown node1.local * (swarm) generic Running tcp://192.168.1.102:2376 node1.local (master) v1.node2.local - generic Running tcp://192.168.1.101:2376 node1.local v1.
23 / 30
Check
$ curl -sL https://download.getcarina.com/dvm/latest/install.sh | sh
Downloading dvm.sh...######################################################################## 100,0%Downloading bash_completion######################################################################## 100,0%Downloading dvm-helper...######################################################################## 100,0%
Docker Version Manager (dvm) has been installed to /home/em/.dvmRun the following command to start using dvm. Then add it to your bash profile (e.g. ~/.bashrc or ~/.bash_profile) to complete the installation.
source /home/em/.dvm/dvm.sh
$ source /home/em/.dvm/dvm.sh$ dvm install 1.10.2Installing 1.10.2...Now using Docker 1.10.2
24 / 30
Install DVMDocker (Client) Version Manager
$ docker version
Client: Version: 1.10.2 API version: 1.22 Go version: go1.5.3 Git commit: c3959b1 Built: Mon Feb 22 22:37:33 2016 OS/Arch: linux/amd64
Server: Version: 1.11.1 API version: 1.23 Go version: go1.5.4 Git commit: 5604cbe Built: Tue Apr 26 23:43:49 2016 OS/Arch: linux/amd64
$ dvm ls-> 1.10.2 system (1.11.1)$ dvm use 1.10.2
25 / 30
Install DVMDocker (Client) Version Manager
$ eval $(docker-machine env --swarm node1.local)
$ docker infoContainers: 3 Running: 3 - Paused: 0 - Stopped: 0Images: 6Server Version: swarm/1.2.1Role: primaryStrategy: spreadFilters: health, port, containerslots, dependency, affinity, constraintNodes: 2 node1.local: 192.168.1.102:2376 - Containers: 2 - Reserved CPUs: 0 / 4 - Reserved Memory: 0 B / 971.8 MiB - ServerVersion: 1.10.2 node2.local: 192.168.1.101:2376 - Containers: 1 - Reserved CPUs: 0 / 4Kernel Version: 4.1.17-hypriotos-v7+ - Operating System: linux - Architecture: armCPUs: 8 - Total Memory: 1.898 GiB
26 / 30
Ready forTesting
$ docker run -d -p 8080:80 hypriot/rpi-nano-httpd95458ecf5c3fe3577f92cbbb408bc02facef7de4618f266717805e0422f8e30b
$ docker psCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES95458ecf5c3f hypriot/rpi-nano-httpd "/httpd 80" 8 seconds ago Up df3a453ac694 hypriot/rpi-swarm:1.2.1 "/swarm join --advert" 17 minutes ago Up c0d26b7f6d6f hypriot/rpi-swarm:1.2.1 "/swarm join --advert" 45 minutes ago Up c796d9b32052 202851b321f9 "/swarm manage --tlsv" 46 minutes ago Up
27 / 30
Ready forTesting
Refs
28 / 30
Refs1. Installing Docker Swarm on HypriotOS2. Let Docker Swarm all over your Raspberry Pi Cluster3. How to setup a Docker Swarm cluster with Raspberry Pi's4. getcarina/dvm: Docker Version Manager5. Manage Docker clients with the Docker Version Manager | Carina by
Rackspace
29 / 30
ENDEueung Mulyana
http://eueung.github.io/docker-stuff/clusterCodeLabs | Attribution-ShareAlike CC BY-SA
30 / 30