docker @ relateiq presentation
Post on 08-Sep-2014
2.507 Views
Preview:
DESCRIPTION
TRANSCRIPT
10/17/2013 - Docker @ RelateIQ - Scott Bessler & John Fiedler
Docker @ RelateIQ
@
Scott Bessler scott@relateiq.com@scottbessler
File Repohttps://github.com/relateiq/docker_public
John Fiedlerjohn@relateiq.com@johnfiedler
Blogblog.relateiq.com
RelateIQ Twitter@relateiq
Agenda
● Part 1 - What we did with Docker● Part 2 - Why we want to replace Chef● Part 3 - Future plans using Docker
Part 1. Things were getting out of control... Bring on hack day!
Hack Day!
+
+ =
=
What we needed to build
Kafka
Storm
Redis
Cassandra
MongoDB
Zookeeper
Jetty
Nginx
+ Private Docker Registry
Min
imum
Cru
shin
g IT
Docker Files
Docker Files
Elasticsearch
End product
$ devenv start
$ devenv-inner start
Kafka
Redis
Cassandra
MongoDB
Zookeeper
ubuntu
MAC OS X
OneCommand!
VMContainers
Elasticsearch
Orchestration scripts
● Controls Vagrant● Controls inner script
devenv.sh (outer) [up|update|ssh]
devenv-inner.sh (inner) [stop|start|kill|update|status|restart]
start(){
mkdir -p $APPS/zookeeper/data
mkdir -p $APPS/zookeeper/logs
ZOOKEEPER=$(docker run \
-d \
-p 2181:2181 \
-v $APPS/zookeeper/logs:/logs
server:4444/zookeeper)
echo "Started ZOOKEEPER in
container $ZOOKEEPER"
echo "Wiring containers together… "
(later slide)kill/stop(){
echo "Killing all docker containers:"
docker ps | tail -n +2 |cut -d ' ' -f 1 | xargs docker kill
}
update(){
apt-get update
apt-get install -y lxc-docker
docker pull server:4444/zookeeper
docker pull server:4444/redis
docker pull server:4444/cassandra
docker pull server:4444/elasticsearch
docker pull server:4444/mongo
docker pull server:4444/kafka
}
● Controls Docker containers
Dockerfile’s
FROM server:4444/oracle-java7
RUN apt-get update
RUN apt-get install -y git curl build-essential make gcc wget
RUN cd /opt && wget http://apache.mirrors.pair.com/cassandra/1.2.9/apache-
cassandra-1.2.9-bin.tar.gz
RUN cd /opt && tar zxf apache-cassandra-*.tar.gz
RUN rm /opt/*.tar.gz
RUN mv /opt/apache-cassandra-* /opt/cassandra
RUN apt-get install -y lsof
ADD cassandra.yaml /opt/cassandra/conf/cassandra.yaml
ADD cassandra-env.sh /opt/cassandra/conf/cassandra-env.sh
ADD log4j-server.properties /opt/cassandra/conf/log4j-server.properties
ADD start.sh /opt/cassandra/bin/start.sh
ADD cassandra-topology.properties /opt/cassandra/conf/cassandra-topology.
properties
RUN chmod 755 /opt/cassandra/bin/start.sh
RUN mkdir /logs
...
VOLUME [ "/logs" ]
...
EXPOSE 7000
….
CMD "/opt/cassandra/bin/start.sh"
#!/bin/bash
echo "Cassandra node
configuration:"
echo $CASS_SEEDS
echo $CASS_TOKEN
echo $CASS_LOCAL_IP
HOST=`hostname`
echo "127.0.0.1 $HOST" >>
/etc/hosts
/opt/cassandra/bin/cassandra -fstar
t.sh
Std
Imag
eDockerfile
Networking - port forwarding
Question:Dude, where’s mongo?
Answer:Port forwarding
Example:> Docker file
expose 9999 (dockerfile)> Docker
docker run -p 8000:9999> VAGRANT
config.vm.forward_port 27018, 8000> localhost:8000
Tip:Always use the same port if you can
MongoDB
dockerrun -p 9999:27018
Machine
Containers
localhost:27019
vagrantconfig.vm.forward_port 9999,27019
docker fileexpose 27018
Networking - connecting two containers
KafkaZookeeper
192.168.1.1 192.168.1.2
docker
Machine
Containers
vagrant
pipework br1
Question:Kafka: Dude, where’s Zookeeper?
Answer:Pipework https://github.com/jpetazzo/pipework
Example:> startup the container
KAFKA = docker run -e zooip=192.168.1.1> Pipework
pipework br1 $KAFKA 192.168.1.2 pipework br1 $ZOOKEEPER 192.168.1.1 (dockerfile)
Tip: Use this for clustering
Best practices
● Data● Logs● Don’t end with tailing ● End with foreground
execution● Start script for runtime
configuration● 42 layers, combine
Dockerfile lines● Up Vagrant RAM default
KafkaMongoDB
VM
*Start Script*Foreground Execution*42 layers only
/logs/mongo/data/mongo
/logs/kafka/data/kafka
Machine
Containers
What we found *Abstraction
*Standard Data and log dirs*Static IP
Chef
4 reasons why we want to replace Chef with Docker in prod
Docker Dockerfile● Admittedly less powerful● What you see is what you
get● Environment variables● Inspect environment at
any step
1. Dynamic Configuration Is Too Complex
Chef● Dynamic configuration
through complex attribute system
Docker● Once it’s in, it’s in.● Even if you change an
image, only incremental changes need to be sent to hosts
Chef● Dependencies are
external forever● Slower. You pay the price
on every node deployed● Under load? Want a new
node? Uh oh, the file download is missing.
● Node creation needs to be foolproof
● Or just use chef-solo to bake images
2. External Dependencies Cause Flaky Provisioning
Chef● Configuration change? Try
to apply it to running node● Which changes restart
which services?● Removed something?
○ knife ssh○ tombstone
● Another reason to just bake images
Docker● Ship configuration
changes as image deltas● Nearly instant restarts● Easier to be disciplined
and have each node be identical
3. Configuration Changes Create an Inconsistent State
Dockerfile● Typo? Fix it and start from
the previous successful command
● Containers encourage single-purpose instances (put your monitoring on the host)
Chef● Error? Throw it all out.● Every iteration/test takes:
○ Boot time○ Every step
● Can’t test every host● Can’t test every
combination of cookbooks○ Monitoring○ Logging
4. Developers! Developers! Developers!
Part 3. Whats next? What do we want?
What’s Next for Docker @ RelateIQ?● Replace Chef search (simple node database in
elasticsearch)● Monitoring via StatsD and/or Datadog
Wish List● Dockerfile repo/trusted builds (mentioned on docker-
dev)● Centralized Docker host management● Mac host (no more virtualbox!)
We’re hiring! https://www.relateiq.com/jobs.html
top related