Stop Being Lazy and Test Your Software!

Laura FrankSoftware Engineer, Codeship

0.7.0

Codeship

ImageLayers

Panamax

Berlin

Agenda

Continuous Delivery

Why Do We Test?

Testing with Docker Compose

Faster Testing with Docker

Why Do We Test?Motivations and frustrations

Testing software has been a practice since the very first machines were used

for computing.

The women who programmed the ENIAC to

calculate trajectories periodically checked the

results with a correct, hand-computed table.

working software in production

testingcode reviews

version controlpair programming

high-availability architecture

MotivatorsUpdate application without breaking things!Validate functionality of updatesBe able to trust deployment checks (in CI/CD workflow)Confirm that refactoring didn’t break existing functionality

Why do you skip tests?

45%

15%5%

35% Too slow to run suite 35%Annoying extra step

15%Don’t see the point

5%

Slows down deployment 45%

10

FrustratorsIt takes me an hour to write a single testMy new tests just duplicate existing coverages so there’s no

point (integration overlapping with unit)It takes my test suite over 20 minutes to run, so I don’t run

it locallyTesting distracts me from my normal development workflowSetting up a testing environment is a huge painIt takes too long to deploy when I have a huge test suite

FrustratorsIt takes me an hour to write a single testMy new tests just duplicate existing coverages so there’s no

point (integration overlapping with unit)It takes my test suite over 20 minutes to run, so I don’t run

it locallyTesting distracts me from my normal development workflowSetting up a testing environment is a huge painIt takes too long to deploy when I have a huge test suite

Using Docker can alleviate some frustrations

associated with testing.

Testing with Docker ComposeIt’s easy, I promise

Docker and testing are a great pair.

❌✔

With Docker Compose, it is incredibly easy to create consistent, reproducible environments.

❌✔

Many of us already useDocker Compose to setup dev environments.

…But we stop there.

Use your Docker Composeenvironment for testing

as well.

app: build: . command: rails server -p 3000 -b '0.0.0.0'

volumes: - .:/app

ports: - '3000:3000'

links: - postgres

postgres:image: postgres:9.4 ports:

- '5432'

A simple Rails app

?!

git clone && docker-compose up

hackity hack

How do I make tests happen?

Development workflow

app: build: . command: rails server -p 3000 -b '0.0.0.0' volumes:

- .:/app ports:

- '3000:3000'links:

- postgrespostgres:

image: postgres:9.4 ports:

- '5432'

A simple Rails app

app: build: . command: rspec specvolumes:

- .:/app ports:

- '3000:3000'links:

- postgrespostgres:

image: postgres:9.4 ports:

- '5432'

A simple Rails app

In most cases, we need to do a bit of setup

before running tests.

You can run one-off commands against a service

with Docker Compose.

docker-compose run -e "RAILS_ENV=test" \ app rake db:setup

docker-compose run -e "RAILS_ENV=test" \ app rspec path/spec.rb

docker-compose up #-d if you wish

Usual Docker Compose development environment

Run rake task to set up db

Then run tests against Docker services

Docker delivers apredictable, reproducible

testing environment. Period.

Continuous Integration, Testing, and DockerFail fast and not in production!

Continuous integration and testing go hand in hand.

👯

Relying on CI/CD withoutadequate test coverage

is not a great idea.

Would you rather…

Find out your code is broken by seeing a failed run of your CI/CD system?

Find out your code is broken by seeing 500s on 500s on 500s in production?

✔

💩

What about running tests like this…

…inside a container?

A word of warning!

—Jérôme Petazzoni

“Docker-in-Docker is not 100% made of sparkles,

ponies, and unicorns.

37

Docker in Docker

CAUTION!

YMMVSee https://jpetazzo.github.io

- Mixing file systems == bad time- Build cache == bad time

Shared daemon – what we really want

Parallel Testing with DockerBreak it up.

Why do you skip tests?

45%

15%5%

35% Too slow to run suite 35%Annoying extra step

15%Don’t see the point

5%

Slows down deployment 45%

42

When developing, it’s easy to think of a container as

a small VM that runs a specific workload.

✔

But if we change our thinkingto treat containers as just processes, then we can do

some pretty cool stuff.

✔

A syntax similar to Docker Compose forservice definition…

47

web: build: image: my_app dockerfile_path: Dockerfile links: - redis - postgresredis: image: redispostgres: image: postgres

And use it to also define testing steps…

48

- type: parallel steps: - service: demo command: ruby check.rb - service: demo command: rake teaspoon suite=jasmine - service: demo command: rake test

- type: parallel steps:

- service: demo command: ruby check.rb

- service: demo command: rake teaspoon suite-jasmine

- service: demo command: rspec spec

Each step is run independently in a container

49

- service: demo command: ruby check.rb

- service: demo command: rake teaspoon suite-jasmine

- service: demo command: rspec spec

And each container may be located in a range of availability zones

50

Docker makes this possible byproviding the means to create

a predictable, reproducibletesting environment.

Resources#keepshipping

—Edsger Dijkstra

“Testing can be a very effective way to show the presence of bugs, but is

hopelessly inadequate for showing their absence.”

54

Resources

Highly recommended talks about development, testing, and lots of interesting stuff: https://speakerdeck.com/searls

Ruby gem for parallel tests: grosser/parallel_testsParallel Docker testing: Jet (from Codeship) CI/CD with Docker: http://pages.codeship.com/dockerRunning commands with Docker Compose: http://

docs.docker.com/composeThe perils of Docker-In-Docker: https://jpetazzo.github.io

This talk: slideshare.net/rheinwein

Testing does not guarantee that

our software works.

We test to know when our software is

definitely broken.

Thank you!Laura Frank@rhein_weinlaura@codeship.com