introduction to automated testing

Introduction to Automated Testing

by Lars Thorup, ZeaLakeMay 26th 2011

Who is Lars Thorup?

• Software developer

• Coach: Teaching TDD and automated testing

• Advisor: Assesses software projects and companies

• Founder and CEO of BestBrains and ZeaLake

Why are we here today?

• How do we do automated testing?– Write test programs– Run the tests automatically

• Why should we write tests?– Enjoy more efficient and predictable course of development– Find and fix bugs fast– Prevent bugs from reappearing– Improve the design of our software

Different kinds of automated tests

• Unit tests– Tests individual pieces of code and the interaction between code blocks

• System tests– Tests the entire system against the requirements

• Performance tests– Tests non functional requirements

Unit tests or system tests?

• Unit tests are efficient– Fast to run (a few seconds)– Robust and predictable– Easy to write– Is written together with the code it is testing

• System tests are thorough– Tests all layers together– Most efficient way to create a set of tests for existing code

Can we automate performance tests?

• Performance tests are brittle– Tip: create performance trend curves instead

So, how do we actually do this?

• IsNumeric– C#– Ruby

• TransferFunds– C++

How do we run the tests automatically?

• From our programming environment (IDE)– Command line: make test– Right click | Run Tests

• On every commit– Setup a build server

• Hudson / Jenkins• TeamCity

– Let the build server run all tests– Get build notifications– Keep the build green

• Fixing a broken build has priority over any other development task

How can tests help improve our design?

• The software design needs to evolve over time• A refactoring modifies the design without changing behavior• Tests ensure that behavior is not accidentally changed

• Without tests, refactoring is scary– and with no refactoring, the design decays over time

• With tests, we have the courage to refactor– so we continually keep our design healthy

What is good design?

• One element of good design is loose dependencies– Use interfaces (for static languages)– Inject dependencies

• Avoid this

• Do this instead

Are we wasting valuable developer time writing tests?

• No• The time spent writing tests is not taken from the time spent

coding– It is taken from the time otherwise spent on manual testing and debugging

• The cost of a bug keeps increasing until we fix it

• Find bugs fast– Avoid losing customer confidence– Free QA to do exploratory testing

so they find the hard-to-find bugs– Spend less time trying to figure out

what is causing the bug and how to fix it

• Avoid spending time testing again

How do we get started?

• When we have a lot of existing code without tests– Create a set of system tests to get a safety net

• When we are writing new code– Write unit tests in conjunction with the new code

• Set up a standard test environment for our specific application– Test data

• Automate the creation of standard testdata in a local database– External dependencies

• Write stubs to use in the tests

What does a real-world project look like?

• wizerize.com– Web application: C# and JavaScript– 3 years of production– 2-4 developers

• 40% test code, 60% production code (in lines of code)• 71% code coverage of unit tests• 614 unit tests – run in 1 minute• 54 system tests – run in 20 minutes• No functional errors seen by end users in production (yet)

Where can I read more?

• http://googletesting.blogspot.com/• http://testdrivendeveloper.com/• http://codesheriff.blogspot.com/

Which tools do we use?

Environment ToolC# NUnitJavaScript qUnitHTML-based UI WatiN, SeleniumC++ CppUnit, googletestPython unittestRuby Test::UnitC check, cunitJava JUnit... ...