under test

Under Test

TDD Techniques from the TrenchesMay 24, 2011Matt Thurston

Agenda

TDD

Legacy Code

TechniquesBenefits

Pitfalls

TDD

Legacy Code

TechniquesBenefits

Pitfalls

Rediscovery

Nutshell

Failing Test

Passing Test

Re-factor

Why get code ‘Under Test’?

Cost of Change

Change

Simple Code Change (“Hack”)versus

Tough Design Change (“Right Thing”)

Technical Debt

Motivator

TDD

Legacy Code

TechniquesBenefits

Pitfalls

What is Legacy Code?

• Inherited?• Old version?• Previous platform?• ‘C’?

Legacy Code = Code Without Tests

Confident Change

• Changing legacy code– What will break? – Will we find the break?

Under Test

• TDD Greenfields vs Legacy Code Retrofits

TDD

Legacy Code

TechniquesBenefits

Pitfalls

Defining ‘Unit Test’

• Our Unit is the C++ class• Unit Tests in C++

Issue #1

• TDD requires immediate feedback

What should CTRL + B do?

• Build and test as much as possible within a few seconds.

Deployment Fidelity

Development

Test Deployment

Production Deployment

Deployment Fidelity

• Deployment Fidelity typically dictates Test Fidelity

Test Fidelity

Unit Integration System

Test Fidelity

Unit(Seconds)

Integration(Minutes)

System(Hours)

Introduce New Fidelity?

• TDD requires immediate feedback• If necessary, introduce a new fidelity

Issue #2

• ‘Random’ test failures

Dependency Inversion

• E.g.– Foo uses files to store configuration information– Instead, inject FooConfiguration with• Virtual bool FooConfiguration::isEnabled() = 0;

Dependency Inversion

• Big Benefits:– Improves isolation– Simplifies reasoning about collaborations

Issue #3

• Difficult component

Difficult Components

• Lack of modularity• Lots of dependencies• Lack of coherent design• Full of bugs• Etc.

Integration Test First

• Works well for Difficult Components• Provides confidence when re-factoring for unit

testability

Issue #4

• Getting inherited legacy code under test

Characterization Tests

Failing Test

Capture Output

Passing Test

Characterization Tests

• Capture reality not expected behavior• The first test is the hardest

Failing a Unit Test

• If a unit test has never failed once, how do you know it works?

Mutation Testing

Mutation Testing

Passing Test

Mutate

Failing Test

Un-mutate

TDD

Legacy Code

TechniquesBenefits

Pitfalls

Semantic Memory

Input ‘A’ Output ‘B’

Tests are never wrong

• Tests encode semantics of the software– (even if the sematic is wrong)

Comment on comments

• “When the code and the comments disagree, both are probably wrong.” – Norm Schryer

Under Test vs. Coding Speed

Dreaming of Code

TDD

Legacy Code

TechniquesBenefits

Pitfalls

Continuous Testing

• Share tests• Don’t let test code rot– Integrate with build / CI tool

Mocks, Stubs, Fakes

• Take care with terminology overloading

Mocks, Stubs, Fakes

• Take care with terminology overloading• Mock: – Object used to mock collaborator behavior in a unit test

• Stub:– Satisfies linkage from a collaborator, provides no

implementation (e.g. asserts)• Fake:– Implements a collaborator’s functionality in a synthetic

fashion

Not Just the Tests

• Code coverage• Static analysis (e.g. Lint, -Wall)• Dynamic analysis (e.g. valgrind)

Quality?

• TDD / Unit Testing does not imply Quality

Questions? Comments?

TDD

Legacy Code

TechniquesBenefits

Pitfalls