ios test-driven development
DESCRIPTION
Experiencing a new way of developingTRANSCRIPT
iOS Test-Driven Development
Experiencing a new way of working
I’m going to talk about…
• Some definitions
• Some concepts
• Unit Tests
• The TDD cycle
• Some principles
!
• TDD: Pros & Cons
• 3 kind of verifications
• External dependencies
• iOS Tests (practice part!)
• References
Some definitions
–Wikipedia
Test-driven development (TDD) is a software development process that relies on the
repetition of a very short development cycle. !
- !
Kent Beck, who is credited with having developed or ‘rediscovered' the technique, stated in 2003 that TDD encourages simple designs and inspires confidence.
–jayway.com
Test-driven development, or TDD for short, is a simple software development practice where unit tests, small focused test cases, drive the development forward.
!
- !
Test cases are written before any production code. !
Not all the tests are written up front, it’s rather that a small test is written, then a small piece of production code is
written that only allows that test to pass.
–Sunetos, Inc.
TDD is an approach to development that uses automated software tests to drive the design and development of an
application iteratively. !
- !
It pushes the developer to focus on one thing at a time, and usually one fairly small thing.
Some concepts
The idea behind test-driven development is that it makes you think about what the code you are writing
needs to do while you are designing it.
As a side effect, we get some safety against breaking things in the future.
OUR work, designing & coding
TDD motivates rapid addition of new functionality to your code, because you are free to make any change you want to your code and can rapidly
get feedback on whether the change introduced any problems…
TDD motivates rapid addition of new functionality to your code, because you are free to make any change you want to your code and can rapidly
get feedback on whether the change introduced any problems…
… or not!
A common mistake in any discipline of software engineering is to only ask about code and test the
“happy path” without specifying or discovering what happens in case of error.
A common mistake in any discipline of software engineering is to only ask about code and test the
“happy path” without specifying or discovering what happens in case of error.
That IS NOT GOOD.
A common mistake in any discipline of software engineering is to only ask about code and test the
“happy path” without specifying or discovering what happens in case of error.
That IS NOT GOOD.
And, guess what? TDD discourages that.
What is the cost of fixing a defect vs. preventing it? !
!
!
Learning and implementing TDD at the beginning is hard.
!
!
!
However, once you make it over the learning curve of TDD, your time-to-market will be the same, but with
many more benefits.
TDD changes design from a process of invention
to a process of discovery.
TDD changes design from a process of invention
to a process of discovery.
where the developer thinks hard about what a unit code should do and then
implements it
TDD changes design from a process of invention
to a process of discovery.
where the developer thinks hard about what a unit code should do and then
implements it
where the developer adds small increments or functionality and then extracts structure
from the working code
“Preventing bugs is not a TDD goal. It’s more like a side effect.”
Unit Tests
What is a Unit Test?
A unit test is a piece of code (usually a method) that invokes another piece of code and checks the correctness of some assumptions afterwards.
A unit test must test a single unit of code, which is usually a method or function within a class.
What is a Unit Test?
Unit testing will be performed against a System Under Test (SUT).
SUTTests
If assumptions turn out to be wrong, the test has failed. Otherwise, the test has passed.
Properties of a Unit Test
Any unit test should have the following properties:
❖ It should be automated and repeatable.
❖ It should be easy to implement.
❖ Once it’s written, it should remain for future use.
❖ Anyone should be able to run it.
❖ It should run quickly.
❖ Verify that a relatively small piece of code is doing what is intended to do !
❖ Narrow in scope !
❖ Easy to write and execute !
❖ Intended for the use of the programmer !
❖ Testers and users downstream should benefit from seeing less bugs !
❖ Done in total isolation
❖ Demonstrate that different pieces of the system work together !
❖ Can cover whole applications !
❖ Require more effort to put together !
❖ Intended for the use of non-‐programmers !
❖ Their output is the integrated system ready for System Testing !
❖ Done altogether
UNIT TESTS INTEGRATION TESTS
(taken from my previous talk about KIF Integration Tests)
Unit tests are usually composed by 3 parts:
1. Preparation // given
// when
// then
these initial conditions…
I execute this code…
I expect this to happen…
2. Execution
3. Verification
// givenCounter *counter = [[Counter alloc] init];
[counter increment];
// when
// thenXCTAssertTrue(counter.count == 4, @“3 incremented should give us 4”);
[counter setCount:3];
Unit tests are usually composed by 3 parts:
The TDD cycle
TDD cycle
Red-Green-Refactor approach
Write a failing test
Write a failing test
Write the simplest
code to pass the test
Write a failing test
Write the simplest
code to pass the test
Refactor! to make code
clean
Write a failing test
Write the simplest
code to pass the test
Refactor! to make code
clean
Write a failing test
Write the simplest
code to pass the test
Refactor! to make code
clean
Both testing and production code
KISS
To make sure we are adding something new
1.
Write a failing test
To make sure we are adding something new
- (void)testOneIncrementedShouldYieldTwo { // given Counter *counter = [[Counter alloc] init]; [counter setCount:1]; ! // when [counter increment]; ! // then XCTAssertEqual(counter.count, 2, nil); }
1.
Tests SUT
@implementation Counter !- (void)increment { !} !@end
Write a failing test
To make sure we are adding something new
1.
Tests SUT
@implementation Counter !- (void)increment { !} !@end
Write a failing test
To make sure we are adding something new
Leaving this method empty we will make the test fail.
1.
Tests SUT
Write the simplest
code to pass the test
KISS - Keep It Simple, Stupid
@implementation Counter !- (void)increment { !} !@end
2.
Tests SUT
Write the simplest
code to pass the test
KISS - Keep It Simple, Stupid
@implementation Counter !- (void)increment { !} !@end
_count = 2;
2.
Tests SUT
Write the simplest
code to pass the test
KISS - Keep It Simple, Stupid
@implementation Counter !- (void)increment { !} !@end With this simple line of code,
we are making the test pass.
_count = 2;
2.
Tests SUT
Now, you may complain “This implementation is really stupid…”
And, it is!!! …It has to be! !This is because it satisfies in the cleanest way all the tests
we have so far.
Now, you may complain “This implementation is really stupid…”
And, it is!!! …It has to be! !This is because it satisfies in the cleanest way all the tests
we have so far.
WAIT A SECOND…
Wouldn’t it be better if we made the method contain something like “_count ++;”
instead?!!
So that we’d be making sure it’ll pass some future tests we’d have to implement?
Come on, why not??? !
Explain yourself!
“We want to live in the present. Not in the future.” !
Because the present will guide us in ways we might not expect.
And that IS GOOD!
4 questions that will help you create the simplest thing that could possibly work
The situation:
• You had a failing test.
• You went to production code and made the test pass in the simplest way you thought possible.
• But… Was it really the simplest way?
4 questions that will help you create the simplest thing that could possibly work
The answer:
• We need to define what we consider being simple.
• For that, we’ll look at the code we’ve written and ask ourselves the following…
4 questions that will help you create the simplest thing that could possibly work
Can I implement the same solution in a way that is…
4 questions that will help you create the simplest thing that could possibly work
Can I implement the same solution in a way that is…
4 questions that will help you create the simplest thing that could possibly work
• …more hardcoded…
Can I implement the same solution in a way that is…
4 questions that will help you create the simplest thing that could possibly work
• …more hardcoded…
• …closer to the beginning of the method I wrote it in…
Can I implement the same solution in a way that is…
4 questions that will help you create the simplest thing that could possibly work
• …more hardcoded…
• …closer to the beginning of the method I wrote it in…
• …less indented (in as “less” scopes as possible, like if’s, loops, try-catch, etc)…
Can I implement the same solution in a way that is…
4 questions that will help you create the simplest thing that could possibly work
• …more hardcoded…
• …closer to the beginning of the method I wrote it in…
• …less indented (in as “less” scopes as possible, like if’s, loops, try-catch, etc)…
• …shorter (literally less characters to write) yet still readable…
Can I implement the same solution in a way that is…
4 questions that will help you create the simplest thing that could possibly work
• …more hardcoded…
• …closer to the beginning of the method I wrote it in…
• …less indented (in as “less” scopes as possible, like if’s, loops, try-catch, etc)…
• …shorter (literally less characters to write) yet still readable…
…AND STILL MAKE ALL THE TESTS PASS?
4 questions that will help you create the simplest thing that could possibly work
Example
@implementation Counter !- (void)increment { !} !@end
test 1 incr = 2
Tests
4 questions that will help you create the simplest thing that could possibly work
Example
@implementation Counter !- (void)increment { !} !@end
Tests
test 1 incr = 2
_count = 2;
4 questions that will help you create the simplest thing that could possibly work
Example
@implementation Counter !- (void)increment { !} !@end
test 1 incr = 2test 2 incr = 3
Tests
_count = 2;
test 1 incr = 2test 2 incr = 3
4 questions that will help you create the simplest thing that could possibly work
Example Tests
_count ++;
@implementation Counter !- (void)increment { !} !@end
test 1 incr = 2test 2 incr = 3
4 questions that will help you create the simplest thing that could possibly work
Example Tests
_count ++; if (_count == 1) { _count = 2; } else if (_count == 2) { _count = 3; }
@implementation Counter !- (void)increment { !} !@end
test 1 incr = 2test 2 incr = 3
4 questions that will help you create the simplest thing that could possibly work
Example Tests
_count ++; if (_count == 1) { _count = 2; } else if (_count == 2) { _count = 3; } Simpler than
@implementation Counter !- (void)increment { !} !@end
3.
Refactor! to make code
cleanBoth testing and production code.
3.
Refactor! to make code
cleanBoth testing and production code.
We have nothing to refactor so far because our code remains clean…
3.
Refactor! to make code
cleanBoth testing and production code.
We have nothing to refactor so far because our code remains clean…
So, we can skip this step.
3.
Don’t refactor when you have failing tests.
Always start refactoring from a clean state.
Make sure all your tests keep passing after refactoring.
Some principles
The pillars of good tests
Every test you write must be:
1. Trustworthy
2. Maintainable
3. Readable
Tips to have trustworthy tests
★ Decide when to remove or change tests.
★ Avoid test logic.
★ Test only one thing.
★ Make tests easy to run.
★ Assure code coverage.
Ya Ain’t Gonna Need It
Ya Ain’t Gonna Need It
The "YAGNI" principle
The "YAGNI" principle
If you write tests that describe what's needed of your app code, and you only write code that passes those tests,
you will never write any code you don’t need.
This encourages production code to be simple, and avoids wasting time writing code that won’t have any
effect later.
A test-driven app should have no unused code, and no (or very little) untested code.
The "YAGNI" principle
Remember: If you find yourself thinking during the refactoring stage that there are some changes you could make to have the code support more conditions, stop.
!
Why aren’t those conditions tested for in the test cases? Because those conditions don’t arise in the app.
!
So, don’t waste time adding the support, because… !
!
The "YAGNI" principle
Remember: If you find yourself thinking during the refactoring stage that there are some changes you could make to have the code support more conditions, stop.
!
Why aren’t those conditions tested for in the test cases? Because those conditions don’t arise in the app.
!
So, don’t waste time adding the support, because… !
!
YA AIN’T GONNA
NEED IT
The "YAGNI" principle
Remember: If you find yourself thinking during the refactoring stage that there are some changes you could make to have the code support more conditions, stop.
!
Why aren’t those conditions tested for in the test cases? Because those conditions don’t arise in the app.
!
So, don’t waste time adding the support, because… !
!
YA AIN’T GONNA
NEED IT
Should we test private methods?
Should we test private methods…?
To answer that, let's observe the following fact: !
You have already tested your private methods.
By following the TDD's red-green-refactor approach, you designed your objects’ public APIs to do the work those
objects need to do. !
With that work specified by the tests, you are free to organize the internal plumbing of your classes as you see fit.
Your private methods have already been tested because all you are doing is refactoring behavior that you already have
tests for.
You should never end up in a situation where a private method is untested or incompletely tested, because you
create them only when you see an opportunity to clean up the implementation of public methods.
This ensures that the private methods exist only to support the classes’ public behavior, and that they must be invoked
during testing because they are definitely being called from public methods.
TDD: Pros & Cons
✓ A 2005 study found that using TDD meant writing more tests and, in turn, developers who wrote more tests tended to be more productive.
✓ Programmers using pure TDD reported they only rarely felt the need to invoke a debugger.
✓ TDD offers more than just simple validation of correctness, but can also drive the design of a program.
✓ TDD gives the development team, and subsequent users, a greater level of confidence in the code.
TDD benefits
✓ Despite the fact that more code is required with TDD because of the testing code, the total code implementation time could be shorter.
✓ TDD can lead to more modularized, flexible and extensible code.
✓ TDD suggests a better modularization, easier reuse and testing of the developed software products.
✓ TDD raises the overall code quality in the projects, which saves time due to a better maintainability and an easier bug-fixing process.
TDD benefits
๏ Increase the amount of test code coverage.
๏ Increase the amount of test code coverage relative to the amount of code churn.
๏ Reduce the amount of bug reopening.
๏ Reduce the average bug-fixing time (the time from "bug opened" to "bug closed").
Some TDD goals
TDD does not perform sufficient testing in situations where full functional tests are required to determine success or failure due to extensive use of unit tests.
If the developer misinterprets the requirements specification for the module being developed, both the tests and the code will be wrong, as giving a false sense of correctness.
A high number of passing unit tests may bring a false sense of security, result in in fewer additional software testing activities, such as integration testing and compliance testing.
TDD shortcomings
Overtesting can consume time both to write the excessive tests, and later, to rewrite the tests when requirements change.
Hard learning curve. Very difficult at the beginning due to drastic changes which are involved in the process.
TDD shortcomings
Team progress and output measured with and without tests
source: The Art Of Unit Testing - Roy Osherove
Real case of a Pilot Project
That’s why it’s important to emphasize that, although unit testing can increase the amount of time it takes to implement a feature, the time balances out over the product’s release cycle because of increased quality and maintainability.
Is TDD worth it?
(taken from Jon Reid’s blog “Quality Coding”)
3 kind of verifications
System Under Test
SUT
messageSUT
message
return valueSUT
message
return valueSUT
- (id)calculateSomethingFromArg:(id)arg;
message
return value
Return Value Verification
SUT
- (id)calculateSomethingFromArg:(id)arg;
SUT
- (void)doSomething;
SUT
- (void)doSomething;
initial state
SUT
- (void)doSomething;
initial state
message
SUT
- (void)doSomething;
initial state
message
state accessor
result state
State Verification
SUT
- (void)doSomething;
initial state
message
state accessor
result state
State Verification
SUT
- (void)doSomething;
initial state
message
state accessor
result state
State-based testing (also called state verification) determines whether the exercised method worked correctly by examining the SUT after the method is exercised.
SUTDependent
object
SUTmessage Dependent
object
SUTmessage Dependent
object
SUTmessage Fake
object
Interaction Verification
SUTmessage Fake
object
(a.k.a. Behavior Verification)
Interaction Verification
SUTmessage Fake
object
(a.k.a. Behavior Verification)
Interaction testing is testing how an object sends input to or receives input from other objects—how that object interacts with other objects.
3 kind of verifications
Working with external dependencies
What is an external dependency?
Real ObjectSUT
CollaboratorClass
?request something
expect a value back
HOW?
We shouldn’t care
External dependency
An external dependency is an object in your system that your code under test interacts with, and over
which you have no control.
External dependencies
• Filesystems
• Managers
• Handlers
• Connections
• Formatters
• Etc
How are we going to deal with an external dependency
when we need to test our SUT?
We need to make sure our external dependency works as we expect, since
we are NOT testing it, but our SUT.
FAKE OBJECTS
A fake object is just an object that mimics the behavior of a real object in controlled ways.
!We define how we want the fake object to respond to certain
stimulations, such as method calls.
Fake Objects
Real ObjectSUT
CollaboratorClass
?request something
expect a value back
HOW?
We shouldn’t care
External dependency
A fake object is just an object that mimics the behavior of a real object in controlled ways.
!We define how we want the fake object to respond to certain
stimulations, such as method calls.
Fake Objects
Fake ObjectSUT
CollaboratorClass
request something
expect a value back
We tell it what to respond
In such a fast, controllable and
kind of hardcoded way.
Fake Objects
Fake ObjectSUT
CollaboratorClass
request something
expect a value back
We tell it what to respond…
Test Class … and then we assert the result.
(AGAINST WHOM…?)
Fake Objects
STUBSUT
Test Class
communication
assert against
SUT
Fake Objects
MOCKSUT
Test Class
communication
assert against
fake object
Fake Objects
If we assert against…
the fake object the object is a mock.FS
T
the system under test the object is just a stub.FS
T
Fake Objects
If we assert against…
the fake object the object is a mock.FS
T
the system under test the object is just a stub.FS
T
We can have as many stubs as we need F
F
F
Fake Objects
If we assert against…
the fake object the object is a mock.FS
T
We can have at most one mock per test
the system under test the object is just a stub.FS
T
We can have as many stubs as we need F
F
F
F
Fake Objects
If we assert against…
the fake object the object is a mock.FS
T
We can have at most one mock per test
the system under test the object is just a stub.FS
T
We can have as many stubs as we need F
F
F
F
We should never assert against two different things within the same test!
T
Fake Objects
However, we could end up having a configuration like this one…
FakeSUT
Test
Fake
Fake
Fake
Fake Objects
However, we could end up having a configuration like this one…
Where we have multiple stubs, but only one mock.
MockSUT
Test
Stub
Stub
Stub
Fake Objects
However, we could end up having a configuration like this one…
Where we have multiple stubs, but only one mock.
MockSUT
Test
Stub
Stub
Stub
Fake Objects
How do we build fake objects?
Manually
By using a framework
• OCMock
• OCMockito
• Etc
How do we make external dependencies interact with our SUT?
Dependency Injection
C’mon… It ain’t gonna hurt!
THE DEPENDOCTOR
✤ Dependency Injection
✤ Design for Testing
Both are important topics on TDD that won’t be explained in this talk because they are too much extensive that they’d require another complete talk to be minimally understood.
Time for iOS tests!
Objective-C Unit Testing Frameworks
❖ OCUnit!
❖ XCTest (Xcode 5 onwards)
Objective-C Unit Testing Frameworks
❖ OCUnit!
❖ XCTest (Xcode 5 onwards)
Integrates with OS X Server CI using Bots
Objective-C Unit Testing Frameworks
❖ OCUnit!
❖ XCTest (Xcode 5 onwards)
OCHamcrest
OCMockitoIntegrates with OS X Server
CI using Bots
UI tests
@property (strong, nonatomic) IBOutlet UIButton *button;
UI tests
@property (strong, nonatomic) IBOutlet UIButton *button;
- (IBAction)buttonPressed:(id)sender;
UI tests
@property (strong, nonatomic) IBOutlet UIButton *button;
- (IBAction)buttonPressed:(id)sender;
Test 1: Is outlet hooked up?
UI tests
@property (strong, nonatomic) IBOutlet UIButton *button;
- (IBAction)buttonPressed:(id)sender;
Test 1: Is outlet hooked up?
Test 2: Is outlet connected to action?
UI tests
@property (strong, nonatomic) IBOutlet UIButton *button;
- (IBAction)buttonPressed:(id)sender;
Test 1: Is outlet hooked up?
Test 2: Is outlet connected to action?
Test 3: Invoke action directly.
References
Test-Driven iOS Development
by Graham Lee
…for iOS TDD developers
It’s like the holy bible…
2012 Objective-C
The Art Of Unit Testing
by Roy Osherove
with Examples in .NET
2009 .NET
xUnit Test Patterns
by Gerard Meszaros
Refactoring Test Code
2007 Java
Working Effectively With Legacy Code
by Michael Feathers
2005 Java, C++, C
http://en.wikipedia.org/wiki/Test-driven_development
Wikipedia
http://en.wikipedia.org/wiki/Mock_object
couldn’t be missing…
Other websites and articleshttp://www.jayway.com/2010/01/15/test-driven-development-in-xcode/ !http://www.sunetos.com/items/2011/10/24/tdd-ios-part-1/ !https://www.youtube.com/watch?v=S5MvykD3yiE (iOS Unit Testing Like A Boss by Matt Darnal) !http://www.mockobjects.com/ !http://osherove.com/blog/2010/1/6/tdd-4-questions-that-will-help-you-create-the-simplest-thing.html !http://martinfowler.com/articles/mocksArentStubs.html
That’s all, folks! (for now…)
I hope you start liking TDD :)
Pablo Villar - April 2014 @ InakaLabs