daimi(c) henrik bærbak christensen1 test doubles: stubs, spies, mocks, etc
Post on 22-Dec-2015
217 views
TRANSCRIPT
DAIMI (c) Henrik Bærbak Christensen 2
Motivation
Thorough testing requires objects to be tested in isolation – to create a harness/environment where defects/complexity in other objects do not invalidate/complicate our testing.
We have discussed drivers and stubs for this purpose.– The stub notion is actually rather broad– A classic natural science approach:
• Define terminology much more precisely• Get a deeper insight into the subject matter as we analyze
and clarify terminology...
DAIMI (c) Henrik Bærbak Christensen 3
Stubs and Stubs?
Definition 1 (My book)
– Focus: Just get the TDD iteration 1 cycle to compile and get a red bar...
Definition 2 (Burnstein)– Stub: auxiliary code representing modules a UUT
calls.• “display a message that it has been called; return a value
from a table, ...”
DAIMI (c) Henrik Bærbak Christensen 4
Stubs and Stubs
Thus stubs is used in different contexts to mean different things– “minimally compilable” or “do something test related”
In practice there are a lot of different things you would like a stub to do.
Gerard Meszaros defines a clearerterminology by classifying the various uses of stubs...
DAIMI (c) Henrik Bærbak Christensen 5
xUnit Pattern: Test Double
“Superclass”: Test Double– SUT: System under test (=UUT)– DOC: Depended-on Component
When?– Slow tests– DOC is
• not available• not under test control• has side-effects
Compare
:UUT:driver :stub1. 2.
DAIMI (c) Henrik Bærbak Christensen 6
Terminology
indirect output– the output a UUT
generates, not visible by our driver, but passed as parameters, protocols used, etc., to the DOCs
indirect input– the input a Unit Under Test
receives, not by parameter passing, instance variables, etc., but from results computed by DOCs.
Indirect Input
Indirect Output
DAIMI (c) Henrik Bærbak Christensen 7
Test Double
Solution:– Replace DOC with a double
• like stunt doubles in movies...
– Requires:• that this is possible!!!
GoF’s 1st principle: Program to an interface...
- Parameterize object creation!
DAIMI (c) Henrik Bærbak Christensen 8
Double classification
Meszaros classify several types of doubles according to the specific testing perspective
DAIMI (c) Henrik Bærbak Christensen 10
Test Stub
Context– In many circumstances, the environment or context in which the
system under test (SUT) operates very much influences the behavior of the SUT. To get good enough control over the indirect inputs of the SUT, we may have to replace some of the
context with something we can control, a Test Stub.
Examples:– ?
DAIMI (c) Henrik Bærbak Christensen 11
Test Stub Examples
Typical examples are
– Stubbing sensors or hardware• In a meteorological system it is important to test wind
calculations over north when the wind direction changes from 359 degrees to 0 degrees.
– Stubbing random behaviour• A dice must be put under test control
DAIMI (c) Henrik Bærbak Christensen 12
Stub variations
Responder– used to inject valid indirect inputs: happy paths
Saboteur– used to inject invalid indirect inputs
Temporary Test Stub– a stand in for a not-yet-implemented DOC – the first
TDD production code implementation is always of this kind.
• this is what I called a stub in my book...
Entity Chain Snipping– replace a network of objects with a single one
DAIMI (c) Henrik Bærbak Christensen 13
Test Stub
Conclusion:– the primary purpose of the stub is
– to control the UUT’s input space
that is– we get testing control over the input space +
environment in order to specify test cases • = (input, environment, expected output).
– usually has methods/means for the test to specify the returned indirect inputs
DAIMI (c) Henrik Bærbak Christensen 15
Test Spy
Context:– In many circumstances, the environment or context in which the
SUT operates very much influences the behavior of the SUT. To get good enough visibility of the indirect outputs of the SUT, we may have to replace some of the context with something we can use to capture these outputs of the SUT.
DAIMI (c) Henrik Bærbak Christensen 16
Test Spy Examples
A Test Spy can– record the parameters passed to it
• verify indirect computed output equals expected
– the order in which DOC methods were called• verify the protocol between UUT and DOC
A Test Spy does not fail, it merely records interaction.
The Spy is inspected after the test execution in order to verify that indirect output was correct.
DAIMI (c) Henrik Bærbak Christensen 17
Implementation notes
Test Spy inspection variations: Retrieval interface:
– The spy must have additional methods to extract the stored indirect output
Self Shunt:– The test case class itself implements the DOC
interface and is thus feed the indirect output directly to be cached in local variables
Inner Test Double– use an inner anonymous class as self shunt
DAIMI (c) Henrik Bærbak Christensen 18
Examples
A classic example is an abstraction that communicates state changes via the Observer pattern– observer notification is an (important!) side-effect of
state-changing method calls, but it is not externally visible: it is indirect output
– register a SpyListener that counts the number of observer updates received.
DAIMI (c) Henrik Bærbak Christensen 20
Examples
Gerry: An artificial Backgammon player– for all valid moves given board and dice
• make move, compute value of board• if (value > bestvalue) { remember this move; }
But does it compute the proper moves? Is it really the best move that is taken?
movehook.considerMove(move); normally considerMove is the empty method.
DAIMI (c) Henrik Bærbak Christensen 22
Fake Object
Context:– The SUT often depend on other components or systems. The
interactions with these other components may be necessary but the side-effects of these interactions as implemented by the real depended-on component (DOC), may be unnecessary or even detrimental. A Fake Object is a much simpler and lighter weight implementation of the functionality provided by the DOC without the side effects we choose to do without.
DAIMI (c) Henrik Bærbak Christensen 23
Fake Object
Stub versus Fake Object?– Fake Object has “realistic” behaviour– Fake Object is not instrumented with the indirect
inputs, it defines them itself.– Less focus on testing aspects of the SUT, more focus
on making it work.
DAIMI (c) Henrik Bærbak Christensen 24
Examples
Fake Database– replace database with in-memory HashTables
In-Memory Database– semi-real database but not disk-based
Fake Web Service– hard-coded or table-driven web server
Fake Service Layer– fake the domain layer
DAIMI (c) Henrik Bærbak Christensen 25
Examples
The TestGame uses a LargeAreaTestImpl instance that is a fake object (fake domain layer) of an Area.
DAIMI (c) Henrik Bærbak Christensen 27
Mock Object
Context:– A test double that verifies
the indirect outputs
DAIMI (c) Henrik Bærbak Christensen 28
Mock Object Workings
Mock objects are somewhat more complex to define but are powerful to verify UUT behaviour with respect to the DOC.– Define Mock object with same interface as DOC– Configure mock with expectations
• values to return (like test stub)• the methods that must be called
– including sequence/protocol and call count
– expected parameters
– The mock will fail if these expectations are not met• fail fast!
– Thus test driver needs not verify anything!
DAIMI (c) Henrik Bærbak Christensen 29
Mock Test appearance
Thus test cases using mock objects look rather different than traditional JUnit tests.
Example:– TestGame needs to define 5 Loader doubles (Area-,
Actor-, etc.) to test the Game implementation in isolation.
• Exercise: Are they Stubs, Fake Objects, Spies, Mocks?
DAIMI (c) Henrik Bærbak Christensen 30
Example
Assume that we want to ensure that Game calls the ActorLoader in the proper way (verify indirect outputs):
– That load is invoked with valid argument only once– That getHero is invoked one or several times– That getHero is not invoked before load
– And we do not want to implement all the house keeping ourselves!
DAIMI (c) Henrik Bærbak Christensen 31
jMock
jMock will generate Mock Objects dynamically!
You simply ask it to – generate an object given any interface– tell it which method + parameters to expect– tell it what objects/values to return
and off you go!
DAIMI (c) Henrik Bærbak Christensen 33
Example
Expect method “load” to be executed once only, with a BufferedReader as parameter, and return the string “Actor loaded” when invoked.
Expect method “getHero” to be executed at least once, with no parameters, after load has been executed, and return the hero object reference.
DAIMI (c) Henrik Bærbak Christensen 34
Mock types
Mock objects may use strict or lenient policy for protocol checking:– strict: All methods must be invoked in exact same
sequence as when mock is “programmed” (expectations set)
– lenient (nice): Methods may be invoked in any sequence.
jMock defaults to lenient but the after method allows protocol conventions to be checked, strict policy.
DAIMI (c) Henrik Bærbak Christensen 35
Considerations
Mock objects must be programmed in advance– thus we must be able to predict UUT indirect output in
advance – a hard-core whitebox requirement...
Mock objects are responsible for failing– thus exceptions thrown must be able to pass out of
the UUT• may not be possible if it is embedded in a EJB container or
similar...
DAIMI (c) Henrik Bærbak Christensen 36
TDD with Mocking
Freeman et al. argue that Mock object TDD unit tests follow a certain template/pattern:
– create mock instance– set expectations– invoke UUT with mock– verify mock consistency
DAIMI (c) Henrik Bærbak Christensen 37
Mock Roles, Not Objects
I believe in designs based upon the principles of– Role and responsibility driven designs
• think roles with sets of responsibilities and protocols of collaboration
– Compositional designs• behaviour arise from highly focused objects collaborating
which leads to– shallow class hierarchies– strong focus on interfaces for expressing roles– factor out object creation
Freeman et al. comes to the same conclusions...
DAIMI (c) Henrik Bærbak Christensen 38
Mocks out there
Many applications use third-party API’s that would be nice to mock:
– Databases: A Mocked JDBC
– WebServers: A Mocked service API
DAIMI (c) Henrik Bærbak Christensen 39
Example: JDBC Mocking
public void testTransferOk() throws SQLException { prepareResultSet(); Bank bank = new Bank(); bank.connect(); bank.transfer(1, 2, 5000); bank.disconnect(); verifySQLStatementExecuted("select balance"); verifySQLStatementExecuted("update account"); verifySQLStatementParameter("update account", 0, 1, new Integer(-5000)); verifySQLStatementParameter("update account", 0, 2, new Integer(1)); verifySQLStatementParameter("update account", 1, 1, new Integer(5000)); verifySQLStatementParameter("update account", 1, 2, new Integer(2)); verifyCommitted(); verifyNotRolledBack(); verifyAllResultSetsClosed(); verifyAllStatementsClosed(); verifyConnectionClosed();}
MockRunner provides mock objects for a JDBC connection as wellas J2EE abstractions.
DAIMI (c) Henrik Bærbak Christensen 40
Summary
Testing units in isolation is important– unit and integration testing– test-driven development
Units have more inputs and outputs than visible from the parameter list and instances variables– especially true in object-oriented programming– indirect inputs: data from DOCs– indirect output: data to DOCs
DAIMI (c) Henrik Bærbak Christensen 41
Summary
Test Doubles allow you to get access to, inspect, and verify indirect input and output
Stub: focus on indirect input Spy: focus on indirect output (record/verify) Mock: focus on indirect output (fail fast)
– frameworks to generate doubles dynamically
Fake object: light-weight semi-realistic behaviour