![Page 1: Factories, mocks and spies: a tester's little helpers](https://reader038.vdocuments.us/reader038/viewer/2022103004/55cfdbccbb61ebf9018b460a/html5/thumbnails/1.jpg)
Factories, mocks, spies…
…and other tester’s little helpers
Carles Barrobés twitter: @technomilk github: @txels
![Page 2: Factories, mocks and spies: a tester's little helpers](https://reader038.vdocuments.us/reader038/viewer/2022103004/55cfdbccbb61ebf9018b460a/html5/thumbnails/2.jpg)
Testing is a very broad topic
!
…with its own special lingo
![Page 3: Factories, mocks and spies: a tester's little helpers](https://reader038.vdocuments.us/reader038/viewer/2022103004/55cfdbccbb61ebf9018b460a/html5/thumbnails/3.jpg)
blackbox whitebox regression unit-test
integration-test service-test pyramid
icecream-cone factory assertion spy SuT
…
![Page 4: Factories, mocks and spies: a tester's little helpers](https://reader038.vdocuments.us/reader038/viewer/2022103004/55cfdbccbb61ebf9018b460a/html5/thumbnails/4.jpg)
Let’s start with a question… !
Why do we write tests?
![Page 5: Factories, mocks and spies: a tester's little helpers](https://reader038.vdocuments.us/reader038/viewer/2022103004/55cfdbccbb61ebf9018b460a/html5/thumbnails/5.jpg)
We write tests to save money We tell the computer how to do [tedious] testing for us, faster and cheaper
Writing tests == automation
![Page 6: Factories, mocks and spies: a tester's little helpers](https://reader038.vdocuments.us/reader038/viewer/2022103004/55cfdbccbb61ebf9018b460a/html5/thumbnails/6.jpg)
SuT: System-under-Test*Your “system” as a black box:
I am system !
(with a spec, if you’re lucky)
in
out
in: data, stimuli out: data, observable behaviour
![Page 7: Factories, mocks and spies: a tester's little helpers](https://reader038.vdocuments.us/reader038/viewer/2022103004/55cfdbccbb61ebf9018b460a/html5/thumbnails/7.jpg)
SuT: System-under-TestYour “system” as a white/gray box:
I am system !
(and you can see what’s inside me)
in
out
…BTW I rely on a bunch of external stuff
![Page 8: Factories, mocks and spies: a tester's little helpers](https://reader038.vdocuments.us/reader038/viewer/2022103004/55cfdbccbb61ebf9018b460a/html5/thumbnails/8.jpg)
SuT: System-under-TestYour “system” as a white/gray box:
zin
out
…BTW I rely on a bunch of external stuff
Explicit/Injected dependencies Implicit/Hardcoded
dependencies
![Page 9: Factories, mocks and spies: a tester's little helpers](https://reader038.vdocuments.us/reader038/viewer/2022103004/55cfdbccbb61ebf9018b460a/html5/thumbnails/9.jpg)
Anatomy of [manual] testingTake your code up to the point you want to test Run the specific feature you are testing Verify that it worked
![Page 10: Factories, mocks and spies: a tester's little helpers](https://reader038.vdocuments.us/reader038/viewer/2022103004/55cfdbccbb61ebf9018b460a/html5/thumbnails/10.jpg)
Anatomy of a test casedef test_something_works(): !
prepare <blank line>
exercise <blank line>
verify
aka “Arrange, Act, Assert”
![Page 11: Factories, mocks and spies: a tester's little helpers](https://reader038.vdocuments.us/reader038/viewer/2022103004/55cfdbccbb61ebf9018b460a/html5/thumbnails/11.jpg)
Anatomy of a test casedef test_something_works(): !
prepare <blank line>
exercise <blank line>
verify
Get your code to a known state:
Generate test data Navigate
Isolate and monitor: Set up mocks Set up spies
![Page 12: Factories, mocks and spies: a tester's little helpers](https://reader038.vdocuments.us/reader038/viewer/2022103004/55cfdbccbb61ebf9018b460a/html5/thumbnails/12.jpg)
Anatomy of a test casedef test_something_works(): !
prepare <blank line>
exercise <blank line>
verify
Call your code: result = something(data)
![Page 13: Factories, mocks and spies: a tester's little helpers](https://reader038.vdocuments.us/reader038/viewer/2022103004/55cfdbccbb61ebf9018b460a/html5/thumbnails/13.jpg)
Anatomy of a test casedef test_something_works(): !
prepare <blank line>
exercise <blank line>
verify
Validate results: Assertions on results
Check observed behaviour:
Check reports from your mocks and spies
![Page 14: Factories, mocks and spies: a tester's little helpers](https://reader038.vdocuments.us/reader038/viewer/2022103004/55cfdbccbb61ebf9018b460a/html5/thumbnails/14.jpg)
Time for another question… !
Which of those is the hardest?
![Page 15: Factories, mocks and spies: a tester's little helpers](https://reader038.vdocuments.us/reader038/viewer/2022103004/55cfdbccbb61ebf9018b460a/html5/thumbnails/15.jpg)
Time for the actual talk… !
Let’s look at tools that can help us with the hard
bits
![Page 16: Factories, mocks and spies: a tester's little helpers](https://reader038.vdocuments.us/reader038/viewer/2022103004/55cfdbccbb61ebf9018b460a/html5/thumbnails/16.jpg)
Tools for test setup(I find the preparation phase to be the hardest bit)
[Complex] test data: factories Test objects with behaviour: mocks Instrument internals: spies
![Page 17: Factories, mocks and spies: a tester's little helpers](https://reader038.vdocuments.us/reader038/viewer/2022103004/55cfdbccbb61ebf9018b460a/html5/thumbnails/17.jpg)
FactoriesGoal: make it easy to generate complex test data structures !
Tool of choice: factory boy*
* I’ve tried others, but I prefer this one
![Page 18: Factories, mocks and spies: a tester's little helpers](https://reader038.vdocuments.us/reader038/viewer/2022103004/55cfdbccbb61ebf9018b460a/html5/thumbnails/18.jpg)
Factories: use casesCreate test data with simple statements
Let the factory fill [irrelevant] details
Black/white box testing Explicit dependencies
![Page 19: Factories, mocks and spies: a tester's little helpers](https://reader038.vdocuments.us/reader038/viewer/2022103004/55cfdbccbb61ebf9018b460a/html5/thumbnails/19.jpg)
Factories: simplicityExample: we need a django model instance for our test.
It has lots of mandatory fields… ..but in this test we only care about “title”
![Page 20: Factories, mocks and spies: a tester's little helpers](https://reader038.vdocuments.us/reader038/viewer/2022103004/55cfdbccbb61ebf9018b460a/html5/thumbnails/20.jpg)
Factories: simplicityNot this: publisher = Publisher.objects.create(name=‘Old Books’)
book = Book.objects.create(title=‘Tirant Lo Blanc’,
author=‘Joanot Martorell’,
date=1490,
publisher=publisher)
But this: book = BookFactory(title=‘Tirant Lo Blanc’)
![Page 21: Factories, mocks and spies: a tester's little helpers](https://reader038.vdocuments.us/reader038/viewer/2022103004/55cfdbccbb61ebf9018b460a/html5/thumbnails/21.jpg)
Factory Boy in actionimport factory
from books.models import Book, Publisher
!class PublisherFactory(factory.DjangoModelFactory): FACTORY_FOR = Publisher
name = ‘Test Publisher’
city = ‘Barcelona’
!class BookFactory(factory.DjangoModelFactory): FACTORY_FOR = Book
title = ‘Test Book’
author = ‘Some Random Bloke’
year = 2015
publisher = factory.SubFactory(PublisherFactory)
![Page 22: Factories, mocks and spies: a tester's little helpers](https://reader038.vdocuments.us/reader038/viewer/2022103004/55cfdbccbb61ebf9018b460a/html5/thumbnails/22.jpg)
Maintainability FTW!When you maintain large tests suites, you want to maximise reuse [& DRYness]
Defaults and rules for building your objects live in a central place - easy to adapt
E.g. adding a mandatory field is no longer a pain
Not tied to your test framework
![Page 23: Factories, mocks and spies: a tester's little helpers](https://reader038.vdocuments.us/reader038/viewer/2022103004/55cfdbccbb61ebf9018b460a/html5/thumbnails/23.jpg)
Factory Boy: nicetiesUse a sequence for unique values: name = factory.Sequence( lambda num: 'Name {}’.format(num) ) !
Lazy attributes to populate “late”: slug = factory.LazyAttribute(
lambda obj: slugify(obj.name)
)
![Page 24: Factories, mocks and spies: a tester's little helpers](https://reader038.vdocuments.us/reader038/viewer/2022103004/55cfdbccbb61ebf9018b460a/html5/thumbnails/24.jpg)
Factory Boy: nicetiesFuzzy (randomised) values title = factory.fuzzy.FuzzyText() # u'phPEZzNqfkXv'
gender = factory.fuzzy.FuzzyChoice(('m', 'f')) # 'm'
age = factory.fuzzy.FuzzyChoice(18, 45) # 27 ... !
Coming soon: Faker support (realistic values) name = factory.Faker('name') # u'Isla Erdman'
email = factory.Faker('email') # u'[email protected]'
![Page 25: Factories, mocks and spies: a tester's little helpers](https://reader038.vdocuments.us/reader038/viewer/2022103004/55cfdbccbb61ebf9018b460a/html5/thumbnails/25.jpg)
SpiesGoal: check if something happened inside your code !
Tool of choice: kgb** I don’t know others in Python, used Jasmine (JS)
![Page 26: Factories, mocks and spies: a tester's little helpers](https://reader038.vdocuments.us/reader038/viewer/2022103004/55cfdbccbb61ebf9018b460a/html5/thumbnails/26.jpg)
Spies: use casesWhen it’s hard to have externally observable behaviour
It’s a bit like adding monitoring to your tests
“Blackbox” testing (with some inside knowledge) Implicit dependencies
![Page 27: Factories, mocks and spies: a tester's little helpers](https://reader038.vdocuments.us/reader038/viewer/2022103004/55cfdbccbb61ebf9018b460a/html5/thumbnails/27.jpg)
Spies: how toYou know a little what your system does under the hood You “spy” on a method that should be called (the spy is a wrapper that “calls through”)
Your spy reports on how that method was called
![Page 28: Factories, mocks and spies: a tester's little helpers](https://reader038.vdocuments.us/reader038/viewer/2022103004/55cfdbccbb61ebf9018b460a/html5/thumbnails/28.jpg)
KGB in actionfrom unittest import TestCase
from kgb import spy_on !def add_three(number): return number + 3 !def do_stuff(number): return add_three(number + 1) !class SpyOnTest(TestCase): def test_spy_on_add_three(self):
with spy_on(add_three) as spy: result = do_stuff(15)
self.assertEqual(spy.last_call.args, (16,)) self.assertTrue(spy.called_with(16))
![Page 29: Factories, mocks and spies: a tester's little helpers](https://reader038.vdocuments.us/reader038/viewer/2022103004/55cfdbccbb61ebf9018b460a/html5/thumbnails/29.jpg)
KGB extrasYou can replace the spied on method and make it do nothing or something else
!
with spy_on(SomeClass.add_stuff, call_fake=add_two):
![Page 30: Factories, mocks and spies: a tester's little helpers](https://reader038.vdocuments.us/reader038/viewer/2022103004/55cfdbccbb61ebf9018b460a/html5/thumbnails/30.jpg)
MocksGoal: make it easy to simulate behaviour of a dependency !
Tool of choice: mock*
* There are others, I haven’t tried them
![Page 31: Factories, mocks and spies: a tester's little helpers](https://reader038.vdocuments.us/reader038/viewer/2022103004/55cfdbccbb61ebf9018b460a/html5/thumbnails/31.jpg)
Mocks: use casesYour SuT has explicit callback dependencies (objects it calls)
You want to feed valid objects and inspect what your system did to them
Simulate hard to reproduce conditions (e.g. exceptions)
![Page 32: Factories, mocks and spies: a tester's little helpers](https://reader038.vdocuments.us/reader038/viewer/2022103004/55cfdbccbb61ebf9018b460a/html5/thumbnails/32.jpg)
Mocks vs FactoriesFactories generate “real production objects” Mocks generate fake objects (that you can throw anything at)
![Page 33: Factories, mocks and spies: a tester's little helpers](https://reader038.vdocuments.us/reader038/viewer/2022103004/55cfdbccbb61ebf9018b460a/html5/thumbnails/33.jpg)
mock in action>>> from mock import Mock
>>> user = Mock(username='Fred')
>>> user.username
'Fred'
>>> user.save(force=True)
<Mock name='mock.save()' id='4492633872'>
>>> args, kwargs = user.save.call_args_list[0]
>>> kwargs
{'force': True}
![Page 34: Factories, mocks and spies: a tester's little helpers](https://reader038.vdocuments.us/reader038/viewer/2022103004/55cfdbccbb61ebf9018b460a/html5/thumbnails/34.jpg)
mocking calls>>> user.save.return_value = True
>>> user.save()
True
>>> user.save.side_effect = Exception('Boom')
>>> user.save()
-------------------------------------------------
Exception Traceback (most recent call last)
...
!
Exception: Boom
![Page 35: Factories, mocks and spies: a tester's little helpers](https://reader038.vdocuments.us/reader038/viewer/2022103004/55cfdbccbb61ebf9018b460a/html5/thumbnails/35.jpg)
mock extras: “patch”Patch existing code and replace it with a mock for the duration of a test Similar use cases to spies [but without “call through”]
![Page 36: Factories, mocks and spies: a tester's little helpers](https://reader038.vdocuments.us/reader038/viewer/2022103004/55cfdbccbb61ebf9018b460a/html5/thumbnails/36.jpg)
Mock: “patch” use casesYour SuT has hardcoded dependencies but you want to test it in isolation You want to accelerate your tests [by bypassing expensive calls]
![Page 37: Factories, mocks and spies: a tester's little helpers](https://reader038.vdocuments.us/reader038/viewer/2022103004/55cfdbccbb61ebf9018b460a/html5/thumbnails/37.jpg)
patch in actionfrom mock import patch
!
class MockPatchTest(TestCase):
@patch('test_sample.add_stuff')
def test_do_stuff_calls_add(self, add_stuff):
add_stuff.return_value = ‘whatever'
!
result = do_stuff(123)
!
add_stuff.assert_called_once_with(123)
self.assertEqual(result, 'whatever')
![Page 38: Factories, mocks and spies: a tester's little helpers](https://reader038.vdocuments.us/reader038/viewer/2022103004/55cfdbccbb61ebf9018b460a/html5/thumbnails/38.jpg)
Tools for test validationBuilt-in assert_ functions from your test tool (nose, unittest)
assert statement (if you use py.test it will give useful reporting) Matchers (hamcrest)
![Page 39: Factories, mocks and spies: a tester's little helpers](https://reader038.vdocuments.us/reader038/viewer/2022103004/55cfdbccbb61ebf9018b460a/html5/thumbnails/39.jpg)
MatchersGoal: reusable conditions for assertions !
Tool of choice: hamcrest
![Page 40: Factories, mocks and spies: a tester's little helpers](https://reader038.vdocuments.us/reader038/viewer/2022103004/55cfdbccbb61ebf9018b460a/html5/thumbnails/40.jpg)
Matchers: use casesYou want to check complex or custom conditions in a DRY way
Matchers can be composed - no need for “combinatory” assertions or assertTrue(<complex expression>)
![Page 41: Factories, mocks and spies: a tester's little helpers](https://reader038.vdocuments.us/reader038/viewer/2022103004/55cfdbccbb61ebf9018b460a/html5/thumbnails/41.jpg)
hamcrest highlightsA single assertion: assert_that Many matchers out of the box (plus you can write your own)
Useful reporting on mismatches (no more “False is not True” errors)
Composite matchers: all_of, any_of, not_
![Page 42: Factories, mocks and spies: a tester's little helpers](https://reader038.vdocuments.us/reader038/viewer/2022103004/55cfdbccbb61ebf9018b460a/html5/thumbnails/42.jpg)
hamcrest in actiondef test_any_of(self):
result = random.choice(range(6))
assert_that(result, any_of(1, 2, 3, 4, 5))
!
!
!
AssertionError:
Expected: (<1> or <2> or <3> or <4> or <5>)
but: was <0>
![Page 43: Factories, mocks and spies: a tester's little helpers](https://reader038.vdocuments.us/reader038/viewer/2022103004/55cfdbccbb61ebf9018b460a/html5/thumbnails/43.jpg)
hamcrest in actiondef test_complex_matcher(self):
user = UserFactory()
assert_that(
user.email,
all_of(
not_none(),
string_contains_in_order('@', '.'),
not_(contains_string('u'))
)
)
!AssertionError:
Expected: (not None and a string containing '@', '.' in order and not a string containing 'u')
but: not a string containing 'u' was '[email protected]'
![Page 44: Factories, mocks and spies: a tester's little helpers](https://reader038.vdocuments.us/reader038/viewer/2022103004/55cfdbccbb61ebf9018b460a/html5/thumbnails/44.jpg)
Custom matchersYou can write your own matchers The syntax is a bit verbose, so I wrote matchmaker to make it easier
![Page 45: Factories, mocks and spies: a tester's little helpers](https://reader038.vdocuments.us/reader038/viewer/2022103004/55cfdbccbb61ebf9018b460a/html5/thumbnails/45.jpg)
Custom matchers…from hamcrest.core.base_matcher import BaseMatcher
!
class IsEven(BaseMatcher):
def _matches(self, item):
return item % 2 == 0
!
def describe_to(self, description):
description.append_text('An even number')
!
def is_even():
return IsEven()
![Page 46: Factories, mocks and spies: a tester's little helpers](https://reader038.vdocuments.us/reader038/viewer/2022103004/55cfdbccbb61ebf9018b460a/html5/thumbnails/46.jpg)
…using matchmakerfrom matchmaker import matcher
!
@matcher
def is_even(item):
"An even number"
return item % 2 == 0
![Page 47: Factories, mocks and spies: a tester's little helpers](https://reader038.vdocuments.us/reader038/viewer/2022103004/55cfdbccbb61ebf9018b460a/html5/thumbnails/47.jpg)
Custom matchers in usedef test_custom_matcher(self):
user = UserFactory()
assert_that(user.age, is_even())
!
AssertionError:
Expected: An even number
but: was <19>
![Page 48: Factories, mocks and spies: a tester's little helpers](https://reader038.vdocuments.us/reader038/viewer/2022103004/55cfdbccbb61ebf9018b460a/html5/thumbnails/48.jpg)
More custom matchers@matcher
def ends_like(item, data, length):
"String whose last {1} chars match those for '{0}'"
return item.endswith(data[-length:])
!def test_custom_matcher(self):
user1, user2 = UserFactory(), UserFactory()
assert_that(
user.email,
ends_like(user2.email, 4),
)
!AssertionError:
Expected: String whose last 4 chars match those for '[email protected]'
but: was '[email protected]'
![Page 49: Factories, mocks and spies: a tester's little helpers](https://reader038.vdocuments.us/reader038/viewer/2022103004/55cfdbccbb61ebf9018b460a/html5/thumbnails/49.jpg)
Thanks!Any questions?
Carles Barrobés twitter: @technomilk github: @txels