who will guard the guards themselves? how to trust your automation and avoid deceit
TRANSCRIPT
Who will guard the
guards themselves?
How to trust your automation and avoid deceit
Bas Dijkstra
www.ontestautomation.com
@_basdijkstra
_I build test automation solutions
_I teach test automation, service virtualization
_I write www.ontestautomation.com
TechBeacon, StickyMinds, …
Continuous Testing
DEV ??? PROD
testing testing testing
testing
“Sed quis
custodiet ipsos
custodes?”
But who will guard the guards themselves?
--- Juvenal, Satire VI, lines 347–348 ---
Confidence“Our product is
able to perform
function X”
Control Trust
“I control the
execution of X”
“I trust our
product to be able
to perform X”
I want to be confident about the ability of our
product to perform function X
Automated tests are used to build trust in our
product performing function X correctly
I need to be able to trust my automated tests
around function X
“I’m not upset that
you lied to me,
I’m upset that from
now on I can’t
believe you”
--- Friedrich Nietzsche ---
False positives
Persistent > fix it or fire it
Intermittent > ‘flaky tests’
https://youtu.be/XnkWkrbzMh0
Richard Bradshaw – Selenium Conf Berlin
False negatives
False negative: exhibit A
100%
code coverage!
False negative: exhibit B
still
If you want to trust your tests,
start building tests you can trust on
Keep your tests fresh
Do code review on your test code
Test your tests periodically
Review your suite from time to time
Don’t be afraid to throw away tests
Tools to help you
keep your tests fresh
Mutation testing
http://pitest.org/
(Java)
Property-based testing
https://github.com/pholser/junit-quickcheck
(Java)
http://blog.jessitron.com/2013/04/property-based-testing-what-is-it.html
_Email: [email protected]
_Blog: www.ontestautomation.com
_LinkedIn: www.linkedin.com/in/basdijkstra
_Twitter: @_basdijkstra