![Page 1: Test automation and Agile software development](https://reader031.vdocuments.us/reader031/viewer/2022030318/5a649fc27f8b9a2c568b65fb/html5/thumbnails/1.jpg)
Test automation and Agile
software development
Bas Dijkstra
www.ontestautomation.com
@_basdijkstra
![Page 2: Test automation and Agile software development](https://reader031.vdocuments.us/reader031/viewer/2022030318/5a649fc27f8b9a2c568b65fb/html5/thumbnails/2.jpg)
Who are you?
![Page 3: Test automation and Agile software development](https://reader031.vdocuments.us/reader031/viewer/2022030318/5a649fc27f8b9a2c568b65fb/html5/thumbnails/3.jpg)
Who am I?
I help teams and organizations to implement automation
Blogs (ontestautomation.com, TechBeacon, StickyMinds)
Trainer (test automation, service virtualization)
![Page 4: Test automation and Agile software development](https://reader031.vdocuments.us/reader031/viewer/2022030318/5a649fc27f8b9a2c568b65fb/html5/thumbnails/4.jpg)
Test automation
experience?
![Page 5: Test automation and Agile software development](https://reader031.vdocuments.us/reader031/viewer/2022030318/5a649fc27f8b9a2c568b65fb/html5/thumbnails/5.jpg)
Test automation
experiences?
![Page 6: Test automation and Agile software development](https://reader031.vdocuments.us/reader031/viewer/2022030318/5a649fc27f8b9a2c568b65fb/html5/thumbnails/6.jpg)
What whould you like
to have learned after
tonight?
![Page 7: Test automation and Agile software development](https://reader031.vdocuments.us/reader031/viewer/2022030318/5a649fc27f8b9a2c568b65fb/html5/thumbnails/7.jpg)
What are we going to
do?
Role of and realism about test automation
Principles for automation
Demos, examples, case studies, requests
Test automation strategy
![Page 8: Test automation and Agile software development](https://reader031.vdocuments.us/reader031/viewer/2022030318/5a649fc27f8b9a2c568b65fb/html5/thumbnails/8.jpg)
How?
Why?
What?
![Page 9: Test automation and Agile software development](https://reader031.vdocuments.us/reader031/viewer/2022030318/5a649fc27f8b9a2c568b65fb/html5/thumbnails/9.jpg)
Why test automation?
![Page 10: Test automation and Agile software development](https://reader031.vdocuments.us/reader031/viewer/2022030318/5a649fc27f8b9a2c568b65fb/html5/thumbnails/10.jpg)
Why test automation?
Does not replace testers and testing
Does not save you money (at least not from the start)
Shorten feedback loop
Free time for other activities
Support testing activities
![Page 11: Test automation and Agile software development](https://reader031.vdocuments.us/reader031/viewer/2022030318/5a649fc27f8b9a2c568b65fb/html5/thumbnails/11.jpg)
Can you do without
test automation?
![Page 12: Test automation and Agile software development](https://reader031.vdocuments.us/reader031/viewer/2022030318/5a649fc27f8b9a2c568b65fb/html5/thumbnails/12.jpg)
What kind of tests
are you going to
automate?
![Page 13: Test automation and Agile software development](https://reader031.vdocuments.us/reader031/viewer/2022030318/5a649fc27f8b9a2c568b65fb/html5/thumbnails/13.jpg)
Highly automatable
Verifications (does A equal B?)
Correctness of algorithms and calculations
Deterministic tests
Machinal activities
![Page 14: Test automation and Agile software development](https://reader031.vdocuments.us/reader031/viewer/2022030318/5a649fc27f8b9a2c568b65fb/html5/thumbnails/14.jpg)
Not automatable
Validations (is this the behavior we want?)
User experience
Tests that have an open answer
Human activities: experimentation, evaluation, …
![Page 15: Test automation and Agile software development](https://reader031.vdocuments.us/reader031/viewer/2022030318/5a649fc27f8b9a2c568b65fb/html5/thumbnails/15.jpg)
There is no “manual testing”; there is
testing
There are no “manual testers”; there
are testers
Checking —an element of testing, a
tactic of testing— can be automated
http://www.developsense.com/blog/2017/11/the-end-of-manual-testing/
--- Michael Bolton ---
![Page 16: Test automation and Agile software development](https://reader031.vdocuments.us/reader031/viewer/2022030318/5a649fc27f8b9a2c568b65fb/html5/thumbnails/16.jpg)
“You should automate
100% of the tests
that should be
automated”
--- Alan Page ---
![Page 17: Test automation and Agile software development](https://reader031.vdocuments.us/reader031/viewer/2022030318/5a649fc27f8b9a2c568b65fb/html5/thumbnails/17.jpg)
“Don’t become the
world’s best
automator of useless
checks”
--- Bas Dijkstra ---
![Page 18: Test automation and Agile software development](https://reader031.vdocuments.us/reader031/viewer/2022030318/5a649fc27f8b9a2c568b65fb/html5/thumbnails/18.jpg)
How do you automate
your tests?
![Page 19: Test automation and Agile software development](https://reader031.vdocuments.us/reader031/viewer/2022030318/5a649fc27f8b9a2c568b65fb/html5/thumbnails/19.jpg)
Scope
Unit tests
Integration tests
End to end tests
Nonfunctionals (performance, security, …)
![Page 20: Test automation and Agile software development](https://reader031.vdocuments.us/reader031/viewer/2022030318/5a649fc27f8b9a2c568b65fb/html5/thumbnails/20.jpg)
Test automation pyramid
Unit
Integratie
E2E
_ Customer journeys
_ (Often) user interface-driven
_ ‘Ultimate integration test’
_ Layer / component integration
_ Data / business logic exposure
_ (Often) API-driven
_ ‘The forgotten layer’
_ Component level
_ Code driven
_ Mocks and stubs
![Page 21: Test automation and Agile software development](https://reader031.vdocuments.us/reader031/viewer/2022030318/5a649fc27f8b9a2c568b65fb/html5/thumbnails/21.jpg)
Tools and tool selection
![Page 22: Test automation and Agile software development](https://reader031.vdocuments.us/reader031/viewer/2022030318/5a649fc27f8b9a2c568b65fb/html5/thumbnails/22.jpg)
In the end, the tool
is not important
![Page 23: Test automation and Agile software development](https://reader031.vdocuments.us/reader031/viewer/2022030318/5a649fc27f8b9a2c568b65fb/html5/thumbnails/23.jpg)
How does automation
fit into Agile?
Testing often > short feedback loop required
It all starts at the basis (unit tests)
Too many E2E tests too soon > lots of maintenance
Avoid the user interface whenever possible
![Page 24: Test automation and Agile software development](https://reader031.vdocuments.us/reader031/viewer/2022030318/5a649fc27f8b9a2c568b65fb/html5/thumbnails/24.jpg)
Test automation
in the DoD
Do all features delivered need to be tested?
Do they all require a full set of automated tests?
100% code coverage?
Do you need to include automation in the DoD in the
first place?
![Page 25: Test automation and Agile software development](https://reader031.vdocuments.us/reader031/viewer/2022030318/5a649fc27f8b9a2c568b65fb/html5/thumbnails/25.jpg)
https://leanpub.com/testingindevops
![Page 26: Test automation and Agile software development](https://reader031.vdocuments.us/reader031/viewer/2022030318/5a649fc27f8b9a2c568b65fb/html5/thumbnails/26.jpg)
Case
Online banking application
Reasonably mature
Testing everything every sprint takes too long
Proposal: use test automation
![Page 27: Test automation and Agile software development](https://reader031.vdocuments.us/reader031/viewer/2022030318/5a649fc27f8b9a2c568b65fb/html5/thumbnails/27.jpg)
Demo
ParaBank(Application under test)
![Page 28: Test automation and Agile software development](https://reader031.vdocuments.us/reader031/viewer/2022030318/5a649fc27f8b9a2c568b65fb/html5/thumbnails/28.jpg)
Case
_Craft an automation implementation strategy
_What do you need? Which questions do you have?
_What would you do in the first sprint?
_What would you do in the second sprint?
_Present your strategy to your coworkers
![Page 29: Test automation and Agile software development](https://reader031.vdocuments.us/reader031/viewer/2022030318/5a649fc27f8b9a2c568b65fb/html5/thumbnails/29.jpg)
And what about…
Test data?
Test environments?
Repeatability?
Reusability?
Planning?
Who?
Features or tests?
![Page 30: Test automation and Agile software development](https://reader031.vdocuments.us/reader031/viewer/2022030318/5a649fc27f8b9a2c568b65fb/html5/thumbnails/30.jpg)
How do you become a
good automation
engineer?
![Page 31: Test automation and Agile software development](https://reader031.vdocuments.us/reader031/viewer/2022030318/5a649fc27f8b9a2c568b65fb/html5/thumbnails/31.jpg)
A good automation engineer…
Has experience in software testing
Has experience in software development
Asks ‘why?’ before asking ‘what?’ or ‘how?’
Creates realistic expectations
![Page 32: Test automation and Agile software development](https://reader031.vdocuments.us/reader031/viewer/2022030318/5a649fc27f8b9a2c568b65fb/html5/thumbnails/32.jpg)
Why do things still go wrong?
Testers are expected to do automation ‘on the side’
Features get precedence over tests
Inefficient automation (automate it all,
automate everything through the UI, …)
Lack of knowledge and experience (craftsmanship)
![Page 33: Test automation and Agile software development](https://reader031.vdocuments.us/reader031/viewer/2022030318/5a649fc27f8b9a2c568b65fb/html5/thumbnails/33.jpg)
Automated Testing
Continuous Testing
![Page 34: Test automation and Agile software development](https://reader031.vdocuments.us/reader031/viewer/2022030318/5a649fc27f8b9a2c568b65fb/html5/thumbnails/34.jpg)
Automated tests
Focused
Informative
Trustworthy
Repeatable
Continuous Testing
![Page 35: Test automation and Agile software development](https://reader031.vdocuments.us/reader031/viewer/2022030318/5a649fc27f8b9a2c568b65fb/html5/thumbnails/35.jpg)
Focused
Write tests at the right level
As close to the implementation as possible
Lower level > narrower scope
Lower level > faster to write, faster to run
![Page 36: Test automation and Agile software development](https://reader031.vdocuments.us/reader031/viewer/2022030318/5a649fc27f8b9a2c568b65fb/html5/thumbnails/36.jpg)
Informative
Your automation should provide useful feedback
Know who the recipients of this feedback are
Reporting (for humans, for systems)
Clean, readable and maintainable code
Clear and unambiguous messages
![Page 37: Test automation and Agile software development](https://reader031.vdocuments.us/reader031/viewer/2022030318/5a649fc27f8b9a2c568b65fb/html5/thumbnails/37.jpg)
Trustworthy
Decisions are made based on test results
You should therefore be able to trust those results
Stable test execution
Eliminate false positives
Eliminate false negatives
![Page 38: Test automation and Agile software development](https://reader031.vdocuments.us/reader031/viewer/2022030318/5a649fc27f8b9a2c568b65fb/html5/thumbnails/38.jpg)
Repeatable
Test data management (this is hard!)
Adopt the right strategy for your tests
Test environments and dependencies
Mocks, stubs, service virtualization
![Page 39: Test automation and Agile software development](https://reader031.vdocuments.us/reader031/viewer/2022030318/5a649fc27f8b9a2c568b65fb/html5/thumbnails/39.jpg)
Test automation
… is a craft in and of itself, done by craftspeople
… is not something you do ‘on the side’
… can shorten the feedback loop in Agile development
… supports testers, does not replace testers
![Page 40: Test automation and Agile software development](https://reader031.vdocuments.us/reader031/viewer/2022030318/5a649fc27f8b9a2c568b65fb/html5/thumbnails/40.jpg)
Contact
_Email: [email protected]
_Blog: https://www.ontestautomation.com
_LinkedIn: https://www.linkedin.com/in/basdijkstra
_Twitter: @_basdijkstra