![Page 1: Test Automation Zombie Apocalypse · 2014-01-23 · Automation Zombie Apocalypse" Presented by: Dale Emery DHE Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f8617f188123416b81e5f55/html5/thumbnails/1.jpg)
BT8 Session 6/6/2013 2:15 PM
“How to Survive the Coming Test Automation Zombie Apocalypse"
Presented by:
Dale Emery DHE
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073 888‐268‐8770 ∙ 904‐278‐0524 ∙ [email protected] ∙ www.sqe.com
![Page 2: Test Automation Zombie Apocalypse · 2014-01-23 · Automation Zombie Apocalypse" Presented by: Dale Emery DHE Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f8617f188123416b81e5f55/html5/thumbnails/2.jpg)
Dale Emery DHE
Since 1980, Dale Emery has worked in both IT organizations and software product development companies as a developer, manager, process steward, trainer, and consultant. He helps people apply the agile values of communication, feedback, simplicity, courage, and respect to software development. Dale's combination of deep technical expertise and extensive organizational development experience makes him particularly effective in working with software teams. In 2007 Dale received the Ward Cunningham Gentle Voice of Reason Award, which the Agile Alliance created to recognize Dale’s unique contribution to the agile community. Dale's personal mission is to help people create joy, value, and meaning in their work. Learn more about Dale at dhemery.com.
![Page 3: Test Automation Zombie Apocalypse · 2014-01-23 · Automation Zombie Apocalypse" Presented by: Dale Emery DHE Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f8617f188123416b81e5f55/html5/thumbnails/3.jpg)
How to Survive the Coming Test Automation
Zombie Apocalypse
Dale Emeryhttp://dhemery.com
@dhemery
![Page 4: Test Automation Zombie Apocalypse · 2014-01-23 · Automation Zombie Apocalypse" Presented by: Dale Emery DHE Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f8617f188123416b81e5f55/html5/thumbnails/4.jpg)
Too gruesome to display
![Page 5: Test Automation Zombie Apocalypse · 2014-01-23 · Automation Zombie Apocalypse" Presented by: Dale Emery DHE Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f8617f188123416b81e5f55/html5/thumbnails/5.jpg)
The Trouble With Zombies
LegionRelentlessInfectious
Eat your brains
Usually not as prettyas the one
on the previous slide
![Page 6: Test Automation Zombie Apocalypse · 2014-01-23 · Automation Zombie Apocalypse" Presented by: Dale Emery DHE Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f8617f188123416b81e5f55/html5/thumbnails/6.jpg)
Test Automation ZombiesLegion
Relentless
Infectious
Eat your brains
Look just like ideas
Oddly attractive
![Page 7: Test Automation Zombie Apocalypse · 2014-01-23 · Automation Zombie Apocalypse" Presented by: Dale Emery DHE Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f8617f188123416b81e5f55/html5/thumbnails/7.jpg)
The Trouble with Test Automation Zombies
Sometimes the best you can do
They are easy
Lull you into doing them by habit
Focus on the short term
At the expense of the (not-so-)long term
![Page 8: Test Automation Zombie Apocalypse · 2014-01-23 · Automation Zombie Apocalypse" Presented by: Dale Emery DHE Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f8617f188123416b81e5f55/html5/thumbnails/8.jpg)
Pound FoolishZombies
![Page 9: Test Automation Zombie Apocalypse · 2014-01-23 · Automation Zombie Apocalypse" Presented by: Dale Emery DHE Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f8617f188123416b81e5f55/html5/thumbnails/9.jpg)
The Record and Playback Zombie
AppealLots of tests quickly, (almost) for free
DangerRecorded tests are
unexpressive, brittle, unmaintainable
Can’t record testsuntil the software is done
![Page 10: Test Automation Zombie Apocalypse · 2014-01-23 · Automation Zombie Apocalypse" Presented by: Dale Emery DHE Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f8617f188123416b81e5f55/html5/thumbnails/10.jpg)
The Automation is Easy Zombie
AppealInexpensive automators
DangerTests written by unskilled programmersare unexpressive, brittle, unmaintainable
![Page 11: Test Automation Zombie Apocalypse · 2014-01-23 · Automation Zombie Apocalypse" Presented by: Dale Emery DHE Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f8617f188123416b81e5f55/html5/thumbnails/11.jpg)
Surviving Pound Foolish Zombies
Remember:Test automation is real software development
Test automation takes timeBig cost is maintenance
Writing maintainable code is hard
![Page 12: Test Automation Zombie Apocalypse · 2014-01-23 · Automation Zombie Apocalypse" Presented by: Dale Emery DHE Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f8617f188123416b81e5f55/html5/thumbnails/12.jpg)
More Surviving Pound Foolish Zombies
Stop record and playback
Refactor recorded tests to make themexpressive, resilient, maintainable
Use real programmers...
with the experience to value maintainability...
and the programming skillto write maintainable test code
![Page 13: Test Automation Zombie Apocalypse · 2014-01-23 · Automation Zombie Apocalypse" Presented by: Dale Emery DHE Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f8617f188123416b81e5f55/html5/thumbnails/13.jpg)
DisplacementZombies
![Page 14: Test Automation Zombie Apocalypse · 2014-01-23 · Automation Zombie Apocalypse" Presented by: Dale Emery DHE Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f8617f188123416b81e5f55/html5/thumbnails/14.jpg)
The Automate Last ZombieAppeal
“Can’t automate tests before system is written”
DangerFeedback is delayed
The system is harder to test(not designed for testability)
Missed opportunityto use tests as guidance
![Page 15: Test Automation Zombie Apocalypse · 2014-01-23 · Automation Zombie Apocalypse" Presented by: Dale Emery DHE Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f8617f188123416b81e5f55/html5/thumbnails/15.jpg)
The Dedicated Test Automator Zombie
AppealKeep developers focused on developing
DangerTest automation lags development
Feedback is delayedTypically uses less-skilled programmers
![Page 16: Test Automation Zombie Apocalypse · 2014-01-23 · Automation Zombie Apocalypse" Presented by: Dale Emery DHE Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f8617f188123416b81e5f55/html5/thumbnails/16.jpg)
The Test Automation Group Zombie
AppealKeep entire development team
focused on developing
DangerSame as dedicated test automators
Organizational boundariesmake delays even worse
![Page 17: Test Automation Zombie Apocalypse · 2014-01-23 · Automation Zombie Apocalypse" Presented by: Dale Emery DHE Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f8617f188123416b81e5f55/html5/thumbnails/17.jpg)
Surviving Displacement Zombies
Create automatable examplesbefore development begins
Developers automate testsDone includes test automation
![Page 18: Test Automation Zombie Apocalypse · 2014-01-23 · Automation Zombie Apocalypse" Presented by: Dale Emery DHE Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f8617f188123416b81e5f55/html5/thumbnails/18.jpg)
MathZombies
![Page 19: Test Automation Zombie Apocalypse · 2014-01-23 · Automation Zombie Apocalypse" Presented by: Dale Emery DHE Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f8617f188123416b81e5f55/html5/thumbnails/19.jpg)
The Test Case Count Zombie
AppealConfidence that the system is tested
DangerFocuses on ease of automating each test
Instead of on the value of the tests
![Page 20: Test Automation Zombie Apocalypse · 2014-01-23 · Automation Zombie Apocalypse" Presented by: Dale Emery DHE Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f8617f188123416b81e5f55/html5/thumbnails/20.jpg)
The Code Coverage Zombie
AppealConfidence that the system is tested
DangerConfidence is unwarranted
(executed does not mean tested)
![Page 21: Test Automation Zombie Apocalypse · 2014-01-23 · Automation Zombie Apocalypse" Presented by: Dale Emery DHE Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f8617f188123416b81e5f55/html5/thumbnails/21.jpg)
Surviving Math Zombies
Focus on automating teststhat provide value
Use code coverage toolsonly to identify what is not tested
![Page 22: Test Automation Zombie Apocalypse · 2014-01-23 · Automation Zombie Apocalypse" Presented by: Dale Emery DHE Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f8617f188123416b81e5f55/html5/thumbnails/22.jpg)
StealthZombies
![Page 23: Test Automation Zombie Apocalypse · 2014-01-23 · Automation Zombie Apocalypse" Presented by: Dale Emery DHE Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f8617f188123416b81e5f55/html5/thumbnails/23.jpg)
The Dedicated First Responder Zombie
AppealRelieves the development team from having
to run, maintain, respond to tests
DangerDelays feedback
(through first responders)
![Page 24: Test Automation Zombie Apocalypse · 2014-01-23 · Automation Zombie Apocalypse" Presented by: Dale Emery DHE Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f8617f188123416b81e5f55/html5/thumbnails/24.jpg)
The Hide the Broken Feature Zombie
AppealReduce distraction by known failures
DangerReduces urgency to fix the features
(Disable tests for known-broken features)
![Page 25: Test Automation Zombie Apocalypse · 2014-01-23 · Automation Zombie Apocalypse" Presented by: Dale Emery DHE Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f8617f188123416b81e5f55/html5/thumbnails/25.jpg)
The Hide the Flaky Test Zombie
AppealReduce wasted effort diagnosing tests
DangerDiscards potentially useful test results
Reduces urgency to make tests more robust
(Disable or ignore flaky tests)
![Page 26: Test Automation Zombie Apocalypse · 2014-01-23 · Automation Zombie Apocalypse" Presented by: Dale Emery DHE Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f8617f188123416b81e5f55/html5/thumbnails/26.jpg)
The Automatic Rerun Zombie
AppealReduce wasted effort diagnosing tests
DangerDiscards potentially useful test results
Reduces motivation to make tests more robustIncreases suite run time
![Page 27: Test Automation Zombie Apocalypse · 2014-01-23 · Automation Zombie Apocalypse" Presented by: Dale Emery DHE Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f8617f188123416b81e5f55/html5/thumbnails/27.jpg)
Surviving Stealth ZombiesFix flaky tests now
Fix broken features now
Bypass flaky technology
Analyze every failure
Include automated testsin code promotion procedures
Reserve automatic reruns for exceptional cases
![Page 28: Test Automation Zombie Apocalypse · 2014-01-23 · Automation Zombie Apocalypse" Presented by: Dale Emery DHE Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f8617f188123416b81e5f55/html5/thumbnails/28.jpg)
HackerZombies
![Page 29: Test Automation Zombie Apocalypse · 2014-01-23 · Automation Zombie Apocalypse" Presented by: Dale Emery DHE Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f8617f188123416b81e5f55/html5/thumbnails/29.jpg)
The Fixed Wait Zombie
AppealEasy way to deal with variable response times
DangerFixed waits always increase
Test suite execution timebecomes maximally pessimistic
Thread.sleep(8000);
![Page 30: Test Automation Zombie Apocalypse · 2014-01-23 · Automation Zombie Apocalypse" Presented by: Dale Emery DHE Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f8617f188123416b81e5f55/html5/thumbnails/30.jpg)
Surviving The Fixed Wait Zombie
Bypass slow, variable technologies
Poll for the relevant condition
Arrange to be notifiedwhen the condition becomes satisfied
Create a utility for fixed waits
Instrument the utilityto gather information
![Page 31: Test Automation Zombie Apocalypse · 2014-01-23 · Automation Zombie Apocalypse" Presented by: Dale Emery DHE Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f8617f188123416b81e5f55/html5/thumbnails/31.jpg)
The Dependency Chain Zombie
AppealSpeed up test suites
DangerPrevents running tests independently
Skips subsequent testseven if they would provide value
Harder to understandwhat responsibility each test is testing
![Page 32: Test Automation Zombie Apocalypse · 2014-01-23 · Automation Zombie Apocalypse" Presented by: Dale Emery DHE Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f8617f188123416b81e5f55/html5/thumbnails/32.jpg)
Surviving The Dependency Chain Zombie
Make tests independent
Bypass slow technologiesfor setup and verification
Let each testestablish its own preconditions
![Page 33: Test Automation Zombie Apocalypse · 2014-01-23 · Automation Zombie Apocalypse" Presented by: Dale Emery DHE Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f8617f188123416b81e5f55/html5/thumbnails/33.jpg)
The Predefined Test Database Zombie
AppealSpeed up test suites
Often already exists for manual testing
DangerHides essential details from test code
Harder to understand what responsibilityeach test is testing
![Page 34: Test Automation Zombie Apocalypse · 2014-01-23 · Automation Zombie Apocalypse" Presented by: Dale Emery DHE Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f8617f188123416b81e5f55/html5/thumbnails/34.jpg)
Surviving The Predefined Test Database Zombie
Express every essential detaildirectly in the text of the test code
Let each testcreate its own data
Use the builder patternto create non-trivial test data
![Page 35: Test Automation Zombie Apocalypse · 2014-01-23 · Automation Zombie Apocalypse" Presented by: Dale Emery DHE Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f8617f188123416b81e5f55/html5/thumbnails/35.jpg)
The Chewy GUI Zombie
AppealAvailable and accessible to automators
Tests whole system
DangerMakes tests dependent on most volatile interface
Increases test execution timeHarder to diagnose failures
(Testing only through the GUI)
![Page 36: Test Automation Zombie Apocalypse · 2014-01-23 · Automation Zombie Apocalypse" Presented by: Dale Emery DHE Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f8617f188123416b81e5f55/html5/thumbnails/36.jpg)
SurvivingThe Chewy GUI Zombie
Bypass the GUI whenever it isnot essentialfor the test
![Page 37: Test Automation Zombie Apocalypse · 2014-01-23 · Automation Zombie Apocalypse" Presented by: Dale Emery DHE Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073](https://reader034.vdocuments.us/reader034/viewer/2022042711/5f8617f188123416b81e5f55/html5/thumbnails/37.jpg)
How to Survive the Coming Test Automation
Zombie Apocalypse
Dale Emeryhttp://dhemery.com
@dhemery
Now You Know