![Page 1: Software Test Automation in Agile Projects - Keytorc Approach](https://reader034.vdocuments.us/reader034/viewer/2022042817/55a6867c1a28abe2088b46a9/html5/thumbnails/1.jpg)
Software Test Automation in
Agile Projects
![Page 2: Software Test Automation in Agile Projects - Keytorc Approach](https://reader034.vdocuments.us/reader034/viewer/2022042817/55a6867c1a28abe2088b46a9/html5/thumbnails/2.jpg)
Agenda
» Test Automation Basics
» Continuous Integration/Delivery in Nutshell
» Stages of CD Pipeline
» Best Practices – Tools & Patterns
» Next Generation Approaches
![Page 3: Software Test Automation in Agile Projects - Keytorc Approach](https://reader034.vdocuments.us/reader034/viewer/2022042817/55a6867c1a28abe2088b46a9/html5/thumbnails/3.jpg)
Test Automation Basics
![Page 4: Software Test Automation in Agile Projects - Keytorc Approach](https://reader034.vdocuments.us/reader034/viewer/2022042817/55a6867c1a28abe2088b46a9/html5/thumbnails/4.jpg)
Test Automation
The major key to the success of software test automation:
To reduce manual testing activities and redundant test operations using a systematic
solution to achieve a better testing coverage.
Test automation is not always necessary,
appropriate, or cost effective. In cases
where we are making decisions
based upon an expected return on
investment, analysis can direct us to
where test automation can benefit us.
These returns are best computed by
comparing the costs and gains achieved
through test automation over
manual testing.
![Page 5: Software Test Automation in Agile Projects - Keytorc Approach](https://reader034.vdocuments.us/reader034/viewer/2022042817/55a6867c1a28abe2088b46a9/html5/thumbnails/5.jpg)
Automation Decision in Sequential Models
Frequent regression testing
Repeated test case execution is required
Faster feedback to the developers
Reduce the human effort
Test same application on multiple environments
![Page 6: Software Test Automation in Agile Projects - Keytorc Approach](https://reader034.vdocuments.us/reader034/viewer/2022042817/55a6867c1a28abe2088b46a9/html5/thumbnails/6.jpg)
Pyramid of Test Automation
Business Facing
Technology Facing
![Page 7: Software Test Automation in Agile Projects - Keytorc Approach](https://reader034.vdocuments.us/reader034/viewer/2022042817/55a6867c1a28abe2088b46a9/html5/thumbnails/7.jpg)
Continuous Integration/Delivery
İn Nutshell
![Page 8: Software Test Automation in Agile Projects - Keytorc Approach](https://reader034.vdocuments.us/reader034/viewer/2022042817/55a6867c1a28abe2088b46a9/html5/thumbnails/8.jpg)
Definition of Continuous Integration
Continuous Integration is a software development practice where members of a team
integrate their work frequently, usually each person integrates at least daily - leading to
multiple integrations per day. Each integration is verified by an automated build
(including test) to detect integration errors as quickly as possible.
(Martin Fowler – Thoughtworks)
![Page 9: Software Test Automation in Agile Projects - Keytorc Approach](https://reader034.vdocuments.us/reader034/viewer/2022042817/55a6867c1a28abe2088b46a9/html5/thumbnails/9.jpg)
Continuous Integration Practices
Maintain a Single Source Repository
Automate the Build
Make Your Build Self-Testing
Everyone Commits To the Mainline Every Day
Every Commit Should Build the Mainline on an Integration Machine
Fix Broken Builds Immediately
Keep the Build Fast
Test in a Clone of the Production Environment
Make it Easy for Anyone to Get the Latest Executable
Everyone can see what's happening
Automate Deployment
![Page 10: Software Test Automation in Agile Projects - Keytorc Approach](https://reader034.vdocuments.us/reader034/viewer/2022042817/55a6867c1a28abe2088b46a9/html5/thumbnails/10.jpg)
A Basic Pipeline
![Page 11: Software Test Automation in Agile Projects - Keytorc Approach](https://reader034.vdocuments.us/reader034/viewer/2022042817/55a6867c1a28abe2088b46a9/html5/thumbnails/11.jpg)
Continuous Delivery
![Page 12: Software Test Automation in Agile Projects - Keytorc Approach](https://reader034.vdocuments.us/reader034/viewer/2022042817/55a6867c1a28abe2088b46a9/html5/thumbnails/12.jpg)
What to Automate
![Page 13: Software Test Automation in Agile Projects - Keytorc Approach](https://reader034.vdocuments.us/reader034/viewer/2022042817/55a6867c1a28abe2088b46a9/html5/thumbnails/13.jpg)
Stages of
Continuos Delivery Pipeline
![Page 14: Software Test Automation in Agile Projects - Keytorc Approach](https://reader034.vdocuments.us/reader034/viewer/2022042817/55a6867c1a28abe2088b46a9/html5/thumbnails/14.jpg)
An Example Toolset
![Page 15: Software Test Automation in Agile Projects - Keytorc Approach](https://reader034.vdocuments.us/reader034/viewer/2022042817/55a6867c1a28abe2088b46a9/html5/thumbnails/15.jpg)
Testing Phase: Commit Stage
CommitStage
AutomatedAcceptance
Stage
Non-FunctionalTest Stage
Manual Tests
SanityCheck
Rather be a scheduled
job
![Page 16: Software Test Automation in Agile Projects - Keytorc Approach](https://reader034.vdocuments.us/reader034/viewer/2022042817/55a6867c1a28abe2088b46a9/html5/thumbnails/16.jpg)
Commit Stage
• Run against every check-in
• If it fails, fix it immediately
• Creates a release candidate
Hint: If next stage(s) (Automated Acceptance Tests) take a significant amount of time
(e.g. More than 30 minutes), embed a small subset of them into commit stage. So,
feedback interval will be decreased to act fast on major incidents.
![Page 17: Software Test Automation in Agile Projects - Keytorc Approach](https://reader034.vdocuments.us/reader034/viewer/2022042817/55a6867c1a28abe2088b46a9/html5/thumbnails/17.jpg)
Testing Phase: Automated Acceptance Stage
CommitStage
AutomatedAcceptance
Stage
Non-FunctionalTest Stage
Manual Tests
Sanity Check
![Page 18: Software Test Automation in Agile Projects - Keytorc Approach](https://reader034.vdocuments.us/reader034/viewer/2022042817/55a6867c1a28abe2088b46a9/html5/thumbnails/18.jpg)
Automated Acceptance Stage
Tips:
Focus on multi-threading for race conditions
Run tests in parallel (Use a supporting framework)
Design atomic scenarios
Manage data strictly
• End-to-end tests in production-like environment
• If it fails, fix it immediately triggered when upstream stage passes
![Page 19: Software Test Automation in Agile Projects - Keytorc Approach](https://reader034.vdocuments.us/reader034/viewer/2022042817/55a6867c1a28abe2088b46a9/html5/thumbnails/19.jpg)
Testing Phase: Non-Functional Test Stage
CommitStage
AutomatedAcceptance
Stage
Non-FunctionalTest Stage
Manual Tests
Sanity Check
![Page 20: Software Test Automation in Agile Projects - Keytorc Approach](https://reader034.vdocuments.us/reader034/viewer/2022042817/55a6867c1a28abe2088b46a9/html5/thumbnails/20.jpg)
Non-Functional Test Stage (Performance)
• Select most suitable test cases (Either widely used or critical for a business)
• Test against a production replica environment, for example staging (As much
as possible)
• Focus on subject matter practices (Anything!)
• Use automate-acceptance tests with counters (As a first step maybe)
![Page 21: Software Test Automation in Agile Projects - Keytorc Approach](https://reader034.vdocuments.us/reader034/viewer/2022042817/55a6867c1a28abe2088b46a9/html5/thumbnails/21.jpg)
Testing Phase: Manual Tests
CommitStage
AutomatedAcceptance
Stage
Non-FunctionalTest Stage
Manual Tests
Sanity Check
![Page 22: Software Test Automation in Agile Projects - Keytorc Approach](https://reader034.vdocuments.us/reader034/viewer/2022042817/55a6867c1a28abe2088b46a9/html5/thumbnails/22.jpg)
Manual Tests
• Focus on non-repetative, unautomated test cases, showcases
• Create room for creativity with exploratory testing
• Realize people dependent tests, like usability
![Page 23: Software Test Automation in Agile Projects - Keytorc Approach](https://reader034.vdocuments.us/reader034/viewer/2022042817/55a6867c1a28abe2088b46a9/html5/thumbnails/23.jpg)
Pipeline Integrated
![Page 24: Software Test Automation in Agile Projects - Keytorc Approach](https://reader034.vdocuments.us/reader034/viewer/2022042817/55a6867c1a28abe2088b46a9/html5/thumbnails/24.jpg)
Pipeline Integrated
![Page 25: Software Test Automation in Agile Projects - Keytorc Approach](https://reader034.vdocuments.us/reader034/viewer/2022042817/55a6867c1a28abe2088b46a9/html5/thumbnails/25.jpg)
Pipeline Patterns & AntiPatterns
Insufficient Parallelization
Ideal Time < 10-15 Mins
No more than 1 hour
Hint: Prefer a paralellizable unit testing framework. TestNG for Java and MbUnit for .Net
might be good choices.
![Page 26: Software Test Automation in Agile Projects - Keytorc Approach](https://reader034.vdocuments.us/reader034/viewer/2022042817/55a6867c1a28abe2088b46a9/html5/thumbnails/26.jpg)
Pipeline Patterns & AntiPatterns
Inflexible Workflow
![Page 27: Software Test Automation in Agile Projects - Keytorc Approach](https://reader034.vdocuments.us/reader034/viewer/2022042817/55a6867c1a28abe2088b46a9/html5/thumbnails/27.jpg)
References
http://www.romaniatesting.ro/to-automate-or-not-to-automate-testing
http://www.softwarequalitymethods.com/papers/star99%20model%20paper.pdf
http://www.continuousdelivery.com/2010/09/deployment-pipeline-anti-patterns/
http://www.testng.org/doc/documentation-main.html#parallel-tests
Referencing Lisa Crispin, Agile Testing Book 2009
Continuous Delivery Workshop Material created by Jez Humble, Martin Fowler, Tom Sulston, & Neal Ford