Test First, Fail Fast.
Simplifying the Tester’s Transition to DevOps
Presented to you by
Adam SatterfieldVP of Testing
Mush HondaVP of Testing
Agenda
❏ Why Testers Fail in DevOps❏ Current DevOps Landscape❏ The Fail Fast Principle❏ Road to Becoming a DevOps Tester❏ Looking Forward❏ How Can KMS Help You?❏ Questions?❏ Appendix
Why Testers Fail with DevOps
When defining the role of a tester
becomes
DevOpsDevelopment Operations
Why Testers Fail with DevOps
When defining the role of a tester
becomes
DevOpsDevelopment Operations
dEvopSEffective Specialists
Why Testers Fail with DevOps
Info on Effective Specialists in DevOps is nonexistent
Emphasis is on TOOLS rather than the criteria for testing SUCCESS
?
So why DO testers fail?
Testers ‘check’ (with automation tools) rather than test ‘Checking’ is a basic functionality (component, happy path)
So why DO testers fail?
Testers ‘check’ (with automation tools) rather than test ‘Checking’ is a basic functionality (component, happy path)
‘Testing’ is traditionally done in Silos
(Ops-owned shared environments, test cases)
So why DO testers fail?
Testers ‘check’ (with automation tools) rather than test ‘Checking’ is a basic functionality (component, happy path)
‘Testing’ is traditionally done in Silos
(Ops-owned shared environments, test cases) No activity/effort baselines
Current DevOps Landscape
Solution = DevOps
BUT
DevOps is TOO BIG
DevOps integration targets product delivery, continuous testing, quality testing, feature development, and maintenance releases in order to improve reliability and
security and provide faster development and deployment cycles. (Wikipedia)“
”
Current DevOps Landscape
Solution = DevOps
BUT
DevOps is TOO BIG
DevOps integration targets product delivery, continuous testing, quality testing, feature development, and maintenance releases in order to improve reliability and
security and provide faster development and deployment cycles. (Wikipedia)“
”Make DevOps actionable with our Fail Fast (Context Driven) Principle
Fail Fast Principle
Does NOT replace testing (still require Testers) Rapid, Prioritized Feedback
Tester owned Automated Acceptance Tests
Fail Fast Principle
Does NOT replace testing (still require Testers) Rapid, Prioritized Feedback
Tester owned Automated Acceptance Tests
Tests extend beyond components and happy
paths
Fail Fast Principle
Does NOT replace testing (still require Testers) Rapid, Prioritized Feedback
Tester owned Automated Acceptance Tests
Tests extend beyond components and happy
paths
Tests execution priority & ‘failed build’ criteria driven by current needs of team
Fail Fast Principle
Does NOT replace testing (still require Testers)
Own/Manage their environments (using tools,
cloud services, etc)
Testers/Effective Specialists:
Rapid, Prioritized FeedbackTester owned Automated
Acceptance Tests
Tests extend beyond components and happy
paths
Tests execution priority & ‘failed build’ criteria driven by current needs of team
Fail Fast Principle
Does NOT replace testing (still require Testers)
Own/Manage their environments (using tools,
cloud services, etc)
Testers/Effective Specialists:
‘Manual’ effort baselined with overall team timelines, before
& after transition
Rapid, Prioritized FeedbackTester owned Automated
Acceptance Tests
Tests extend beyond components and happy
paths
Tests execution priority & ‘failed build’ criteria driven by current needs of team
Why Fail Fast?
Baselining: Measurable time-savings vs SWAG
Need a quantifiable way to measure time savings
Before DevOps
Why Fail Fast?
Baselining: Measurable time-savings vs SWAG
After DevOps
Need a quantifiable way to measure time savings
Why Fail Fast?
Prevention is better than a cure (of a Defect)
Knight Capital Trading Bug
The Road to Becoming a DevOps Tester
Process
Skills
Tools
Measurements
Build your ROADMAP to adopt the Fail-fast principle and help you become successful
Process
Agile Base
Compliments DevOps
Continuous Integration
Key = testing & validating
builds at each step
Testing in PRODUCTION
A Quality Process
Why create ‘Prod-like’ when you can test in
Prod?
Quality of Process = Quality of Product
Skills
Become the CODING TESTER with your increased knowledge of code
NETWORKing - know how computers talk to each other (HTTP & REST)
Context Driven Testing (CDT) - map out complex ideas into actionable lists
Knowledge of backend technologies - API, Webservice, DB testing, etc.
Measurements
Process Health
● Defect Lifecycles● Story maturity● Automation execution cycles
Product Health
● Support calls/tickets● Defect Escapes● Technical debt & Code Coverage using SonarQube (by Release)
Tying it All Together
How do you learn all these things?
Self learning and research Your growth is your responsibility
Recommended learning courses in appendix
Looking Forward
In 2016, Gartner projected that over 25% of Enterprise 2000 companies would be adopting DevOps.
Looking forward to 2017, it is projected that upwards of 85% of Enterprise level companies will be attempting DevOps projects.
Testers are a key piece to delivering rapid, RELIABLE and QUALITY software to customers. Continuous Testing is a Key Component to a solid DevOps implementation
DevOps IS the path forward for that. It is not a fad and testers will need to adopt and learn or fall behind
How can KMS Help?
DevOps Efficiency AuditReduce cycle time and deploy faster with a full efficiency audit of your system.
IDENTIFY EFFICIENCY BOTTLENECKThorough metrics gathering including measuring current cycle times
DETERMINE EFFICIENCY GAINSStep-by-step process analysis from requirements gathering to final QA verification
IMPROVE EFFICIENCY ALTOGETHERRecommended tools and frameworks included with implementation strategy guide + included AWS setup!
Appendix
https://enterprisersproject.com/article/2017/1/devops-trends-emerging-2017-and-beyond
http://www.devopsdigest.com/2017-devops-predictions-1
http://www.gartner.com/newsroom/id/2999017
Reference to slides 14 & 15
ftp://ftp.software.ibm.com/software/rational/info/do-more/RAW14109USEN.pdf
https://www.ibm.com/developerworks/rational/library/dec04/bessin/
Tools Appendix
Jenkins:
https://www.getpostman.com/docs/integrating_with_jenkins
https://www.tutorialspoint.com/jenkins/jenkins_automated_testing.htm