building a testing strategy - qaiquest.org · • six thinking hats. edward debono • six action...
TRANSCRIPT
![Page 1: Building a Testing Strategy - qaiquest.org · • Six Thinking Hats. Edward DeBono • Six Action Shoes. Edward DeBono Building a Testing Strategy ©Karen N. Johnson, 2009 Slide 23](https://reader034.vdocuments.us/reader034/viewer/2022050323/5f7d5cd7538fa64b5240ef24/html5/thumbnails/1.jpg)
• Karen N. Johnson
Building a Testing Strategy
Karen N. Johnson
Building a Testing Strategy 1© Karen N. Johnson, 2009
![Page 2: Building a Testing Strategy - qaiquest.org · • Six Thinking Hats. Edward DeBono • Six Action Shoes. Edward DeBono Building a Testing Strategy ©Karen N. Johnson, 2009 Slide 23](https://reader034.vdocuments.us/reader034/viewer/2022050323/5f7d5cd7538fa64b5240ef24/html5/thumbnails/2.jpg)
Who am I?
Karen N. Johnson
Independent Software Test Consultant http://www.karennjohnson.com
Hosted on Tech Target http://searchsoftwarequality.techtarget.com/
Co‐founder of WREST Workshop http://www.wrestworkshop.com/Home.html
My blog http://www.testingreflections.com/blog/3804
Building a Testing Strategy © Karen N. Johnson, 2009 Slide 2
![Page 3: Building a Testing Strategy - qaiquest.org · • Six Thinking Hats. Edward DeBono • Six Action Shoes. Edward DeBono Building a Testing Strategy ©Karen N. Johnson, 2009 Slide 23](https://reader034.vdocuments.us/reader034/viewer/2022050323/5f7d5cd7538fa64b5240ef24/html5/thumbnails/3.jpg)
Within the context of building a test strategy, what matters?
• Building a test strategy isn’t about writing a document.
• Building a test strategy is about thinking creatively and strategically. It is the thinking and planning behind the test strategy and the ongoing tactical adjustments throughout the project that matter.
Building a Testing Strategy © Karen N. Johnson, 2009 3
![Page 4: Building a Testing Strategy - qaiquest.org · • Six Thinking Hats. Edward DeBono • Six Action Shoes. Edward DeBono Building a Testing Strategy ©Karen N. Johnson, 2009 Slide 23](https://reader034.vdocuments.us/reader034/viewer/2022050323/5f7d5cd7538fa64b5240ef24/html5/thumbnails/4.jpg)
Creative Thinking
Building a Testing Strategy © Karen N. Johnson, 2009 4
![Page 5: Building a Testing Strategy - qaiquest.org · • Six Thinking Hats. Edward DeBono • Six Action Shoes. Edward DeBono Building a Testing Strategy ©Karen N. Johnson, 2009 Slide 23](https://reader034.vdocuments.us/reader034/viewer/2022050323/5f7d5cd7538fa64b5240ef24/html5/thumbnails/5.jpg)
Research
• Troll online forums on specific technology topics
• Scour for ideas from the specific technology research
• Use testing conferences and social networking to connect to testers in the same/similar space
• Sometimes researching a new technology is the best place to begin collecting ideas so you can host stronger brainstorming and investigation sessions
Building a Testing Strategy © Karen N. Johnson, 2009 5
![Page 6: Building a Testing Strategy - qaiquest.org · • Six Thinking Hats. Edward DeBono • Six Action Shoes. Edward DeBono Building a Testing Strategy ©Karen N. Johnson, 2009 Slide 23](https://reader034.vdocuments.us/reader034/viewer/2022050323/5f7d5cd7538fa64b5240ef24/html5/thumbnails/6.jpg)
Solitary Think Sessions
• Leave the office• Dedicate a timed think session• Think about potential catastrophes• Plan investigations and
conversations• Acknowledge challenges • Draw out ideas in any format that
helps; including multiple formats• Give your self permission to follow
your idea paths without editing
Building a Testing Strategy © Karen N. Johnson, 2009 6
![Page 7: Building a Testing Strategy - qaiquest.org · • Six Thinking Hats. Edward DeBono • Six Action Shoes. Edward DeBono Building a Testing Strategy ©Karen N. Johnson, 2009 Slide 23](https://reader034.vdocuments.us/reader034/viewer/2022050323/5f7d5cd7538fa64b5240ef24/html5/thumbnails/7.jpg)
Brainstorming
• Create a safe environment• Build no censor rules• Encourage creative and unusual
ideas• Watch for team reactions to ideas
raised; these reactions can be informal risk assessments
• Note ideas worth taking offline for further investigation
• Record as much information as possible; use an “idea collector”
• Keep ideas for consideration later in the project
• Look for reoccurring concerns that might be raised in different ways
Building a Testing Strategy © Karen N. Johnson, 2009 7
![Page 8: Building a Testing Strategy - qaiquest.org · • Six Thinking Hats. Edward DeBono • Six Action Shoes. Edward DeBono Building a Testing Strategy ©Karen N. Johnson, 2009 Slide 23](https://reader034.vdocuments.us/reader034/viewer/2022050323/5f7d5cd7538fa64b5240ef24/html5/thumbnails/8.jpg)
Creative Thinking
• Research
• Solitary think sessions
• Brainstorming
• Investigation sessions
• Investigation skills
• Academic test review• Perpetual curiosity• Six thinking hats• Six action shoes• Product analysis• Collecting information
Building a Testing Strategy © Karen N. Johnson, 2009 8
![Page 9: Building a Testing Strategy - qaiquest.org · • Six Thinking Hats. Edward DeBono • Six Action Shoes. Edward DeBono Building a Testing Strategy ©Karen N. Johnson, 2009 Slide 23](https://reader034.vdocuments.us/reader034/viewer/2022050323/5f7d5cd7538fa64b5240ef24/html5/thumbnails/9.jpg)
Strategic Thinking
Building a Testing Strategy © Karen N. Johnson, 2009 9
![Page 10: Building a Testing Strategy - qaiquest.org · • Six Thinking Hats. Edward DeBono • Six Action Shoes. Edward DeBono Building a Testing Strategy ©Karen N. Johnson, 2009 Slide 23](https://reader034.vdocuments.us/reader034/viewer/2022050323/5f7d5cd7538fa64b5240ef24/html5/thumbnails/10.jpg)
Vision
• Overall release goals• Individual test plan goals• Individual goals• Have you communicated your
vision clearly and consistently?
Building a Testing Strategy © Karen N. Johnson, 2009 10
![Page 11: Building a Testing Strategy - qaiquest.org · • Six Thinking Hats. Edward DeBono • Six Action Shoes. Edward DeBono Building a Testing Strategy ©Karen N. Johnson, 2009 Slide 23](https://reader034.vdocuments.us/reader034/viewer/2022050323/5f7d5cd7538fa64b5240ef24/html5/thumbnails/11.jpg)
Risk
• Formal methods: FMEA, MMEA
• Informal methods• Project team• Test team• Collect opinions (support,
customers, stakeholders, developers)
Building a Testing Strategy © Karen N. Johnson, 2009 11
![Page 12: Building a Testing Strategy - qaiquest.org · • Six Thinking Hats. Edward DeBono • Six Action Shoes. Edward DeBono Building a Testing Strategy ©Karen N. Johnson, 2009 Slide 23](https://reader034.vdocuments.us/reader034/viewer/2022050323/5f7d5cd7538fa64b5240ef24/html5/thumbnails/12.jpg)
Surrounding the Product
• Service level agreements• Regulations• User acceptance testing
Building a Testing Strategy © Karen N. Johnson, 2009 12
![Page 13: Building a Testing Strategy - qaiquest.org · • Six Thinking Hats. Edward DeBono • Six Action Shoes. Edward DeBono Building a Testing Strategy ©Karen N. Johnson, 2009 Slide 23](https://reader034.vdocuments.us/reader034/viewer/2022050323/5f7d5cd7538fa64b5240ef24/html5/thumbnails/13.jpg)
Strategic Thinking
• Vision• Risk• Surrounding the product• SDLC• Politics• Alliances
• Resources• Money• Estimates• Direction• Tactics• Stakeholders
Building a Testing Strategy 13© Karen N. Johnson, 2009
![Page 14: Building a Testing Strategy - qaiquest.org · • Six Thinking Hats. Edward DeBono • Six Action Shoes. Edward DeBono Building a Testing Strategy ©Karen N. Johnson, 2009 Slide 23](https://reader034.vdocuments.us/reader034/viewer/2022050323/5f7d5cd7538fa64b5240ef24/html5/thumbnails/14.jpg)
The Test Strategy Document
Building a Testing Strategy © Karen N. Johnson, 2009 14
![Page 15: Building a Testing Strategy - qaiquest.org · • Six Thinking Hats. Edward DeBono • Six Action Shoes. Edward DeBono Building a Testing Strategy ©Karen N. Johnson, 2009 Slide 23](https://reader034.vdocuments.us/reader034/viewer/2022050323/5f7d5cd7538fa64b5240ef24/html5/thumbnails/15.jpg)
Possible Components of a Test Strategy Document
• Project scope• Risk analysis• Types of testing• Test environment• Test data• Release cycle• Defect reporting• Resources • Estimates• Project plan• Status reporting• Acceptance criteria
Building a Testing Strategy © Karen N. Johnson, 2009 Slide 15
![Page 16: Building a Testing Strategy - qaiquest.org · • Six Thinking Hats. Edward DeBono • Six Action Shoes. Edward DeBono Building a Testing Strategy ©Karen N. Johnson, 2009 Slide 23](https://reader034.vdocuments.us/reader034/viewer/2022050323/5f7d5cd7538fa64b5240ef24/html5/thumbnails/16.jpg)
Project Scope
• Features in/out for the release
• Testing in/out for release
• Tie features to testing approach
• Scope includes your vision for the release.
Building a Testing Strategy © Karen N. Johnson, 2009 Slide 16
![Page 17: Building a Testing Strategy - qaiquest.org · • Six Thinking Hats. Edward DeBono • Six Action Shoes. Edward DeBono Building a Testing Strategy ©Karen N. Johnson, 2009 Slide 23](https://reader034.vdocuments.us/reader034/viewer/2022050323/5f7d5cd7538fa64b5240ef24/html5/thumbnails/17.jpg)
Types of Testing
• Opportunity to delineate the types of testing.
• Different tools, skills and resources may be needed to tackle each form of testing.
• Explaining more about testing helps clarify the needs of your team.
Building a Testing Strategy © Karen N. Johnson, 2009 17
![Page 18: Building a Testing Strategy - qaiquest.org · • Six Thinking Hats. Edward DeBono • Six Action Shoes. Edward DeBono Building a Testing Strategy ©Karen N. Johnson, 2009 Slide 23](https://reader034.vdocuments.us/reader034/viewer/2022050323/5f7d5cd7538fa64b5240ef24/html5/thumbnails/18.jpg)
Test Environment
• Identify the environment• Compare test to production in
terms of configuration, server class, etc. and where the differences could impact testing.
• If there is a risk that the test environment might not be ready, the strategy is a place to raise the conversation.
• Ongoing maintenance and support for the test environment are possible topics.
Building a Testing Strategy © Karen N. Johnson, 2009 Slide 18
![Page 19: Building a Testing Strategy - qaiquest.org · • Six Thinking Hats. Edward DeBono • Six Action Shoes. Edward DeBono Building a Testing Strategy ©Karen N. Johnson, 2009 Slide 23](https://reader034.vdocuments.us/reader034/viewer/2022050323/5f7d5cd7538fa64b5240ef24/html5/thumbnails/19.jpg)
Release Cycle
• Is the internal release cycle understood?
• Is the cycle for the release the same as in times past?
Building a Testing Strategy © Karen N. Johnson, 2009 19
![Page 20: Building a Testing Strategy - qaiquest.org · • Six Thinking Hats. Edward DeBono • Six Action Shoes. Edward DeBono Building a Testing Strategy ©Karen N. Johnson, 2009 Slide 23](https://reader034.vdocuments.us/reader034/viewer/2022050323/5f7d5cd7538fa64b5240ef24/html5/thumbnails/20.jpg)
Stakeholders
• Release criteria• Defect priority• Decision makers at the
go/no-go meeting• Timing of the release
meeting
Building a Testing Strategy © Karen N. Johnson, 2009 20
![Page 21: Building a Testing Strategy - qaiquest.org · • Six Thinking Hats. Edward DeBono • Six Action Shoes. Edward DeBono Building a Testing Strategy ©Karen N. Johnson, 2009 Slide 23](https://reader034.vdocuments.us/reader034/viewer/2022050323/5f7d5cd7538fa64b5240ef24/html5/thumbnails/21.jpg)
The Test Strategy Document
• Approvers• Project scope• Risk analysis• Types of testing• Test environment• Test data• Release cycle
• Defect reporting• Resources• Milestones & project
plans• Status reporting• Acceptance criteria• Post release
retrospectives
Building a Testing Strategy © Karen N. Johnson, 2009 21
![Page 22: Building a Testing Strategy - qaiquest.org · • Six Thinking Hats. Edward DeBono • Six Action Shoes. Edward DeBono Building a Testing Strategy ©Karen N. Johnson, 2009 Slide 23](https://reader034.vdocuments.us/reader034/viewer/2022050323/5f7d5cd7538fa64b5240ef24/html5/thumbnails/22.jpg)
Conclusions
A well-written strategy is your statement of work, your bill of goods and should be written at the front of the project and updated throughout.
Building a Testing Strategy © Karen N. Johnson, 2009 22
![Page 23: Building a Testing Strategy - qaiquest.org · • Six Thinking Hats. Edward DeBono • Six Action Shoes. Edward DeBono Building a Testing Strategy ©Karen N. Johnson, 2009 Slide 23](https://reader034.vdocuments.us/reader034/viewer/2022050323/5f7d5cd7538fa64b5240ef24/html5/thumbnails/23.jpg)
References
• “Building a Software Test Strategy,” Karen N. Johnson http://www.informit.com/articles/article.aspx?p=1146504
• Webcast: “Making Sense of Software Tests,” Hosted by Tech Target. Presenter: Karen N. Johnson http://searchsoftwarequality.techtarget.com/guide/allInOne/category/0,296296,sid92_tax308063,00.html
• Risk Analysis for Web Testing, Karen N. Johnson http://www.karennjohnson.com/pdf/Risk_Analysis.pdf
• The Seven Basic Principles of the Context-Driven School see: http://www.context-driven-testing.com/
• Harvard Business Review. September 2008. article: “How Pixar fosters collective creativity” by Ed Catmull
• Six Thinking Hats. Edward DeBono• Six Action Shoes. Edward DeBono
Building a Testing Strategy © Karen N. Johnson, 2009 Slide 23
![Page 24: Building a Testing Strategy - qaiquest.org · • Six Thinking Hats. Edward DeBono • Six Action Shoes. Edward DeBono Building a Testing Strategy ©Karen N. Johnson, 2009 Slide 23](https://reader034.vdocuments.us/reader034/viewer/2022050323/5f7d5cd7538fa64b5240ef24/html5/thumbnails/24.jpg)
References
• “Documenting your software test project,” Karen N. Johnson and Michael D. Kelly http://searchsoftwarequality.techtarget.com/tip/0,289483,sid92_gci1284632,00.html
• Webcast: “How to plan your software test projects” Hosted by Tech Target. Presenters: Karen N. Johnson and Michael D. Kelly. Link: http://searchsoftwarequality.bitpipe.com/detail/RES/1196440596_441.html?bucket=WC&topic=306121
• “Developing a Project Test Strategy,” Michael D. Kelly, http://www.informit.com/articles/article.aspx?p=355875
• Satisfice Heuristic Test Planning Context Model: http://www.satisfice.com/tools/satisfice-cm.pdf• Satisfice Test Planning Guide: Building the Plan: http://www.satisfice.com/tools/build-the-plan.pdf• Satisfice Test Plan Evaluation Model: http://www.satisfice.com/tools/tpe-model.pdf• Test Strategy Blog post, James Bach, see: http://www.satisfice.com/blog/archives/63
Building a Testing Strategy © Karen N. Johnson, 2009 Slide 24
![Page 25: Building a Testing Strategy - qaiquest.org · • Six Thinking Hats. Edward DeBono • Six Action Shoes. Edward DeBono Building a Testing Strategy ©Karen N. Johnson, 2009 Slide 23](https://reader034.vdocuments.us/reader034/viewer/2022050323/5f7d5cd7538fa64b5240ef24/html5/thumbnails/25.jpg)
Thank you.
Contact information:
Karen N. Johnson
Site: http://www.karennjohnson.com
Email: [email protected]
Building a Testing Strategy © Karen N. Johnson, 2009 25