1.Michael Mahlberg, 05.06.2012Exploratory Testing The missing half of BDD?Tuesday, June 5, 2012

2. Testing?Tuesday, June 5, 2012 3. AGAIN?Tuesday, June 5, 2012 4. Its not about Railshttp://www.ickr.com/photos/chris_gin/2490980153/Tuesday, June 5, 2012 5. Several StoriesTuesday, June 5, 2012 6. BDD Specication Design by TDDby ExampleContractDefensiveProgramming Software-DevelopmentTuesday, June 5, 2012 7. ExploratoryTestingTuesday, June 5, 2012 8. Tuesday, June 5, 2012 9. Tuesday, June 5, 2012 10. Tuesday, June 5, 2012 11. Tuesday, June 5, 2012 12. Paul Carvalho@can_testTuesday, June 5, 2012 13. Michael BoltonNo, Not this oneTuesday, June 5, 2012 14. Michael Bolton @michaelboltonTuesday, June 5, 2012 15. Markus Grtner @mgaertneTuesday, June 5, 2012 16. Players in the Game James Bach Michael Bolten (not the singer) Cem Kaner Paul CarvalhoTuesday, June 5, 2012 17. RessourcesHumanResource Paul Carvalho http://swtester.blogspot.de/ Michael Bolton http://www.developsense.com/blog/ James Bach http://www.satisce.com/blog/Elisabeth Hendricksonhttp://testobsessed.com/Markus Grtnerhttp://www.shino.de/blog/Tuesday, June 5, 2012 18. Ressources by James BachTopic RessourceSBTM Session Based http://satisce.com/articles/sbtm.pdf Test Management http://people.eecs.ku.edu/~saiedian/ Exploratory TestingTeaching/Fa07/814/Resources/exploratory-testing.pdfHands-on Exploratory http://www.satisce.com/tools/ procedure.pdfTestingRapid Software Testing http://www.satisce.com/rst.pdfTuesday, June 5, 2012 19. The basic misunderstanding is necessary but not sufcientTuesday, June 5, 2012 20. An example from agilesoftware developmentTuesday, June 5, 2012 21. An example from agilesoftware development Working Softwareover Comprehensive DocumentationTuesday, June 5, 2012 22. Derived Assumption: Working software is sufcientTuesday, June 5, 2012 23. Reality:Working software is necessary,but not sufcient!Tuesday, June 5, 2012 24. Tuesday, June 5, 2012 25. Tuesday, June 5, 2012 26. Tuesday, June 5, 2012 27. So whats the problem? Software testing, the process of verifying that asoftware program works as expected Source: http://en.wikipedia.org/wiki/TestTuesday, June 5, 2012 28. So whats the problem? ... but ...Tuesday, June 5, 2012 29. So whats the problem? Test[...]EtymologyFrom Old French test(an earthen vessel,especially a pot in which metals were tried) Source: http://en.wiktionary.org/wiki/testTuesday, June 5, 2012 30. What do test pilots do?Tuesday, June 5, 2012 31. What do test pilots do?Tuesday, June 5, 2012 32. What do test pilots do?Tuesday, June 5, 2012 33. What do test pilots do?Tuesday, June 5, 2012 34. What do test pilots do?Tuesday, June 5, 2012 35. Pushing the envelopeTuesday, June 5, 2012 36. Checking thingsTuesday, June 5, 2012 37. Scripted testingWith a script, you miss thesame things every time. Exploratory Testing @ QUEST 2008" Copyright 2008 Cem KanerTuesday, June 5, 2012 38. Defensive programming - Check all input parameters- Ascertain availability of resources etc. Defensive programming defends against the currentlyimpossible (c2 Wiki)Tuesday, June 5, 2012 39. Design by ContractBetrand MeyerSince 1986Programming language: EiffelTuesday, June 5, 2012 40. Design by ContractBetrand MeyerSince 1986Programming language: Eiffel Preconditions Postconditions InvariantsTuesday, June 5, 2012 41. TDD Unit Tests Black-Box Tests White-Box Tests RSpec etc.Tuesday, June 5, 2012 42. BDD Acceptance Tests (Mostly) Black-Box Tests Based on ScenariosTuesday, June 5, 2012 43. ATDD Bigger gap between test creation andexecution Exclusively Black Box TestingTuesday, June 5, 2012 44. An adage from agile software developmentTuesday, June 5, 2012 45. An adage from agile software developmentINVEST in stories andcome up with SMART tasksTuesday, June 5, 2012 46. INVESTIndependentNegotiableValuableEstimatableSmallTestableTuesday, June 5, 2012 47. SMARTSpecicMesuarableAchievableRealisticTangibleTuesday, June 5, 2012 48. IndependentSpecicNegotiableMesuarableValuableAchievableEstimatableRealisticSmallTangibleTestableTuesday, June 5, 2012 49. Acceptance Criteria To accomodate pre-conditionsAcceptance Criteria can be expressed as Given [Precondition]When [Actor and Action]Then [Observable Result] Naresh Jain Slide 29Tuesday, June 5, 2012 50. Specication by Example... It is the result of a research onhow teams all over the world specify,develop, test and deliver the rightsoftware, without defects, in veryshort iterative delivery cycles. ...Tuesday, June 5, 2012 51. Whats left for the Testers?Tuesday, June 5, 2012 52. DbC vs. TDD Design by Contract implies automated defensive programmingpre- and postconditions and Test Driven Design isinvariants are checked at mostly applied during design-runtime time.Expensive checks areTests are executed wheneverinhibitiveappropriate - but usually notin the productionenvironmentExpensive checks arepossible but unwarrantedTuesday, June 5, 2012 53. Checking vs. TestingChecking is something thatwe do with the motivation ofconfirming existing beliefs Testing is something that we do with the motivation of finding new information. http://www.developsense.com/blog/2009/08/testing-vs-checking/Tuesday, June 5, 2012 54. Test Driven Development is an oxymoronCheck Driven Development?Tuesday, June 5, 2012 55. The Test-CyclesTuesday, June 5, 2012 56. Red-Green-Refactor FailingTest SimplifyCoding Remove RedundancyTuesday, June 5, 2012 57. Exploratory TestingFrom: Paul Carvalhohttp://swtester.blogspot.de/2012/05/what-is-exploratory-testing.htmlTuesday, June 5, 2012 58. So what is Exploratory Testing?"Exploratory Testing is simultaneouslylearning about the system while designing and executing tests, using feedback from the last test to inform the next."Elisabeth Hendrickson, @testobsessed Quoted by Paul CarvalhoTuesday, June 5, 2012 59. Cone of uncertaintyAgile and Iterative Development: A Managers Guide, Craig Larman 2003Tuesday, June 5, 2012 60. Cone of uncertaintyTuesday, June 5, 2012 61. Cone of test-caseshttp://swtester.blogspot.de/2012/05/what-is-exploratory-testing.htmlTuesday, June 5, 2012 62. Agile testing quadrantshttp://lisacrispin.com/wordpress/2011/11/08/using-the-agile-testing-quadrants/Tuesday, June 5, 2012 63. The Prime DirectiveBe Thoughtful and MethodicalThroughout the test procedure, as you complete the tasks, youhave lots of freedom about how you do the work. But you mustwork methodically, and follow the procedure. In the course ofcreating the result for each task, youll nd that you have tomake a lot of guesses, and some of them will be wrong. But youmust think. If you nd yourself making wild and uneducatedguesses about how the product works, areas of instability, oranything else, stop and talk to the Test Manager.James Bach, 08/26/99, http://www.satisfice.com/tools/procedure.pdfTuesday, June 5, 2012 64. Scientic Method Formulate a question HypothesisPrediction Test AnalysisTuesday, June 5, 2012 65. Scientic Method Formulate a question HypothesisPrediction Test AnalysisTuesday, June 5, 2012 66. Exploratory TestingComplete these ve tasks:Identify the purpose of the product.Identify functions.Identify areas of potential instability.Test each function and record problems.Design and record a consistency verication test.Source: http://www.satisce.com/tools/procedure.pdfpg. 11 to 22Tuesday, June 5, 2012 67. Identify the purpose of theproduct With a tangible result (i.e. Document) Value for others - veriable etc. Value for the tester - builds understandingTuesday, June 5, 2012 68. Identify functions By actually using (skimming) the product Explorative (not based on documentation) Tangible output (Function outline)Tuesday, June 5, 2012 69. From James Bach:http://www.satisce.com/tools/procedure.pdfTuesday, June 5, 2012 70. Identify areas of potential instability Results: Documented areas (before testing) Documented data for testing Documented StrategiesTuesday, June 5, 2012 71. Test each function and recordproblems Actually this means: Try to do things the development teams did not think of. Remember: A Problem is just the difference between the expected behavior and the observed behavior - whether it is cause for action has to be decided by the customerTuesday, June 5, 2012 72. Design and record aconsistency verication test Try to extrapolate if the Software wouldrun in (slightly) modied environments.(The original 5 steps are from 1999 (gasp)and relate to specic windows versions -the basic requirements still hold through)Tuesday, June 5, 2012 73. Tool SupportAreaTool URL http://Recording spectorwww.spectorsoft.com/http://RecordingBBTestAssistant www.bbsoftware.co.uk/products.aspx http://RecordingWinkwww.debugmode.com/wink/http://www.sirius-All in OneTestExplorer sqa.com/ http://www.satisce.com/blog/archives/77Tuesday, June 5, 2012 74. Session Based Test Management Original description of ET : 1999 Today : 2012 SBTM: A way to organize exploratorytesting Working in 45 to 90 min Session basedon Charters See: http://satisce.com/articles/sbtm.pdfTuesday, June 5, 2012 75. What are Charters Counterpart to the use case / user story Some of the same attributes - Roles, activities, actors, systems Examples: - Analyze s View menu functionality and report on areas of potential risk.Tuesday, June 5, 2012 76. HierarchyTuesday, June 5, 2012 77. HierarchyAd-Hoc TestingTuesday, June 5, 2012 78. HierarchyExploratory TestingAd-Hoc TestingTuesday, June 5, 2012 79. HierarchySession Based TestingExploratory TestingAd-Hoc TestingTuesday, June 5, 2012 80. HierarchySession Based Test ManagementSession Based Testing Exploratory Testing Ad-Hoc TestingTuesday, June 5, 2012 81. ET-SpectrumTuesday, June 5, 2012 82. ET-SpectrumExploratory Testing according to: http://www.satisce.com/rst.pdfTuesday, June 5, 2012 83. Can we prove it?Testing shows the presence,not the absence of bugs J.N. Buxton and B. Randell, eds, Software Engineering Techniques, April 1970, p. 16. Report on a conference sponsored by the NATO Science Committee, Rome, Italy, 2731 October 1969. via http://en.wikiquote.org/wiki/Edsger_W._DijkstraProgram testing can be used toshow the presence of bugs, butnever to show their absence! Source: Notes On Structured Programming, 1972, at the end of section 3, On TheEdsger Wybe DijkstraReliability of Mechanisms. EWD249 (1970).Tuesday, June 5, 2012 84. TDD Technique to drive the design of your code y, t!r nsa cie s ce uf e s N ot t nBuTuesday, June 5, 2012 85. BDD Technique to capture requirements anddrive desired functionality y, t!r nsa cie s ce uf e s N ot t nBuTuesday, June 5, 2012 86. ET Technique to drive the discovery ofunwanted, unexpected and unpredictedbehaviour y, t!r nsa cie s ce uf e s N ot t nBuTuesday, June 5, 2012 87. About me Consulting on software developmentprocesses and software architecture sincethe last millennium - @MMahlbergGoogle me: Michael MahlbergTuesday, June 5, 2012 88. Discussion?Tuesday, June 5, 2012 89. Flight-EnvelopePhoto Credits http://en.wikipedia.org/wiki/File:PerformanceEnvelope.gif Testpilots: http://www.ickr.com/photos/x-ray_delta_one/4106250812/in/photostream/ Spreading junction http://www.ickr.com/photos/elsie/4164899/in/photostream/ Tokio Tower http://www.ickr.com/photos/wilhelmja/330965980/in/photostream/ Eiffel Tower http://www.ickr.com/photos/gcattiaux/2565101112/in/photostream/ Spectrum http://www.ickr.com/photos/eiriknewth/132859486/in/photostream/ Armor http://www.ickr.com/photos/nicmcphee/1563082378/in/photostream/ Aerobatics http://www.ickr.com/photos/ssandars/6827302/in/photostream/ Airbus http://www.ickr.com/photos/mwanasimba/4097232435/Tuesday, June 5, 2012