Automated Exploratory Tests

Download Automated Exploratory Tests

Post on 25-May-2015

1.802 views

Category:

Technology

0 download

Embed Size (px)

DESCRIPTION

automated exploratory test tools for web application. Make your test agile. Do not overpay for your automation. Test all aspects (performance, layout, analytic) not only concentrate on one specific thing.

TRANSCRIPT

<ul><li> 1. Automated ExploratoryTests for Web ApplicationZbyszek Mokun 2010 Cognifide Limited. In commercial confidence only.</li></ul> <p> 2. ObjectiveHow Cognifide has developed automatedtests and where we are going 2010 Cognifide Limited. In commercial confidence only. 3. IntroductionIn short about process and environment 2010 Cognifide Limited. In commercial confidence only. 4. Methodologies, projects Methodologies Projects Web application Agile: SCRUM, Kanban Migration Frequently releases Content Management No time for manualSystems (CMS)regression Short or medium term Lot of work at the end of projectssprint Automation investment, no CI Environmenttime for money return 2010 Cognifide Limited. In commercial confidence only. 5. Common standards forautomationFunctional tests 2010 Cognifide Limited. In commercial confidence only. 6. Test Automation LifecycleWrite TCWriteautomatedscriptRun Manage Update 2010 Cognifide Limited. In commercial confidence only. 7. Advantages and disadvantages Time consuming Cost Coverage by automated tests Invest and wait for return No duplications during Test dependency regression test Only coded functionality is Functionality is tested tested 2010 Cognifide Limited. In commercial confidence only. 8. Can you believe your tests? Incorrect Xpath (locator) used in tests Not all data were verified (unable to write verifications for alldata) CSS/images or other files wasnt load Frequently design changes require xpath (locator) update Incorrect CSS fixes (designs are not reflected by pagelayout) Analytics doesnt work Performance decrease Few thousands of pages (components variation cant testall of them 2010 Cognifide Limited. In commercial confidence only. 9. Migration (upgrade) projects No existing automated test for regression test Few thousands of pages Quick response needed Iterative development 2010 Cognifide Limited. In commercial confidence only. 10. Automated ExploratoryTestsNew approach, theory part 2010 Cognifide Limited. In commercial confidence only. 11. We need automated testsbecause ...Source: http://www.testfacebook.com 2010 Cognifide Limited. In commercial confidence only. 12. What should we test? Two perspectives:UserClient 2010 Cognifide Limited. In commercial confidence only. 13. AspectsCI Environment ClientUserAnalytics Layout Functionality Performance 2010 Cognifide Limited. In commercial confidence only. 14. Looking for a symptomSymptom is a departure from normal function or feelingwhich is noticed by a patient, indicating the presence of disease or abnormality. A symptom is subjective, observed by the patient, and not measured. http://wikipedia.org 2010 Cognifide Limited. In commercial confidence only. 15. Symptoms Layout Performance ContentURL strategyHTML code BugJavaScriptLink checkererrorsServerValidators response (W3C) (404) Analytics 2010 Cognifide Limited. In commercial confidence only. 16. Oracle HeuristicsIt is used by comparing the output(s) of the system under test,for a given test case input, to the outputs that the oracledetermines that product should have. other products (second program that uses a different algorithm toevaluate the same mathematical expression as the productunder test) a consistency oracle that compares the results of one testexecution to another for similarity, a model-based oracle that uses the same model to generate andverify system behavior, or a human beings judgment (i.e. does the program "seem" tothe user to do the correct thing?) 2010 Cognifide Limited. In commercial confidence only. 17. Oracle Heuristic for GUI 2010 Cognifide Limited. In commercial confidence only. 18. Reference symptoms Previous version of application Reference data saved on disc Other version of the same application (different servers) The same script takes pattern and compare One tool for different projectsPerformanceLayoutContent HTML code monitoring 2010 Cognifide Limited. In commercial confidence only. 19. Robots Scripts that are looking for symptoms The same script for all projects Open Source toolsJavaScript Server responseValidators Analytics Link checkerURL Strategyerrors(404)(HTLM, CSS) 2010 Cognifide Limited. In commercial confidence only. 20. Exploratory testingExploratory testing is an approach to software testing that is concisely described as simultaneous learning, test design, test execution and test result interpretation Cem Kaner 2010 Cognifide Limited. In commercial confidence only. 21. What are Automated ExploratoryTests?Meny different definition: Record and play manual exploratory test sessions Combined manual and automated tests Test steps automation Blackbox tests with input/output data generator Not at all possible due to human factor 2010 Cognifide Limited. In commercial confidence only. 22. What are Automated ExploratoryTests?Tests supporting exploratory testing, not necessarily based onthe test scenario, searching areas where there may be errorsor a change in the application, allowing the examination ofapplications faster and more deeply or not to test sitesavailable to man. Zbyszek MokunAutomated Tests do not provide complete information. Theyprovide partial information that might be useful Cem Kaner 2010 Cognifide Limited. In commercial confidence only. 23. Automated Exploratory Tests -advantages One tool Gather all symptoms Use in all projects Quick to introduce to project (half a day) Resistant to appliciaton changes Do not require deep knowledge about application,support the learning process Integrated with Continuous Intergration env Strong solution if combined with automatedfunctional tests Migration or upgrade tests Iterative development 2010 Cognifide Limited. In commercial confidence only. 24. Track the propagation ofchanges in the code- See what pages/components are infected by change- Positive false (code change affect all necessary places?) 2010 Cognifide Limited. In commercial confidence only. 25. Automated ExploratoryTestsPractical part 2010 Cognifide Limited. In commercial confidence only. 26. Report example of use 2010 Cognifide Limited. In commercial confidence only. 27. URL management Sitemap Link crawler CSV file with url list Symptoms as parameter (on, off) Comments (bug id, ...) Hide element Other actions Keep in SVN 2010 Cognifide Limited. In commercial confidence only. 28. Pattern management Easy to use (overwrite pattern) Checkbox (check all) Commit to svn 2010 Cognifide Limited. In commercial confidence only. 29. Dynamic content Reference symptoms need processing Use regexp Hide element Write generic function Robots do not need any additional actions Dynamic components (counters, date, random numbers) Flash/video files Embedded content (twitter, facebook,...) Require functional test Static components about 75% 2010 Cognifide Limited. In commercial confidence only. 30. Mobile version Required (not nice to have), cant avoid testing Mobile pages are more static than desktop version Do not use flash Not all symptoms are easy to check 2010 Cognifide Limited. In commercial confidence only. 31. Narzdzia Selenium II/WebDriver Komparatory (ImageMagick, Pretty Diff) Walidatory (W3C) Performance (JMeter, Proxy) Serwery Proxy (BrowserMob) Firefox/Firebug (NetExport, ConsoleExport, JS Consoleoutput redirector) HAR Viewer (http://www.softwareishard.com/har/viewer/) 2010 Cognifide Limited. In commercial confidence only. 32. Q&amp;A 2010 Cognifide Limited. In commercial confidence only.</p>