Test Automation: An Test Automation: An Architected ApproachArchitected Approach

Dan YoungDan YoungMarch 17th, 2005March 17th, 2005

dyoung@bbandt.comdyoung@bbandt.com

What goes wrong?What goes wrong?

Vendors push record/playback as a Vendors push record/playback as a viable solutionviable solutionManagers want to believe the hype Managers want to believe the hype and believe that they can see results and believe that they can see results immediatelyimmediatelyUsers under pressure rush for any Users under pressure rush for any implementation that worksimplementation that works

Planning an Automation ProjectPlanning an Automation Project

RequirementsRequirements

Balancing PersonnelBalancing Personnel

An Automation PlanAn Automation Plan

Return on InvestmentReturn on Investment

An Architected SolutionAn Architected Solution

Encapsulated functions (functions Encapsulated functions (functions within functions)within functions)

Encapsulated test casesEncapsulated test cases

Managing ResultsManaging Results

Pre-scriptingPre-scripting

An Architected Solution: An Architected Solution: Encapsulated Non-Scripted Encapsulated Non-Scripted

FunctionsFunctionsLow-level “non-Low-level “non-scripted” functions scripted” functions handle common handle common processes that a business processes that a business analyst isn’t necessarily analyst isn’t necessarily concerned with.concerned with.

May accept parametersMay accept parameters

May return a valueMay return a value

Recursion may be used Recursion may be used at this levelat this level

An Architected Solution: An Architected Solution: Encapsulated Scripted FunctionsEncapsulated Scripted Functions

Handle the entire process of what Handle the entire process of what a test case doesa test case does

Accept parameters, and parse Accept parameters, and parse variables as appropriatevariables as appropriate

Most likely do not return a valueMost likely do not return a value

Do write a resultDo write a result

An Architected Solution: An Architected Solution: Sample ApplicationSample Application

Contact Contact Management Management softwaresoftware

Sample Sample Ability: Add Ability: Add New ContactNew Contact

An Architected Solution:An Architected Solution: Encapsulated Scripted Functions II Encapsulated Scripted Functions II

The first step is to parse the parameters The first step is to parse the parameters passed from the test casepassed from the test caseIndividual values are assigned to Individual values are assigned to variables for later usevariables for later use

An Architected Solution:An Architected Solution: Encapsulated Scripted Functions III Encapsulated Scripted Functions III

Function then performs Function then performs the entire test case, the entire test case, including launching the including launching the app, selecting the app, selecting the database, populating the database, populating the fields, verification and fields, verification and cleanupcleanup

Errors (expected or Errors (expected or unexpected) are handled unexpected) are handled within the functionwithin the function

An Architected Solution: An Architected Solution: Encapsulated Test casesEncapsulated Test cases

Test case focuses on a Test case focuses on a particular area of particular area of functionalityfunctionalityTest case consists of Test case consists of initializing variables initializing variables and making a single and making a single function call. function call. Anyone can add test Anyone can add test cases with very little cases with very little training, or even training, or even knowledge of the AUTknowledge of the AUT

An Architected Solution:An Architected Solution: Managing Results: Flat File Model Managing Results: Flat File Model

All results written to an All results written to an external txt or csv fileexternal txt or csv fileResults written to a Results written to a particular directory structure particular directory structure for ease of comparison by a for ease of comparison by a third party comparison third party comparison utilityutilityOnly files that fail Only files that fail comparison need to be comparison need to be examinedexaminedRoot directory of results Root directory of results directory structure is unique directory structure is unique so that results of multiple so that results of multiple test runs can be capturedtest runs can be captured

An Architected Solution:An Architected Solution: Managing Results: Database Model Managing Results: Database Model

Automation tool can write results directly to a database.Automation tool can write results directly to a database.

Queries and Reports can be designed to quickly show Queries and Reports can be designed to quickly show test statustest status

Could be used to populate a web page with up-to-the-Could be used to populate a web page with up-to-the-minute test results.minute test results.

An Architected Solution: An Architected Solution: Pre-ScriptingPre-Scripting

With simplified test With simplified test case entry, all fields case entry, all fields and scripted and scripted functions can be functions can be agreed upon in agreed upon in advanceadvance

Test cases can be set Test cases can be set up before functions up before functions are writtenare written

Selling the Architected SolutionSelling the Architected Solution

Management is under pressure to get results Management is under pressure to get results quickly, but good automation architecture quickly, but good automation architecture requires planning and time for infrastructurerequires planning and time for infrastructure

New projects: Balance the need for quick New projects: Balance the need for quick results with the need for a maintainable results with the need for a maintainable architecturearchitecture

Existing projects: Architecture can be built in Existing projects: Architecture can be built in as automation for particular areas of the as automation for particular areas of the product needs revisedproduct needs revised

Selling the Architected Solution:Selling the Architected Solution:BenefitsBenefits

Long-term cost and risk reduction since a modular approach Long-term cost and risk reduction since a modular approach minimizes maintenance, and a lot of the infrastructure is minimizes maintenance, and a lot of the infrastructure is reusable between projects and productsreusable between projects and productsConsistent levels of automated testing across projectsConsistent levels of automated testing across projectsProvides more defined career paths for QA staffProvides more defined career paths for QA staffIncreased employee satisfaction and productivity since an Increased employee satisfaction and productivity since an employee can focus on areas of interestemployee can focus on areas of interestAnyone with product knowledge can begin entering test cases Anyone with product knowledge can begin entering test cases quicklyquicklyAutomation comes on line more quickly since manual testing Automation comes on line more quickly since manual testing is done simultaneouslyis done simultaneouslyReadily available results save time for development as well as Readily available results save time for development as well as QAQA

Selling the Architected Solution:Selling the Architected Solution:Pitfalls to AvoidPitfalls to Avoid

Include all necessary people in planning Include all necessary people in planning (e.g. project manager, product manager, (e.g. project manager, product manager, testing lead, system architect). Each role testing lead, system architect). Each role may have information that can be designed may have information that can be designed into the initial architecture instead of added into the initial architecture instead of added laterlaterAvoid the temptation to shortcut the processAvoid the temptation to shortcut the processTry to have some idea of what comes after Try to have some idea of what comes after the current projectthe current project

Selling the Architected Solution:Selling the Architected Solution:Pitfalls to Avoid IIPitfalls to Avoid II

Do not over-encapsulateDo not over-encapsulate

ConclusionConclusion

Properly planned and implemented Properly planned and implemented automated testing automated testing willwill reduce project cost reduce project cost and risk in the long termand risk in the long term

Good planning and management support are Good planning and management support are imperative for maintainable, lasting imperative for maintainable, lasting automationautomation

Questions??Questions??