automated testing whitepaper bp functional testing

8/14/2019 Automated Testing Whitepaper BP Functional Testing

1/12

Best practices for implementing automatedunctional testing solutions

White paper


2/12

Table of contentsIntroduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Functional testing versus unit testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Why automate functional testing processes? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

The benefits of automated functional testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

How to identify candidates for automated testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

How to determine the ROI of automated testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Evaluating automated testing software: what to look for . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

HP Quality Center software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

HP Functional Testing software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

HP Business Process Testing software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Checklist: Five keys to successful automated testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .


3/12

IntroductionThere is no question that rigorous functional testing iscritical to successful application development.

The challenge for your developers, QA teams, andmanagement alike is how to speed up testingprocesses and increase accuracy and completenesswithout breaking your already tight budgets.

By automating key elements of functional testing, yourcompany can meet aggressive release schedules, testmore thoroughly and reliably, verify that businessprocesses function correctly, and ultimately generatehigher revenue and customer satisfaction from youronline operations.

Yet the prospect of automating functional testing raisesnew concerns:

What are the costs of automating the testingprocesses and what is the ROI?

Which applications/processes are candidates forautomated testing and which are not?

Will new training be required and how will thatimpact current development project schedules?

What is the proper methodology for automating atesting effort?

What is involved in maintaining automated tests?

When comparing test automation products, which

features are most important?

These and other issues need to be fully examined andunderstood before test automation projects areinitiated.

Functional testing vs. unit testingFunctional testing provides the ability to verify thatapplications work as they shouldthat they do wh

users expect them to do. Functional tests capture usrequirements in a useful way, give both users anddevelopers confidence that the business processesmeet those requirements, and enable QA teams toverify that the software is ready for release.

Functional testing is complementary tobut quitedifferent fromunit testing. Simply put, unit testsindicate whether the code is doing things right;functional tests tell whether the completed applicatis working correctly and providing the properfunctionality. Unit tests are written from the codedevelopers perspective, while functional tests are

written from the end users and business-processperspective.

Abstract: Todays enterprises need to conduct thoroughfunctional testing of their mission-critical applications toverify that all business processes work as expected. Byimplementing automated functional testing, yourcompany can dramatically increase both the speedand accuracy of your testing processes, providing ahigher return on investment (ROI) from softwareprojects while dramatically cutting risk.

This paper presents a brief overview of the advantand challenges of automating functional testing to structure your thinking about how best to approachautomation. It also summarizes the key features anbenefits of HPs automated functional testingsoftwareHP QuickTest Professional software andHP Service Test softwaresome of the industrys mpopular and powerful automated software testingsolutions. The paper concludes with a brief overvieHP Business Process Testing software and presents checklist of the five keys to successful automatedtesting.


4/12


5/12

Deliver higher test accuracy and find more defectsearlier. Test automation gives your developers aneasier way to replicate and document softwaredefects, helping expedite development processes

while verifying correct functionality across allenvironments, data sets, and business processes.

Provide formalized processes. The introduction ofautomated testing encourages your test teams toformalize their processes, resulting in higher testconsistency and better documentation records.

Facilitate the re-use of tests. Once tests are scripted,your developers can use, re-use, and add to the testsuite as they make changes to their applications.There is no need to re-create scripts to test the samefunctionality for each application modification.

Support ERP/CRM implementations.As morecompanies are implementing ERP/CRM solutions, theneed to do end-to-end regression testing is becomingmore frequent and more crucial.

How to identify candidates forautomated testingIn general, it makes sense to focus automation effortson critical business processes, complex applications,and the use cases that comprise them (as opposed tolower-level tasks, such as system-level verifications). Butif your enterprise has multiple software testers

spending many hours per day and still has quality andfunctional problems, you could definitely benefit frommoving to automated testing.

The decision of whether or not to automate the testprocess should be driven by ROI considerations. Ingeneral, there will be a positive return if theapplication requires regression testing, requires

multiple builds/patches/fixes, needs to be tested onumerous hardware or software configurations, ansupports many concurrent users. In addition, ifrepetitive tasks such as data loading and system

configuration are involved, or if the application neto meet a specific service-level agreement (SLA),automation will certainly make economic sense.

Automation does not make sense when testing theusability of a user interface (UI) or for exploratorytesting or applications that are not yet mature.

How to determine the ROI ofautomated testingThe ROI for any investment can be obtained from asimple calculation:

ROI = Net present value of the investment

Total initial cost

When it comes to automating functional testingprocesses, the costs are tangible but the net presenvalue also includes many intangible factors. The beapproach is to determine with as much precision apossible what the hard-dollar costs are and thencompare them to the hard- and soft-dollar benefits automating the test efforts.

Hard-dollar costs to consider in an ROI calculationinclude:

Acquisition costs the cost of initially acquiring ttest automation software.

Hardware coststhe cost of the hardware requirfor functional testing. Typically, no special-purposhardware is needed for functional testing, providthe tester has access to a standard desktop compor workstation with an Ethernet port.


6/12

Labor costsfactor in the cost of trained personnel toscript test cases or build manual tests, including thecosts of recruiting, hiring, paying, and retainingtrained experts.

Training costsdepending on which test automationproduct is selected, the user may need considerabletraining to become proficient at writing scripts forautomated testing. Alternatively, companies mayelect to hire professional services firms to handle theinitial automated test creation.

When weighing the potential rewards of automation, itis important to consider intangible benefits, such ashigher morale and job satisfaction among testers,improved customer satisfaction and loyalty, and anenhanced reputation for reliable software among end

users.

Evaluating automated testingsoftware: what to look forMany vendors offer test automation products. Everysolution has its own strengths and weaknesses, uniquefeatures, and market niche. Each enterprises specificrequirements will determine which option is the best fit.However, there are several key capabilities that shouldbe included in any test automation product:

Scriptless representation of automated tests. Thetesting product should offer a point-and-clickinterface for accessing and interacting with theapplication components under testas opposed topresenting line after line of scripting. Testers shouldbe able to visualize each step of the business processand view and edit test cases intuitively; this willshorten the learning curve for testers and help QAteams meet urgent deadlines.

Integrated data tables. One of the key benefits oautomating functional testing is the ability to pumlarge volumes of data through the system quicklyit is also important to be able to manipulate the

sets, perform calculations, and quickly createhundreds of test iterations and permutations withminimal effort. Your enterprise should look forproducts that offer integrated spreadsheets withpowerful calculation capabilities.

Clear, concise reporting. The ability to run a highvolume of tests is of little benefit if the results of thtests are not easy to understand or interpret. Testproducts should automatically generate reports thdisplay all aspects of the test run and explain theresults in an easy-to-read format. The reports shoprovide specifics about where application failure

occurred and what test data was used; presentapplication screen shots for every step to highligany discrepancies; and provide detailedexplanations of each checkpoint pass and failureshould also be possible to share these reports acan entire QA and development team with nomodifications.


7/12

HP Quality Center softwareHP Quality Center software provides a complete, Web-based, integrated system for performing QA across a

wide range of IT and application environments. Itincludes an integrated suite of role-based applicationsand best practices, as well as an open, scalable andextensible foundationall designed to optimize andautomate key quality activities. HP Quality Centersoftware components include solutions forrequirements, test, defects management, andERP/service-oriented architecture (SOA); functional,regression, and service testing; and business-processdesign and validation.

HP Functional Testing software

The full HP Functional Testing software solutionincludes both HP QuickTest Professional software andHP Service Test software, which are sold as separatepackages. Together they allow your enterprise toautomate functional tests, enabling your IT teams toquickly identify defects and verify that applicationswork correctly prior to going live.

Visibility into application readinessHP QuickTest Professional software is our advancedautomated testing solution for building functional andregression test suites. Its innovative, zero-configuration,keyword-driven approach to structured automation

provides the ability to use natural language to buildtests that verify user interactions and determine thatbusiness processes work as designed. It is one of theindustrys best solutions for functional and regressiontest automation for practically every softwareapplication and environment. It enables your testers tocapture, verify, and replay user interactionsautomatically to identify defects.

HP QuickTest Professional (QTP) software ActiveSctechnology offers one of the industrys shortest learcurves for testers. QTP satisfies the testing needs oboth technical experts and business analysts, enab

your company to deploy higher-quality applicationfaster, with reduced risks and cost.

HP QuickTest Professional version 9.5 softwareprovides support for additional environments andincludes the following core enhancements:

Process guidance provides the ability to embed auser guide within QTP and document and propabest practices as part of the test developmentprocess, with interactive help to walk users througthe activities and link the user to the right applicaareas within QTP. We ship QTP with two built-in

processeskeyword-driven processes and businesprocess testingand you can create your owncustom processes and propagate those as well.

Maintenance Run Mode is designed to address tproduction phase where tests need to be maintaas the application changes. This new executionmode allows you to run the tests against theapplication, and instead of its failing where theapplication changes, you get a pop-up wizard thguides you on how to change the test on the fly, reason for failure, and choices on how to fix it. Ifchanges are more extensive than what can be fix

on the fly, you can add comments and go back tlater.

gure 2: HP Quality Centeroftware components

ua y en er

HP Quality Center dashboard

Foundation

Center Management

Test Director for Quality Center

Functional Testing SAP Security Testing SOA Testing

RequirementManagement

Shared data repository Central administration Workflows Open APIs

QuickTestProfessional BusinessProcessTesting (BPT)

BPT for SAP ChangeImpactTesting

HP QAInspect Service Test Service TestManagement

Risk-based TestManagement

Release Management Defect Management QA Lab Management(add on)


8/12

New QuickTest panes show the list of actions andflows in a test within one flow pane; the resourcepane allows test assets and resources used by the testto be easily viewable and manageable; and a

keyword pane shows the keywords used by the testin one view, so you can drag and drop test objectsinto the test and share keywords.

Checkpoint Management allows you to manage theobject repository and checkpoints within the objectrepository, and it allows shared output using a singleUI to view the shared object repository.

Web Extensibility is a simple applicationprogramming interface (API) to extend the capabilitiesof QTP and support Web controls that are notsupported out of the box. Ajax is an example of anon-standardized technology. Web Extensibility is thebest way to address Ajax and to create support forthose Web controls.

All-in-one installation is now provided on one DVDfor silent installation via Microsoft Installer (MSI).

Extended functionality is available in BusinessProcess Testing for learning SAP business processes,componentizing the test into business processcomponents, and detecting application changes.

The enhancements listed above complement thefunctionality available in previous versions of QTP.

QTP provides support for HP Business Process Testingsoftware to create keyword-driven, component-basedtests, allowing subject-matter experts andHP QuickTest Professional software engineers to worktogether.

Testers can create and use Multiple ApplicationAreas for each part of the application.

A Function Definition Generator quickly documentsand registers new user-defined functions for tests andcomponents.

QTP provides an Open XML report format for tesresults.

It provides the ability to pass data in tests andcomponents using parameters.

An Object Repository Comparison Tool and ObjRepository automation are included for easiermanagement.

Navigate and Learn capabilities simplify objectrepository creation.

QTP provides support for the broadest range oftechnologies in the industry (see Figure 3).

An HP QuickTest Professional software script editool enables editing of multiple test scripts.

There is a single license mechanism for HP QuicProfessional software and add-ins.

HP Service Test software for functional testing of SservicesService-oriented architectures (SOAs) provide a setservices that can be shared and re-used acrossapplication development initiatives. SOA solutionsenable your organization to:

Coordinate services instead of supporting large,monolithic applications

Decouple applications from underlying infrastruc

Shift the orientation from technology to business

HP Service Test software provides a solution thatenables QA engineers to simplify the ongoing testof SOA services and reduce scripting throughautomation. Users can efficiently enable compliancwith corporate standards, even in organizations thuse multiple development tools. HP Service Testsoftware allows you to validate the applicationindependence of a GUI, allows you to begin testinearlier in the lifecycle, and reduces the cost of testi

gure 3. Environments supportedy HP QuickTest Professionaloftware

Classic

Custom C/S PowerBuilder Delphi PowerBuilder 9, 10, 10.5, 11

ERP/CRM PeopleSoft Windows Siebel 5, 6 GUI clients Oracle GUI forms

Common

Web-related environments IE, Netscape JDK, Java Foundation Classes, AWT ActiveX

Custom client/server Windows/Win32 Visual Basic Stingray IBM VisualAge Smalltalk

ERP/CRM

Oracle 11i JD Edwards Web client

Legacy 3270, 5250 emulators, VT100

Operating systems Windows 2000, 2003, XP, Vista, Vista (64-bit)

Emerging

ERP/CRM SAP Siebel 7.x and 8.x PeopleSoft 8.x Oracle 12i

.NET WPF from .NET 3.0 Winforms Webforms HTC/Viewlink

Delphi 8 .NET WinformsAjax-Web extensibilityWeb servicesMacromedia flexWirelessJava SWTCitrixVMware desktop supportMozilla Firefox 2.0, 3.0


9/12

Service Test allows you to expose headless servicesthrough contract inspection and to provide support fortesting from the unit level through complex businessprocesses. The scripts created can be used for both

functional and performance testing. HP Service Testsoftware enables organizations to conduct functionaltest and regression test automation, and it addressesevery major software application and environment. Thesolution satisfies the needs of both technical and non-technical users, and it enables your IT organization todeploy higher-quality services faster, cheaper, and withless risk.

IT can rely on HP Service Test software for functionaltesting and HP LoadRunner software for testing theperformance of services. IT has unique, shared needsfor both functional and load testing as well as specificneeds for each, and HP integrates HP Service Testsoftware and HP LoadRunner so QA can efficientlymanage functional and performance testing ofservices.

HP Business Process Testing softwareIn addition to our functional testing solutions justdescribed, HP Business Process Testing (BPT) softwareprovides higher-level, scripted automated functionaltesting capabilities to business analysts so that they toocan be part of the quality process. HP Business ProcessTesting software works hand-in-hand with HP QuickTestProfessional software to bring non-technical subject-matter experts into the quality process early and in ameaningful way. BPT allows you to start your testautomation even before the application is available. Itempowers your entire testing team to createsophisticated test suites with less training. And itreduces test cycle times and test maintenance costs.

HP Business Process Testing software reduces theoverhead for automated test maintenance andcombines test automation and documentation into single effort. With this solution, organizations are a

to measure the quality of application deliverables abstract business definitions defined within theHP Business Process Testing software framework.Subject matter experts can focus on creating high-ltest flows that mirror actual business processes, whquality engineers concentrate their efforts on areasenable automation.

HP Business Process Testing software also includes

A framework for building reusable, data-driven tcomponents

Support for manual, keyword-driven and scripted

components A Web-based repository for sharing tests and

components

Pure manual components and full manualHP Business Process Testing software execution

Permissions and access control enforcement

An easy automation infrastructure for HP BusinesProcess Testing software generation in HP QuickTProfessional software

An API for creating/managing business componas a technology preview

BPT also has extended capabilities for creating andupdating tests for SAP applications. The HP qualitymanagement solution for SAP applications providesrepeatable and risk-mitigated approach toimplementing, upgrading, or changing your SAPapplications. The solution is powered by HP QualitCenter, one of the leading quality management

Testing functions

ActiveScreenAnalogAuto-documentationCheckpoint managementCode editor and debuggerFunction generatorXML object repositoryInformation flow paneIntegrated data tableKeyword viewMaintenance run mode

Numerous checkpointsODBC and Excel connectivityWeb ExtensibilityAutomation object model

Open multiple function librariesProcess guidanceRapid object importRecord/ReplayRecovery ManagerRun wizardScript and function librariesScriptFusionShared object repositoryBusiness process testingVBScript

XML reportsEnvironment extensibilityResource pane

gure 4. HP QuickTest Professionalesting functions


10/12


11/12

Benefits of manual testing include:

Manual tests can provide fast access to testinginformation. Testing can be started before theapplication is fully stable, providing faster feedback

to the development team.

Manual tests provide the ability to performsimultaneous testing. Manual testing can be donesimultaneously with the development and thedebugging of automated scripts.

Manual test scripts are easy to create. These couldlater be converted to automated scripts.

Manual tests are easy to understand for all users, andthey can be provided to business users for testvalidation and to developers as requirementsdocuments or steps that lead to the discovery of bugs.

Checklist: five keys to successfulautomated testingEven when there is clear evidence that automating atesting effort is economically justified, it can be difficultto determine how best to approach the transition toautomated testing processes. This section outlines fivefundamental principles for implementing automatedsoftware testing processes:

1. Complete a test plan document. Understanding thegoals of the application to be tested is critical to the

success of any testing effort. This includes thoroughup-front planning to verify that test requirements areimplemented correctly. HP TestDirector softwareprovides the ability to manage both test cases andrequirements for all applications to be tested.

2.Subdivide the tests into test cases to be automatIt is probably impossible for your organization tautomate all aspects of a test plan. Automatedtesting should be focused around the complex a

critical business processes that map to functionawithin an application as designed per therequirements. Many organizations find that theyautomating nearly 60 percent of their total numof test cases, leaving 40 percent of tests to beconducted manually.

3.Create automated tests. Using HP QuickTestProfessional software, users can create tests withhaving to do any actual scripting. HP QuickTestProfessional software captures the business procfor the target application itself and allows users create test flows that can then be saved into

HP TestDirector software and managed via the W4.Do data-driven tests to expand test coverage. Us

the built-in data-table feature of HP QuickTestProfessional software, your testers have the abilicreate data-dependent tests that use specifickeywords stored in Excel spreadsheets to populafields in an application. This capability allows ytesters to drive massive volumes of test data throan application.

5.Add verifications to the tests. The actual pass or criteria for a test are then added to the HP QuicProfessional software tests. The criteria includeverifications of the front-end of the application, thmiddle tier, or the back-end database. Built-indatabase verification confirms the values stored the database and verifies transaction accuracy athe data integrity of records that have been updadeleted, or added.


12/12

SummaryFunctional testing need not be a time-consuming orexpensive proposition. By automating functionaltesting, your enterprise can take major steps forward inyour ability to improve automated business processes.Your development and QA teams can increase boththe speed and accuracy of the testing processes, andyour IT department can achieve a higher ROI fromsoftware projects while reducing risk.

By selecting HP QuickTest Professional software orHP Service Test software as your test automationplatform, your enterprise can take advantage of one ofthe most sophisticated yet easy-to-use functional testingtechnology solutions on the market.

To learn more about HP functional testing solutions, visit:www.hp.com/go/software Copyright 20062008 Hewlett-Packard Development Company, L.P. The information contained herein isubject to change without notice. The only warranties for HP products and services are set forth in the expressarranty statements accompanying such products and services. Nothing herein should be construed as constitutingn additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein.

tel is a trademark of Intel Corporation in the United States and other countries. Microsoft, Windows, WindowsT, and Windows Vista are U.S. registered trademarks of Microsoft Corporation. Oracle is a registeredademark of Oracle Corporation and/or its affiliates. Java is a U.S. trademark of Sun Microsystems, Inc.

AA1-1390ENW, July 2008

T h l f b b i

automated testing whitepaper bp functional testing

Documents