archana & tushar building automation frameworks for web services testing

Copyright © 2013 Deloitte Consulting LLC. All rights reserved.0

Tutorial:Building Automation Frameworks for Web Services Testing


Important Terms

Business Case

Framework Architecture

Benefits

Customization

What's next?

Table of Contents


Important terms

SOA – Service Oriented Architecture: is the structure that defines how a

collection of software components communicate with each other to provide

application functionality as services.

Regression Testing: is the testing conducted after changes in the software /

application to ensure that new defects have not been introduced as a result of

the change.

Data Driven Framework: is an automation framework that comprises of re-

usable test logic and multiple data sets that can be used repeatedly to ensure

a wide coverage of variable inputs.

Batch File: is a script file (text file) containing a series of commands and is

executed through the command interpreter.

SOAP UI: is an open source web service testing application for SOA.


Key Features

Solution

SOAPUI tool using groovy scripting and build through ANT

Reporting Structure

Data Driven Framework

Batch Process

Execution

• Data Driven Framework:

reduces maintenance and

improves test coverage

• Batch Process Execution:

facilitates easy execution of

multiple test cases

• Reporting Structure:

enables easy comparison of

test results against base

responses. Concise HTML

reports are generated

showing pass/fail status for

each scenario


Evolution of the Framework

Version 1.0

• Automation of manual testing

• Base Response Comparison

• Test results Summary Report

Version 2.0

• Record Not Found Scenario

• Graphical User Interface

Version 3.0

• Data Mismatch Scenario

• Database response comparison


Framework Architecture v1.0

1.1

So

ap

Re

qu

es

t1.2 Response

1.3

1.4



2.2 Execute Record Not Found

Queries

2.1 Update Test case with Status Flag if Record Not Found Scenario occurs

2.3 Update

Test Data

sheet with new

Data

2.4

2.5

Up

da

te th

e E

xp

ec

ted

Ba

se

Re

sp

on

se



Data

Mismatch

Not MatchedMatched

3.1 Execute Queries for Data Mismatch Scenario on DB

3.3 Compare

DB results

with Request

parameters

3.4

Up

da

teR

eq

ue

st

Pa

ram

ete

rs

3.5 Update Expected Base Response

Query results

3.2


Benefits – Quantitative

0

200

400

600

800

1000

1200

1400

1600

1 2 3 4 5 6 7 8 1 2 3 1 2 3

Manual Effort (Actual)

Automation Effort (Actuals)

Hrs.

R1 (19) R2 (19+42) R3

(19+42+23)

#Cycles

Manual Effort

(Estimated)

Comparison of Automation vs. Manual per Regression cycle


Benefits Qualitative

Reusability

Reliability

Repeatable

Increased turn out time

• This framework can be used across multiple projects and domains.

• This suite correctly diagnosed 70% of all interfaces and

• Provided consistent results across 11 execution cycles and releases.

• Saved ~ 6000 hours in manual testing effort.


Framework (v3.0)

Customizable

SOAP UI Requests

Expected base response

Database environment

Reports

Record Not Found queries

Data Mismatch queries

Re- usable

Test data sheet format/framework

Outcome comparison tool

Reporting Structure

SOAP UI – Database Interaction

Data Mismatch Logic

Record Not Found logic

Customization


What's Next?

Proposed Version 4.0 features:

a. Test data creator - to facilitate the creation of appropriate test data for

the test scenarios to be executed; as the automation framework is

dependent on test data.

b. Better Graphical User Interface – Enhance the GUI to enable the user to

create predefined test suites which are capable of scheduling regression

tests.

About DeloitteDeloitte refers to one or more of Deloitte Touche Tohmatsu, a Swiss Verein, and its network of member firms, each of which is a legally separate and independent entity. Please see www.deloitte.com/about for a detailed description of the legal structure of Deloitte Touche Tohmatsu and its member firms. Please see www.deloitte.com/us/about for a detailed description of the legal structure of Deloitte LLP and its subsidiaries.

Copyright © 2013 Deloitte Development LLC. All rights reserved.Member of Deloitte Touche Tohmatsu

http://www.deloitte.com/about

http://www.deloitte.com/us/about

archana & tushar building automation frameworks for web services testing

Documents