automation through the back door

W3

Test Automation

5/7/2014 11:30:00 AM

Automation through the Back

Door

Presented by:

Seretta Gamba

Steria Mummert ISS GmbH

Brought to you by:

340 Corporate Way, Suite 300, Orange Park, FL 32073

888-268-8770 ∙ 904-278-0524 ∙ [email protected] ∙ www.sqe.com

Seretta Gamba


Seretta Gamba has more than thirty years of experience in software development. As test manager at Steria Mummert ISS GmbH, Seretta was charged with improving the test automation process. After studying the current strategies, she developed a style of keyword-driven testing and a framework to support it. In 2009, the framework was extended to support manual testing. Speaking about this at EuroSTAR, Seretta got the attention of Dorothy Graham who subsequently invited her to contribute a chapter to Dot’s book Experiences of Test Automation. After reading the entire book, Seretta noticed recurring patterns in solving automation problems and is currently intent on cataloguing test automation patterns.

26.04.2014

1

� www.steria-mummert-iss.de

W3 - Automation Through the Back Door

Seretta Gamba



Steria Mummert ISS Presentation.pptx

© Steria Mummert ISS GmbH

Agenda

26.04.2014 2

�The Problem

�The “Back Door” Idea

�The Implementation

�Conclusion and general

discussion

26.04.2014

2




The Problem

26.04.2014 3

�The company

�The application

�The developer team




The Company

26.04.2014 4

Steria Mummert Consulting is one of

the top ten IT companies in Germany

(Counseling, IT-Service). In Germany

and Austria about 1700 employees.

Mummert ISS GmbH was established 2002 as

a 100% subsidiary of Mummert Consulting.

Develops standard software for insurance

companies. About 70 employees.

The Steria Group is one of the

top ten IT-Service companies in

Europe with worldwide about

20000 employees.

26.04.2014

3




The Application

26.04.2014 5

The product is:

� a complete software solution for insurance companies

� a standard software, that enables flexible management of

the products and processes of an insurance company -

both partner (customer) and portfolio can be easily

maintained

System design:

�Completely newly developed standard solution

�Customer oriented application structure




The Developer Team

26.04.2014 6

Test

Automation

Team

Product

Team

New Customer

New Customer

New Customer

X

Old Customer

Old Customer

Old Customer

26.04.2014

4




The „Back Door“ Idea

26.04.2014 7





26.04.2014 8

�What do we need in order to be able to

automate?

� Knowledge of the application under test

� Test specifications

� Automation strategy

� Test Tools

26.04.2014

5





26.04.2014 9

�What can manual testing deliver?




� Test Tools





26.04.2014 10


automate?




� Test Tools

26.04.2014

6




Our Automation Strategy

26.04.2014 11

�Our Command-Driven framework supports the test

automation team by (mostly) eliminating the need to

program new keywords and by automating some of the

steps

Prerequisites:

Test cases Registration of GUI ControlsCommand Library





26.04.2014 12

�Step 1

Capture

Tool Script

�Step 2

Tool Script

ISS

ISS Test Station

Driver

Data-Template

�Step 3

DriverData Suite

Preparation:

26.04.2014

7





26.04.2014 13

�Execution

Test suite Capture-Replay Tool Test results





26.04.2014 14


automate?

�Can manual testing deliver it?

�Can we convince our colleagues to switch over to the

“automation” framework for manual testing?

�Can we extract and use the collected Information for

automation?

��

��

26.04.2014

8




Supporting manual testing

26.04.2014 15

In order to support manual testing with the automation

framework, we had to find out how testers worked and

devise better support.

The original testing procedure was based on excel-test-

sheets, that would be performed again and again for

regression testing.




Weaknesses of manual testing

26.04.2014 16

Test casemaintenance

• Corrections or improvements to the test template can get lost

• Test-specifications are very “concise”

Defectreporting

• Cumbersome defect reporting

• Test cases tend to be repeated in order to report the defects exactly

Summary reporting

• No automatic statistic of the executed test cases versus the fix status of the found defects

26.04.2014

9




Supporting manual testing

26.04.2014 17

Many of the available features in the

automation framework could, with no

change, be employed to support manual

testing.

Add or change test cases.

Overview of the available test cases.

Logging of test results.

Result reports or overviews.

Prioritization of the test cases.

Defect reporting.

Ava

ila

ble

fe

atu

res

Corrections or

improvements to the

test template can get

lost.

No automatic statistic of

the executed test cases

versus the fix status of

defects found.

Cumbersome defect

reporting.

Test cases tend to be

repeated in order to report

the defects exactly.




Needed for supporting manual testing

26.04.2014 18

Support for manual test execution

• Test driven by sequential display of the selected test cases

• Paperless execution

• Support in:- Performing tests.- Maintaining test cases.- Creating defect items.

Import test-template sheets

• Import from the current excel sheets (csv-files) to the test suites in the framework format

• Import functionality driven by an external table.

26.04.2014

10




Needed for supporting manual testing

26.04.2014 19

Detail test execution for developers

• Capture-facility integrated into the framework.

• Generation of a capture-replay script with simultaneous recording.

• Starts or stops by pushing a button.

• Recording automatically attached to defect tracking item

Save application status to file

• Integration of SQL-scripts to extract data from the database.




The Implementation

26.04.2014 20

�Manual testing with the automation

framework

�Automating the manual tests

26.04.2014

11




Manual test execution using the test automation

framework

26.04.2014 21

Set preconditions

and export them to file

Start recording

Perform test case

Export post-

conditions to file

Check

test

case

display

test caseOK

Defect

Start test suite




„Harvesting“ Manual Testing

26.04.2014 22

Test cases

After the testers started working with the framework,

creating more automated tests has been much more

rewarding because some steps have already been

performed

Tool script

26.04.2014

12




Automating the manual tests

26.04.2014 23

Step 1

• Call the framework functionality to transform the captured tool-scripts into Command-Driven scripts

Step 2

• Rework the command-Driven scripts

Step 3

• Create the new test procedures

Step 4

• Insert the test procedures to perform the test cases in the test suite structures




Conclusion

26.04.2014 24

�Short summary of our experiences

�Next steps

�General discussion

26.04.2014

13




Short Summary of our experiences

26.04.2014 25

After some time to allow the testers to get acquainted

with the new process, we started to get the first results.

� The testers had a hard time getting used to the new

formats.

� The import facility, test-sheet �� test suite was well

accepted from the very beginning.

� The recording feature was very well received by both

testers and developers.

� In the beginning the testers were recording either too

much or too little.




Next Steps

26.04.2014 26

<and they lived happily ever after. Well, we still have lots

of effort and work to do.

� Partially automated

tests.

� Sophisticated metrics.

� Integration with test

planning and

management tools .

26.04.2014

14




Conclusion

26.04.2014 27

"If you can't get in the front

door, try the back door".

We couldn't do automation

straight on, but found a way

to do it indirectly by

supporting the testers

where they were.

automation through the back door

Technology