introduction to exploratory testing

Rail Signalling Solutions

Exploratory Testing

2 L905 Overview PresentationAuthor: Codrin Pruteanu

Introduction

The classic testing methods involve testers to create test cases and test scripts using a test basis such as requirements and specification documents (e.g. Doors), in a process known as ‘Scripted Testing’ (ST).

ST led to excessive testing of easy cases, while failing to allow sufficient time to creative testing of more interesting and complex cases

‘Exploratory testing’ (ET) is referred as a new approach

and was introduced by Cem Kaner


About Exploratory Testing

“A style of software testing that emphasizes the

personal freedom and responsibility of the individual

tester to continually optimize the quality of his work

by treating test-related learning, test design,

test execution, and test result interpretation”

(Kaner C., 1983)

Keys:- stimulates the cognitive engagement of the tester

- increases responsibility for managing time properly


What is Exploratory Testing (ET) ?

ET represents: a technique for designing and executing better tests simultaneous learning, test design and test execution adapting the tests as they are performed

ET does not specify: how the test activities should be planned, managed and tracked how the test environment should be set up risk evaluation for safety critical applications which tools should be used


Exploratory Testing: Pro and Con

ET – main advantages: encourages creativity, adaptability and intuition increases the chance of finding bugs in a shorter time period the tester is free to explore new areas as test ideas arise during test execution is useful when complete documentation is not always available is investigative rather than confirmative it’s more fun than ST

ET – disadvantages: depends heavily on the testing skills and domain knowledge of the tester when combined with ST, it involves a risk of redundant tests is not suitable for complex tests with a long execution time does not provide absolute assurance that the most important bugs have been found


Quantity

Qua

lity

ET

ST

Exploratory Testing: case study

Complete Testing = ST + ET


When to apply Exploratory Testing

When ET should be used ?

the test team includes experienced testers

validate the work of a tested component (explore the tested feature)

‘smoke test’ is desired before starting with large-scale test execution

isolate and investigate a particular defect

in an early iteration (product not ready for ST)

improve ST for safety critical applications

a new tester enters the team (learning phase)


Important Notes

ET - important notes !

Preparation, planning and management of test activities

cannot be conducted in an exploratory way!

Traditional techniques should be used instead: documenting test plans,

test strategies and test summary reports

ET is not Agile Testing ! AT is how the test activities are to be managed and tracked ET is a technique for designing and executing tests ET can be considered to be an agile approach to testing


Success stories

Success stories about companies which use ET:

1. Microsoft determine in a limited amount of time if an application works well

on a new version of Windows

addresses both application functionality and stability

2. Adobe Improve the quality of software testing for their main line products on the market

2. Phillips Testing new features in safety-critical life-sustaining medical devices

3. Thales IM product test (Vlaicu Constantinescu)

FEC test team (Mihai Niculita)


Questions about Exploratory Testing

Who Does Exploratory Testing? Any software tester knowingly or unknowingly does it !

What qualities do I need to posses to be able to perform ET? the only limit to perform exploratory testing is your imagination and creativity your ability to explore and create more test cases

Why do I need to perform ET if I already use ST ? ET is used to increase the probability of finding defects but doesn’t replace ST !

What is my job as exploratory tester? explore a component of the system think about what should be tested implement the appropriate tests write down test results


Conclusions

Conclusions:

1. Many testers are trapped in comfortable ways of doing testing: The style of testing is often promoted as “professional”

Sometimes the lack of experience may produce test results with poor coverage

Some relatively unskilled testers may produce inefficient tests

Quantity does not always means quality !

2. In order to achieve a higher product quality (less defects) a tester should: develop his creativity and maximize the value of test results

use both testing techniques: ST and ET !


The End

introduction to exploratory testing

Education

test cases

test design

test strategies

test ideas

test basis

test executionadapting

test environment

agile testing