dependence analysis in reduction of requirement based test suites boris vaysburg luay tahat bogdan...

Dependence Analysis in Reduction of Requirement Based Test Suites

Boris Vaysburg Luay Tahat Bogdan Korel

Computer Science Department Bell Labs Innovations Computer Science Department Illinois Institute of Technology Lucent Technologies Illinois Institute of Technology Chicago, IL USA Naperville, IL USA Chicago, IL USA

Outline

Model-based testing Selective model-based testing Selective test suite reduction using EFSM analysis EFSM dependencies Test Reduction based on EFSM analysis

Static approach Dynamic approach

Conclusions

Model Based Testing

Model based testing is used to test software systems based on a system model

Test cases are generated from the system model

Testing state-based software systems

Model Based Testing

Systems can be modeled using:

EFSM: Extended Finite State Machine SDL: Specification Description Language

Extended Finite State MachineEFSM

EFSM consists of: States Transitions

The following elements are associated with each transition: Event Enabling condition Action(s)

EFSM Transition

State 1 State 2Event(p)[Condition]/Action

EFSM Model of the ATM System

Start S1Card(pin, b)/

Prompt for PIN;attempts = 0

S2PIN(p)

[p == pin]/Display menu

Deposit(d)/b = b + d

Withdrawal(w)/b = b - w

Balance/print(b)

S3

Continue/Display menu

PIN(p)[(p != pin) and (attempts == 3)]/

Display error;Eject card;

Exit/Eject card

PIN(p)[(p != pin) and (attempts < 3)]/

Display error;attempts = attempts+1;

Prompt for PIN;

Exit

T2

T1

T3

T4

T5

T6

T7

T8

T9

Model Based Testing Strategies

State coverage Transition coverage Path coverage Constrained path coverage: each transition can be

traversed at most “n” times in a test

Model-Based Testing

These strategies are used to test the whole system

Model-based testing strategies may generate large test suites

Selective Model-Based Testing

Testers are frequently interested in testing only selected functionality (requirement)

In a system model requirements frequently correspond to transitions

The problem of selective testing of a requirement can be converted to a problem of testing a transition in the model (selective testing of a transition)




S2PIN(p)




Balance/print(b)

S3




Exit/Eject card



Prompt for PIN;

Exit

T2

T1

T3

T4

T5

T6

T7

T8

T9

Selective Testing Strategies with respect to a transition

State coverage Transition coverage Path coverage Constrained path coverage

Each test must contain a transition (requirement) under test


The size of a selective model-based test suite is frequently significantly smaller than the size of a complete system test suite

However, the size of selective test suites still may be large

Reduction of Selective Test Suites

The goal is to develop an approach of reduction of selective test suites

EFSM dependence analysis is used to reduce the size of selective test suites

Test Suite Reduction

Two tests are considered to be “equivalent” with respect to a transition under test if, during traversal of the EFSM model, these tests exhibit the same “pattern” of interactions with respect to the transition under test.


Interactions?

Dependencies between EFSM elements.

EFSM dependence analysis

Dependencies between transitions (“active” elements of EFSM)

Two types of dependencies between transitions in the EFSM are identified: Data dependence Control dependence

Data Dependence

Data dependence captures the notion that one transition defines a value to a variable and another transition may potentially use this value.

Data Dependence



S2PIN(p)




Balance/print(b)

S3




Exit/Eject card



Prompt for PIN;

Exit

T2

T1

T3

T4

T5

T6

T7

T8

T9

Control Dependence

Control dependence captures the notion that one transition may affect traversal of another transition

It is defined based on the concept of post-dominance

Control Dependence



S2PIN(p)




Balance/print(b)

S3




Exit/Eject card



Prompt for PIN;

Exit

T2

T1

T3

T4

T5

T6

T7

T8

T9

Control Dependence

Start S1 S2 S3

Exit

T2

T1

T3

T4

T5

T6

T7

T8

T9

EFSM Dependence Graph

Nodes represent transitions Arcs represent control and data dependencies

Static EFSM Dependence Graph

T5

T6

T8T1

T4

T9

T2

T3

T7data dependence

control dependence


Two tests are considered to be “equivalent” with respect to a transition under test if, during traversal of the EFSM model, these tests exhibit the same “pattern” of interactions with respect to the transition under test.

Test suite reduction

Interactions: Data dependencies Control dependencies

Pattern of interactions: Dependence sub-graph

Static interaction pattern Dynamic interaction pattern



S2PIN(p)




Balance/print(b)

S3




Exit/Eject card



Prompt for PIN;

Exit

T2

T1

T3

T4

T5

T6

T7

T8

T9

Test #1: T1, T4, T6, T8, T6, T8, T5, T8, T9

Test #2: T1, T2, T4, T6, T8, T6, T8, T5, T8, T9

Selective test suite reductionStatic Interaction Pattern

All dependencies that occurred during traversal of the test (path) are mapped to the EFSM dependence sub-graph

Dependencies that do not “influence” a transition under test are removed

The resulting dependence sub-graph is called Static Interaction Pattern

If two tests have the same Static Interaction Pattern with respect to a transition under test, one of these tests is removed from the test suite



S2PIN(p)




Balance/print(b)

S3




Exit/Eject card



Prompt for PIN;

Exit

T2

T1

T3

T4

T5

T6

T7

T8

T9

T1 T4 T6 T8 T6 T8 T5 T8 T9

Test #1:

Dependence Sub-graph

T5

T6

T8T1

T4

T9

Static Interaction Pattern for Test #1

T5

T6

T1

T4



S2PIN(p)




Balance/print(b)

S3




Exit/Eject card



Prompt for PIN;

Exit

T2

T1

T3

T4

T5

T6

T7

T8

T9

T1 T2 T4 T6 T8 T6 T8 T5 T8 T9

Test #2:

Dependence Sub-graph

T5

T6

T8T1

T4

T9

T2

Static Interaction Pattern for Test #2

T5

T6

T1

T4

Test #1 and Test #2 have the same Static Interaction Patterns

Test suite reduction

Significant test reduction can be achieved using Static Interaction Patterns

Test Suite Size Reduction

nSize of Selective

Test Suite

Size of Reduced Test Suite Using Static Interaction Pattern

2 18 3

3 100 7

4 450 14

5 1,806 16

6 6,762 16

7 24,300 16

Selective test suite reduction

Test reduction using Static Interaction Patterns is relatively “coarse”

Static Interaction Patterns are not able to capture intricate interactions

Dynamic Interaction Pattern is used

Dynamic EFSM dependencies

Dynamic dependencies are dependencies that occur during traversal of the test (path)

Dynamic EFSM dependence graph

T1 T2 T4 T6 T8 T6 T8 T5 T8 T9

Dynamic EFSM Dependencies for Test #2

Dynamic EFSM Dependence Graph

Selective test suite reductionDynamic Interaction Pattern

All dependencies that occurred during traversal of the test (path) are used to create a dynamic EFSM dependence graph.

Dependencies that do not “influence” a transition under test are removed.

The resulting dependence sub-graph is called Dynamic Interaction Pattern.

If two tests have the same Dynamic Interaction Pattern with respect to a transition under test, one of these tests is removed from the test suite.

T1 T2 T4 T6 T8 T6 T8 T5 T8 T9

Dynamic EFSM Dependence graph for Test #2

T1 T2 T4 T6 T8 T6 T8 T5 T8 T9

Dynamic Interaction Pattern for Test #2

If two tests have the same Dynamic Interaction Pattern with respect to a transition under test, one of these tests is removed from the test suite

Selective Test Suite Size Reductions

nSize of Selective

Test SuiteSize of Reduced Test

Suite Using Static Interaction Pattern

Size of Reduced Test Suite Using Dynamic

Interaction Pattern

2 18 3 3

3 100 7 7

4 450 14 15

5 1,806 16 31

6 6,762 16 63

7 24,300 16 127

Conclusions

Using EFSM dependence analysis to reduce selective test suites

Significant reduction may be achieved Fault detection capability of the reduced test

suite? The approach is under development We plan to perform an experimental study Develop efficient algorithms of test suite

reduction

End of Presentation

dependence analysis in reduction of requirement based test suites boris vaysburg luay tahat bogdan...

Documents

test slide

efsm model

model selective testing

transition slide

model based testing

selective testing strategies

system model test cases

efsm transition state