an automated test strategy based on uml diagrams

62
An Automated Test Strategy An Automated Test Strategy Based on UML Diagrams Based on UML Diagrams F. Basanieri F. Basanieri *, A. Bertolino *, *, A. Bertolino *, E. Marchetti E. Marchetti *, A. Ribolini *, G. Lombardi **, G. Nucera *, A. Ribolini *, G. Lombardi **, G. Nucera ** ** *PISATEL LAB, IEI-CNR, Pisa, Italy *PISATEL LAB, IEI-CNR, Pisa, Italy ** ERICSSON LAB ITALY ** ERICSSON LAB ITALY PISATEL

Upload: zenia-chaney

Post on 01-Jan-2016

38 views

Category:

Documents


6 download

DESCRIPTION

PISATEL. An Automated Test Strategy Based on UML Diagrams. F. Basanieri *, A. Bertolino *, E. Marchetti *, A. Ribolini *, G. Lombardi **, G. Nucera ** *PISATEL LAB, IEI-CNR, Pisa, Italy ** ERICSSON LAB ITALY. PISATEL. PISATEL - PowerPoint PPT Presentation

TRANSCRIPT

An Automated Test Strategy Based An Automated Test Strategy Based on UML Diagramson UML Diagrams

F. BasanieriF. Basanieri *, A. Bertolino *, *, A. Bertolino *, E. MarchettiE. Marchetti *, A. Ribolini *, *, A. Ribolini *, G. Lombardi **, G. Nucera **G. Lombardi **, G. Nucera **

  *PISATEL LAB, IEI-CNR, Pisa, Italy*PISATEL LAB, IEI-CNR, Pisa, Italy

  ** ERICSSON LAB ITALY** ERICSSON LAB ITALY

PISATEL

Page 2

PISATEL_LAB

PISATEL

PISATELPisa Initiative on Software

Architectures for Telecommunications

The joint conduction by the academia in Pisa and ERI of strategic research projects on software for telecommunications, selected on an annual basis

Page 3

Pisatel Objectives

• Main areas of interest are:– software engineering methods and tools, – software architectures, distibuted systems, networks, real

time systems, and protocols.

• Currently the active projects are:– Integrating Wireless, Wireline and Internet Networks – Analysis and Development of Real Time Software – Quality And Validation of Software Architecture

More details:

http://www.iei.pi.cnr.it/ERI/

PISATEL

An Automated Test Strategy Based An Automated Test Strategy Based on UML Diagramson UML Diagrams

PISATEL

Page 5

Objectives• Use for test planning the same UML diagrams developed

(e.g., by Rational Rose) for design and analysis• Tool-supported generation and maintenance of an

updated and strategic test plan, as the design evolves.

PISATEL

Sequence Diagra...

Sequence Diagram: Desi... Sequence

Diagram: To ...

Sequence Diagram: CheckList...Sequence

Diagram: Kernel / Kernel_Sequ... User Model Design Crit ics To Do List

CheckListKernel

GEF UML Meta-Model Argo

LibrariesArgoUML

ArgoUML vers.0.7

Designer

(from Logical View)

DecisionModel

_decision : String

DecisionModel()GetDecisions()

(from Logical View)GoalModel

_goals : String

GoalModel()getGoals()

(from Logical View)

Decision

_name : String_priority : Integer = 0

Decision()getName()getPriority()

(from Logical View)

+contains

Designer

(from Logical Vi...

Goal

_name : String = "a"_priority : Integer = 3

Goal()getName()getPriority()

(from Logical View)

+contains

: Designer

: Decision...

: Decision : GoalModel

: Goal

1. DecisionModel( ) 1.1. Decision(name, priority)

2. GoalModel( ) 2.1. Goal(name, priority)

3. GetDecisions( ) 3.1. getName( )

3.2. getPriority( )

4. getGoals( ) 4.1. getName( )

4.2. getPriority( )

Test Case getDecisions()getName()/getPriority()

Opening a saved file_decisions

NamingTest Case getDecisions()getName()/getPriority()

Opening a saved file_decisions

Naming Test Case getDecisions()getName()/getPriority()

Opening a saved file_decisions

NamingTest Case

getDecisions()getName()/getPriority()

Opening a saved file_decisions

Naming

Test Case getDecisions()getName()/getPriority()

Opening a saved file_decisions

Naming

Test Case getDecisions()getName()/getPriority()

Opening a saved file_decisions

Naming

Test Case getDecisions()getName()/getPriority()

Opening a saved file_decisions

Naming

Page 6

(Realistic) Preconditions

It is plausible that ….• in early design phases many of the UCs are defined

at a high level and only few of them are properly refined.

• there is not yet the specification of every possible scenario, but only few SDs already exist that realize the defined UCs.

• different people or teams take part to the development, everyone familiar with a different component or functionality.

PISATEL

Page 7

Fundamental Guidelines

• FG1: use the UML diagrams developed during specification and design, without imposing to the UML designers any additional formalism, or ad-hoc effort specifically for testing purposes.

• FG2: test planning for integration testing must cope with high level diagrams, even though not yet completely specified, or refined

PISATEL

Page 8

Trade-off• UML is a notation and does not impose a specific process.• A test strategy needs to refer to a systematic and

documented design processWe want to impose minimal requirements:– Use Cases (UCs) organized into a hierarchy– Express relevant scenarios as Sequence Diagrams (SDs)

PISATEL

Development

Process

Sequence Diagra...

Sequence Diagram: Desi... Sequence

Diagram: To ...

Sequence Diagram: CheckList...Sequence

Diagram: Kernel / Kernel_Sequ... User Model Design Crit ics T o Do List

CheckListKernel

GEF UML Meta-Model Argo

LibrariesArgoUML

ArgoUML vers.0.7

Designer

(from Logical View)

Page 9

Requirements

• The reference documentation is derived by collecting together and automatically organizing the diagrams developed during the design phase without additional manual effort of formalization.

• Inconsistencies and incompleteness are highlighted (Design weaknesses)

PISATEL

Sequence

Diagram: Kernel

/

Kernel_Sequ... Sequence

Diagram: To ...

To Do List

Sequence

Diagra...

Sequence

Diagram: Desi...

User ModelDesign Critics

Kernel

Sequence Diagram:

CheckList...

CheckList

Argo

GEFUML Meta-Model

LibrariesArgoUML

ArgoUML vers.0.7

Designer

(from Logical View)

Page 10

Cow_Suite approach

The processing of the Cow_Suite approach starts by analysing the project description*, and proceeds in parallel with the design and analysis refinement steps.

[*: the internal description of the project model, given by Rational Rose in a .mdl file]

PISATEL

Cow_Suite consists in the junction of two main components: UIT (Use Interaction Test) and CoWTeSt (Cost Weighted Test Strategy)

Cow_Suite =Cowtest pluS UIT Environment

Page 11

Cow_Suite toolThe Cow_Suite tool consists of three elements working in

combination:

PISATEL

1. the Main Tree: a systematic support to decide a. how to distribute the available test effort between the many

potential test cases

b. how to prioritise among the test cases when a functional coverage is fixed.

2. the UIT Tree: automated generation of abstract test cases from the UML diagrams.

3. the Test specification: a systematic support to generate the test procedures and subsequently the test scripts.

Page 12

Cow_Suite Scheme

Test caseTest case

Test case

Test case

Test caseTest case

Test case

Test case

Test case

Test case

Test case

Test case

Test case

Test case

Test case

Test caseTest Test

Procedures Procedures

listlist

Strategy

StrategySele

ction

Selecti

on

Obj 1 Obj 2 Obj 3

Method1()

Method3()

Method2()

U I TU I T

SDs selection according to the strategy chosen

Automatic UIT application for Test

cases derivation

Test specific

ation

Interaction with the user for deriving executable test procedures

Sequence Diagra...

Sequence Diagram: Desi... Sequence

Diagram: To ...

Sequence Diagram: CheckList...Sequence

Diagram: Kernel / Kernel_Sequ... User Model Design Crit ics To Do List

CheckListKernel

GEF UML Meta-Model Argo

LibrariesArgoUML

ArgoUML vers.0.7

Designer

(from Logical View)

Sequence Diagra...

Sequence Diagram: Desi... Sequence

Diagram: To ...

Sequence Diagram: CheckList...Sequence

Diagram: Kernel / Kernel_Sequ... User Model Design Critics To Do List

CheckListKernel

GEF UML Meta-Model Argo

LibrariesArgoUML

ArgoUML vers.0.7

Designer

(from Logical View)

0.250.25 0.50

0.500.50

0.7 0.3

0.30.35 0.15

Cow_SuiteCow_Suite

PISATEL

Page 13

Main steps

• Derive a Reference Tree • Deduce the Critical Profile• Select the Test Strategy• Derive the Test Cases• Implement the Test Procedures

PISATEL

Page 14

1.1 Tree DerivationUse Cases (UCs) and Sequence Diagrams (SDs) of the .mdl project design realized with Rational Rose are automatically organized by Cow_Suite tool in a sort of hierarchical (pseudo-)tree, starting from the Main Use Case Diagram, UCD.

Sequence Diagra...

Sequence Diagram: Desi... Sequence

Diagram: To ...

Sequence Diagram: CheckList...Sequence

Diagram: Kernel / Kernel_Sequ... User Model Design Crit ics T o Do List

CheckListKernel

GEF UML Meta-Model Argo

LibrariesArgoUML

ArgoUML vers.0.7

Designer

(from Logical View)

Hierarchical TreeHierarchical Tree

The case study is ArgoUML, a tool supporting OO design with UML

PISATEL

Page 15

1.2 Tree Derivation

• Looking at the tree the Cow_Suite users can get a complete view of the status of the specification of the various functionalities;

• The tree represents an ordered and organized documentation, that is continuously updated with the latest changes and progressively completed

Hierarchical TreeHierarchical Tree

PISATEL

Page 16

2.1 Deduce the Critical Profile• Systems are naturally composed of different parts realizing one or more

functionalities, each with different importance for the overall system performance.

• The effort devoted to the test of different functionalities must be planned and scheduled considering their relative “importance” (risk-based testing, reliability guided testing, etc…).

• Following level by level the tree structure of the UCs and SDs, the strategy allows the people in charge of the development of one or more functionalities to simply annotate the corresponding node in the tree with a number (weight).

PISATEL

0.75?0.60?0.65?

0.65!!!Sequence

Diagram: To ...

To Do List

Page 17

2.2 Weighted tree derivation

The user annotates every node with a value, leaf weight, representing the relative “importance” of this node (be it a UC or a SD) with respect to the other nodes at the same level in the tree

Hierarchical TreeHierarchical Tree

PISATEL

Page 18

2.3 Weights managementThe weight value belongs to the [0,1] interval and must be assigned so that the sum of the weights associated to all the children of one node is equal to 1.

The weights are user modifiable as the development proceeds.

PISATEL

Page 19

Integration Testing vs Integration Stage

• To realize a complete system functionality, it is generally necessary to execute several actions realizing lower level functionalities

• A system functionality is realized by the different components interaction.

• Integration Testing: testing interactions between system components (processes, packages, subsystems… ) at an appropriate level of abstraction.

• Integration Stage: is useful because generally the functionalities are not specified at the same level of detail and with the same number of UCs and SDs

PISATEL

Page 20

2.4 Integration Stage Selection

• The first integration stage is represented by the main UC and the SDs (if any), which are children of this node (hence they are at level 2 of the tree).

• The i-th integration stage is represented by the UCs positioned in at i-th level of the tree and every SDs, children of these nodes, situated at i+1-th level.

NOTE: the SDs at level i+1 represent the interaction between the different components that realize the functionalities described in the UCs at i-th level.

PISATEL

Page 21

Example: 4th Integration Stage

The Cow_Suite user selects the integration stage (4) at which the test is conducted.

The nodes that are considered by Cow_Suite tool for the subsequent evaluations are all and only those belonging to the 4th integration stage plus all the tree leaves existing at higher levels (3rd, 2nd, 1st).

SDs at 5th tree level

PISATEL

Page 22

2.5 Final weights assignment• The weights assigned to each node are used to define a relative

importance factor, in terms of how risky is that node and how much effort should be put to test it.

• Considering every node, from the root down to the elements belonging to the integration stage considered, its final weight is computed as the product of all the nodes weights on the complete path from the root to this node.

• The final resulting weight is an element of discrimination for choosing amongst the tests to execute.

PISATEL

GEFUML Meta-Model

Libraries

ArgoUML

ArgoUML vers.0.7

Designer

(from Logical View)

0.60.40.3

FW=0.12

FW=0.6

Page 23

3. Cow_Test Strategy

Two different situations are considered in test cases planning:

1: Cowtest_ing with fixed number of tests

• A certain budget is available, which is translated in terms of number of tests, NT, to execute.

• In this case the tool select the most adequate selection of NT tests from among the (many) various test cases that could be potentially conceived.

PISATEL

Test Case

Test Case Test Case

Tes

t Cas

e

Test Case

Tes

t Cas

e

Test Case

Test C

ase

Test Case Test Case

Test C

ase

Test Case Test Case

Test Case Test Case

Test Case Test Case

Test Case

Test Case

Test Case

Test Case Test Case

Test Case Test Case

Test Case Test Case

Test Case

Test Case Test Case

Test Case Test Case

Test Case Test Case

Test Case Test Case

Test Case Test Case

Test Case Test Case

Test Case Test Case

Test Case Test Case

Test Case Test Case

Test Case Test Case

Test Case Test Case

Test Case Test Case

NTNT

Page 24

3. Cow_Test Strategy

2: Cowtest_ing with fixed functional coverage• A certain percentage of functionalities must be covered.

• In this case by the tool is possible to define in advance which are the functionalities to be covered and the minimum number of tests to

execute.

PISATEL

Functional Coverage = 90%

Sequence Diagra...

Sequence Diagram: Desi... Sequence

Diagram: To ...

Sequence Diagram: CheckList...Sequence

Diagram: Kernel / Kernel_Sequ... User Model Design Critics To Do List

CheckListKernel

GEF UML Meta-Model Argo

LibrariesArgoUML

ArgoUML vers.0.7

Designer

(from Logical View)

0.250.25 0.50

0.500.50

0.7 0.3

0.30.35 0.15

10

2 2

11

Minimum Number of tests to execute = 16

Page 25

3.1 Select the Test Strategy

Cowtest_ing with fixed number of tests.

The selection of the most suitable distribution of Test Procedures is developed on the basis of leaves weights.

PISATEL

Page 26

Cowtest_ing with fixed number of testsUsing the resulting final weight at the chosen integration stage, nw, for each SD the number of Test Procedures, NTtest,

can be easily derived

Int-Stage Leaves names Critical profile

2nd Stage/NTest

3rd Stage/NTest

4th Stage/NTest

5th Stage/NTest

1st Stage ArgoUML version 7

1

Libraries 0.5 0.5 250 0.5 0.5 0.5 2nd Stage ArgoUML 0.5 0.5 250 0.5 250 0.5 250 0.5 250 GEF 0.25 0.125 63 0.125 63 63 UML Meta-Model 0.25 0.125 63 0.125 63 63

3rd Stage

Argo 0.5 0.25 125 0.25 0.25 Kernel , 0.7 0.175 0.175 Kernel_SD 0.2 0.035 17 0.035 17 Kernel_NDchild 0.8 0.14 70 Checklist, 0.3 0.075 0.075

4th Stage

Checklist_SD 1 0.075 38 0.075 38 User Model 0.3 0.0525 UserModel_SD, 1 0.0525 26 Design Critics 0.35 0.0613 DesignCritics_SD 1 0.0613 31 To Do List 0.15 0.0263

5th Stage

ToDoList_SD 1 0.0263 13

PISATEL

Page 27

3.2 Select the Test Strategy

Cowtest_ing with fixed functional coverage.

The most critical system functionalities are

selected for reaching the fixed coverage.

Test Procedures are distributed accordingly.

PISATEL

Page 28

Cowtest_ing with fixed functional coverageConsidering the final weights, nw, of every node the tool can: • select the functional test cases to be run, by ordering in decreasing

manner the nw*100 values and summing them, starting from the heaviest one, until the fixed coverage, C, is reached.

• Calculate the minimum number of tests with respect to the fixed coverage percentage.

Leaves names 5th Stage

weights

70%coverage

nwf /NTest

80%coverage/

nwf /NTest t

90%coverage/

nwf /NTest

100%coverage/

nwf /NTest

ArgoUML 0.5 0.667 4 0.61 6 0.533 10 0.5 19

GEF 0.125 0.167 1 0.152 2 0.133 2 0.125 5

UML Meta-

Model

0.125 0.167 1 0.152 2 0.133 2 0.125 5

Checklist_SD 0.075 0.1 1 0.08 1 0.075 3

DesignCritics_SD 0.0613 0.065 1 0.0613 2

UserModel_SD 0.0525 0.056 1 0.0525 2

Kernel_SD 0.035 0.035 1

ToDoList_SD 0.0263 0.0263 1

PISATEL

Page 29

Notes

• To make a more practical prediction in terms of man/hours, or required budget for testing, it would be necessary to be able to estimate the cost of the different test cases.

• An alternative application of Cowtest is considering a certain budget, B, planned for the testing instead of the fixed number of test NT. In this case is possible to evaluate the “relative cost”, b, for the testing of every leaf of the fixed integration stage.

PISATEL

Page 30

Cow_Suite Scheme

Test caseTest case

Test case

Test case

Test caseTest case

Test case

Test case

Test case

Test case

Test case

Test case

Test case

Test case

Test case

Test case

Strategy

StrategySele

ction

Selecti

on

Obj 1 Obj 2 Obj 3

Method1()

Method3()

Method2()

U I TU I T

SDs selection according to the strategy chosen

Automatic UIT application for Test

cases derivation

Sequence Diagra...

Sequence Diagram: Desi... Sequence

Diagram: To ...

Sequence Diagram: CheckList...Sequence

Diagram: Kernel / Kernel_Sequ... User Model Design Crit ics To Do List

CheckListKernel

GEF UML Meta-Model Argo

LibrariesArgoUML

ArgoUML vers.0.7

Designer

(from Logical View)

Sequence Diagra...

Sequence Diagram: Desi... Sequence

Diagram: To ...

Sequence Diagram: CheckList...Sequence

Diagram: Kernel / Kernel_Sequ... User Model Design Critics To Do List

CheckListKernel

GEF UML Meta-Model Argo

LibrariesArgoUML

ArgoUML vers.0.7

Designer

(from Logical View)

0.250.25 0.50

0.500.50

0.7 0.3

0.30.35 0.15

Cow_SuiteCow_Suite

PISATEL

Page 31

For each fixed integration stage, a weighted subtree is derived according to the choosen test criterion. Only the relative SDs are considered for Test Case generation.

SDs selectionSDs selection

3.3 Sequence Diagrams Selection

PISATEL

Final weightN. Test Procedures

Page 32

4. Test Case Derivation

The tool uses the UIT method, Use Interaction Test, to derive the Test Cases.

The method is mainly based on the SDs (particularly objects, messages and parameters)

SDs SelectionSDs Selection

PISATEL

Page 33

4.1 Use Interaction Test methodology

• A method to systematically derive Integration Test suites from UML diagrams.

• Inspired at large by the Category Partition method: it defines classes of functional equivalence by selecting significant values (choices) of relevant categories (parameters or data)

• Incremental test strategy (over the Use Case organizational structure)

PISATEL

Page 34

4.1.1 Sequence Diagram analysis

: Designer

: Decision...

: Decision : GoalModel

: Goal

1. DecisionModel( ) 1.1. Decision(name, priority)

2. GoalModel( ) 2.1. Goal(name, priority)

3. GetDecisions( ) 3.1. getName( )

3.2. getPriority( )

4. getGoals( ) 4.1. getName( )

4.2. getPriority( )

Horizontal axis shows a set of objects, Test Units, that interact through messages.Those are system units separately testable (class testing) for exercising a possible use of system.Example:

DecisionModel, Decision, GoalModel, Goal.

PISATEL

Page 35

4.1.2 Sequence Diagram analysis

: Designer

: Decision...

: Decision : GoalModel

: Goal

1. DecisionModel( ) 1.1. Decision(name, priority)

2. GoalModel( ) 2.1. Goal(name, priority)

3. GetDecisions( ) 3.1. getName( )

3.2. getPriority( )

4. getGoals( ) 4.1. getName( )

4.2. getPriority( )

Vertical axis shows:• the exchanged messages, Interactions Categories, involved in object interactions;• their parameters and relevant inputs, Settings Categories.DecisionModel

Settings: _decisions

Interaction: DecisionModel() getDecisions()

Decision Interactions: Decision(name:String,priority:int)GetName(), GetPriority() ............

PISATEL

Page 36

Additional Information: Class DiagramComplementary analysis of the Class Diagram description (mainly used for searching the Settings Categories).

These are attributes (or a subset of them) of a class (and the corresponding SD’s object) that affect object interactions and are represented by input parameters used in messages or data structures.

PISATEL

DecisionModel

_decision : String

DecisionModel()GetDecisions()

(from Logical View)GoalModel

_goals : String

GoalModel()getGoals()

(from Logical View)

Decision

_name : String_priority : Integer = 0

Decision()getName()getPriority()

(from Logical View)

+contains

Designer

(from Logical Vi...

Goal

_name : String = "a"_priority : Integer = 3

Goal()getName()getPriority()

(from Logical View)

+contains

Page 37

4.1.3 Search of Message Sequences

: Designer

: Decision...

: Decision : GoalModel

: Goal

1. DecisionModel( ) 1.1. Decision(name, priority)

2. GoalModel( ) 2.1. Goal(name, priority)

3. GetDecisions( ) 3.1. getName( )

3.2. getPriority( )

4. getGoals( ) 4.1. getName( )

4.2. getPriority( )

MessageSequences: set of Messages (in temporal order), involved in a SD, used by objects to define and elaborate specific functionalities.

Example:

getDecisions()

getPriority()

PISATEL

Page 38

4.2 Test Case Derivation

For each traceable Message Sequence a Test Case is generated. It contains the list of all its Settings and Interactions Categories and their values.

Detailed Test Case Description

PISATEL

Page 39

Conditions

A Test Case (TC) can be associated to some feasibility conditions formally expressed (notes or specifications tab).

In this case TC is divided in different subcases.

PISATEL

Page 40

Number of Test Procedures

The final weight of every SDs is used to automatically derive the number of Test Procedures associated to each Test Case.

PISATEL

Number of associated Test Procedures

Page 41

Cow_Suite Scheme

Test caseTest case

Test case

Test case

Test caseTest case

Test case

Test case

Test case

Test case

Test case

Test case

Test case

Test case

Test case

Test caseTest Test

Procedures Procedures

listlist

Strategy

StrategySele

ction

Selecti

on

Obj 1 Obj 2 Obj 3

Method1()

Method3()

Method2()

U I TU I T

Test specific

ation

Interaction with the user for deriving executable test procedures

Sequence Diagra...

Sequence Diagram: Desi... Sequence

Diagram: To ...

Sequence Diagram: CheckList...Sequence

Diagram: Kernel / Kernel_Sequ... User Model Design Crit ics To Do List

CheckListKernel

GEF UML Meta-Model Argo

LibrariesArgoUML

ArgoUML vers.0.7

Designer

(from Logical View)

Sequence Diagra...

Sequence Diagram: Desi... Sequence

Diagram: To ...

Sequence Diagram: CheckList...Sequence

Diagram: Kernel / Kernel_Sequ... User Model Design Critics To Do List

CheckListKernel

GEF UML Meta-Model Argo

LibrariesArgoUML

ArgoUML vers.0.7

Designer

(from Logical View)

0.250.25 0.50

0.500.50

0.7 0.3

0.30.35 0.15

Cow_SuiteCow_Suite

PISATEL

Page 42

5.1 Test Procedures Specification

Test Specification:

For each identified category, we consider all its possible values and constraints (“choices”).

PISATEL

The user interacts with the tool for inserting Choices values.

Page 43

5.1.1 InteractionsCategories Specification

Interactions

Categories: objects interactions, exchanged messages.

The tool implements two possible choices for the user:

the default values,

the user values

PISATEL

Page 44

State Diagram

PISATEL

ACTIVE (Priority >0)

INACTIVE (Priority =0)

SNOOZED UNSNOOZED

set Priority to 0, beInactive(), Snooze()set Priority v>0, Wake()

set Priority to 0, beInactive(), RemoveItem

Snooze()

Snooze()

Elapsed Time, Wake()

Unsnooze()RecomputeAllToDoItems[ elapsed time ]

set Priority v>0

Unsnoozed(), Wake()

If a State Diagram is linked to a specific Test Unit (class), further information can be found about the relevant configurations of InteractionCategories

State diagram of the “Critic” class

Page 45

5.1.2 Specification of SettingsCategories

Settings

Categories:

parameters and inputs relevant for the Test Unit.

The tool implements two possible choices for the user:

the default values,

the user values.

PISATEL

Page 46

5.2 Test Procedures Implementation

A Test Procedure is automatically generated from a Test Specification, for each possible combination of choices of every category involved in a Messages Sequence.

PISATEL

Page 47

Test Procedures: an exampleTest Specification:

DecisionModel:

Settings:

_decisions

Naming

Storage

Modularity

Inheritance

Stereotypes

Relationships……..

Interactions:

getDecisions

Opening a new file

Opening a saved file

After a modification and before saving

After a modification and after saving

DecisionModel()Class constructor

Test Procedure

getDecisions()

getName()

Opening a saved file

_decisions

Naming

Note: Such a Test Case must be repeated for all possible values of _decisions

Page 48

Constraints on Choices

Some choices combinations could result contradictory or meaningless. Following the Category Partition methodology, constraints must be added to the SettingCategories choices.

Constraint are specified using: • Properties, that can be assigned to certain choices and testing

for by other choices.• IF Selector, is a conjunction of properties that were previously

assigned to other choices.

PISATEL

Page 49

Example

SettingsCategories:

pattern size:

empty [Property Empty]

single character [Property Not Empty]

many character [Property Not Empty]

quoting

pattern is quoted [Property quoted]

pattern is not quoted [if Not Empty]

pattern is improperly quoted [if Not Empty]

PISATEL

Page 50

SettingsCategories: Specification Constraints

Test Specification:

Decision(_name, _priority):

_name

Naming [if priority<5]

Class Selection [if priority=0]

Storage

Inheritance

Modularity…..

_priority

0 [Property priority=0]

1

2

3

7 [Property not priority<5]

PISATEL

Page 51

SettingsCategories: Specification Constraints

Test Procedure

DecisionModel()

Opening an existing document

Decision( _name, _priority)

_nameNaming [if priority<5]

_priority7 [Property not

priority<5]

Test Procedure

DecisionModel()

Opening an existing document

Decision( _name, _priority)

_name

Naming [if priority<5]

_priority

4

X

Page 52

5.2 Test Suite

The set of all generated Test Procedures is named Test Suite

PISATEL

Page 53

Note• The Test Procedures, as derived by the tool, are not directly executable

( Test Scripts). The derived Test Procedures have to be executed and logged by a test driver.

CowSuite goal is:– giving automatic and systematic support to construct a meaningful

Test Suite. – It is not to directly execute the tests.

Cow_Suite

MDL Analizer

Test TreeGenerator

Weights Manager

Test CaseGenerator

Test CaseBuilder

Test CaseChecker

REI

Rational Rose

.Tlb library

.mdl Files

Test Driver

Page 54

Remarks

• The degree of detail of the Test Suite depends directly on:– the design detail level– granularity of the inserted values (selection of

the constraints and choices of the UIT categories).

• Currently: the final result is a text document containing the Test Suite.

• In future: different format (e.g. XML), that can be accepted in input by the adopted test driver.

Page 55

Features under Construction

Two different situations can occur during the tree construction:

PISATEL

Sequence

Diagram: Kernel

/

Kernel_Sequ... Sequence

Diagram: To ...

To Do List

Sequence

Diagra...

Sequence

Diagram: Desi...

User ModelDesign Critics

Kernel

Sequence Diagram:

CheckList...

CheckList

Argo

GEFUML Meta-Model

Libraries

ArgoUML

ArgoUML vers.0.7

Designer

(from Logical View)

Reuse of subsystems

Disconnected tree

Page 56

Disconnected tree

• Analysing the .mdl file, the structure derived by the Cow_Suite tool is not a connected tree. Some “holes” in the design could be highlighted if existing pieces of a system have not a logical connection.

• More than one disconnected subtrees are derived by the Cow_Suite tool and visualized to the user.

PISATEL

GEFUML Meta-Model

Libraries

ArgoUML

ArgoUML vers.0.7

Designer

(from Logical View)

Sequence

Diagra...

Sequence

Diagram: Desi...

User Model Design Critics

Kernel

Designer

(from Logical View)

Designer

(from Logical View)

Possible solution: inserting dummy nodes and arcs where necessary (according to the logical relation between the disconnected trees).

Page 57

Subsystems Reuse

The reuse of a same subsystem in different parts of the system design causes anomalous connections, i.e. cycles, in the derived structure.

PISATEL

Sequence Diagra...

Sequence Diagram: Desi... Sequence

Diagram: To ...

Sequence Diagram: CheckList...Sequence

Diagram: Kernel / Kernel_Sequ...

ArgoUML

Designer

(from Logical View)

GEF UML Meta-Model

ArgoUML vers.0.7

Libraries

CheckList

User Model Design Critics To Do List

Kernel

Argo

Page 58

Subsystems Reuse

Possible solution:• the first time a subsystem is encountered: associate to it a node and derive for it a corresponding subtree, as usual;

•for the following occurrences of the same subsystem, add to the pseudo-tree a “dummy node” referring back to the

subtree already derived.

PISATEL

Page 59

Conclusions

PISATEL

Tool integrated with the design tool (e.g. Rational Rose)

Systematic and automated support for test planning and test case generation, based on the same UML diagrams developed for design and analysis

Page 60

Future Work

PISATEL

Test case derivation (UIT) to be validated on ERI case studies

Practicality of a test strategy based on manual weights (COW_test)? How to assign the weights?

How to use Cow_Suite within a multi-phase test process (different test stages)?

Page 61

PISATEL

Page 62

Cow_Suite Architecture

REI

Rational Rose

.Tlb library

Cow_Suite

MDL Analizer

Test TreeGenerator

Weights Manager

Test CaseGenerator

Test CaseBuilder

Test CaseChecker

.mdl Files

PISATEL