an automated test strategy based on uml diagrams
DESCRIPTION
PISATEL. An Automated Test Strategy Based on UML Diagrams. F. Basanieri *, A. Bertolino *, E. Marchetti *, A. Ribolini *, G. Lombardi **, G. Nucera ** *PISATEL LAB, IEI-CNR, Pisa, Italy ** ERICSSON LAB ITALY. PISATEL. PISATEL - PowerPoint PPT PresentationTRANSCRIPT
An Automated Test Strategy Based An Automated Test Strategy Based on UML Diagramson UML Diagrams
F. BasanieriF. Basanieri *, A. Bertolino *, *, A. Bertolino *, E. MarchettiE. Marchetti *, A. Ribolini *, *, A. Ribolini *, G. Lombardi **, G. Nucera **G. Lombardi **, G. Nucera **
*PISATEL LAB, IEI-CNR, Pisa, Italy*PISATEL LAB, IEI-CNR, Pisa, Italy
** ERICSSON LAB ITALY** ERICSSON LAB ITALY
PISATEL
Page 2
PISATEL_LAB
PISATEL
PISATELPisa Initiative on Software
Architectures for Telecommunications
The joint conduction by the academia in Pisa and ERI of strategic research projects on software for telecommunications, selected on an annual basis
Page 3
Pisatel Objectives
• Main areas of interest are:– software engineering methods and tools, – software architectures, distibuted systems, networks, real
time systems, and protocols.
• Currently the active projects are:– Integrating Wireless, Wireline and Internet Networks – Analysis and Development of Real Time Software – Quality And Validation of Software Architecture
More details:
http://www.iei.pi.cnr.it/ERI/
PISATEL
An Automated Test Strategy Based An Automated Test Strategy Based on UML Diagramson UML Diagrams
PISATEL
Page 5
Objectives• Use for test planning the same UML diagrams developed
(e.g., by Rational Rose) for design and analysis• Tool-supported generation and maintenance of an
updated and strategic test plan, as the design evolves.
PISATEL
Sequence Diagra...
Sequence Diagram: Desi... Sequence
Diagram: To ...
Sequence Diagram: CheckList...Sequence
Diagram: Kernel / Kernel_Sequ... User Model Design Crit ics To Do List
CheckListKernel
GEF UML Meta-Model Argo
LibrariesArgoUML
ArgoUML vers.0.7
Designer
(from Logical View)
DecisionModel
_decision : String
DecisionModel()GetDecisions()
(from Logical View)GoalModel
_goals : String
GoalModel()getGoals()
(from Logical View)
Decision
_name : String_priority : Integer = 0
Decision()getName()getPriority()
(from Logical View)
+contains
Designer
(from Logical Vi...
Goal
_name : String = "a"_priority : Integer = 3
Goal()getName()getPriority()
(from Logical View)
+contains
: Designer
: Decision...
: Decision : GoalModel
: Goal
1. DecisionModel( ) 1.1. Decision(name, priority)
2. GoalModel( ) 2.1. Goal(name, priority)
3. GetDecisions( ) 3.1. getName( )
3.2. getPriority( )
4. getGoals( ) 4.1. getName( )
4.2. getPriority( )
Test Case getDecisions()getName()/getPriority()
Opening a saved file_decisions
NamingTest Case getDecisions()getName()/getPriority()
Opening a saved file_decisions
Naming Test Case getDecisions()getName()/getPriority()
Opening a saved file_decisions
NamingTest Case
getDecisions()getName()/getPriority()
Opening a saved file_decisions
Naming
Test Case getDecisions()getName()/getPriority()
Opening a saved file_decisions
Naming
Test Case getDecisions()getName()/getPriority()
Opening a saved file_decisions
Naming
Test Case getDecisions()getName()/getPriority()
Opening a saved file_decisions
Naming
Page 6
(Realistic) Preconditions
It is plausible that ….• in early design phases many of the UCs are defined
at a high level and only few of them are properly refined.
• there is not yet the specification of every possible scenario, but only few SDs already exist that realize the defined UCs.
• different people or teams take part to the development, everyone familiar with a different component or functionality.
PISATEL
Page 7
Fundamental Guidelines
• FG1: use the UML diagrams developed during specification and design, without imposing to the UML designers any additional formalism, or ad-hoc effort specifically for testing purposes.
• FG2: test planning for integration testing must cope with high level diagrams, even though not yet completely specified, or refined
PISATEL
Page 8
Trade-off• UML is a notation and does not impose a specific process.• A test strategy needs to refer to a systematic and
documented design processWe want to impose minimal requirements:– Use Cases (UCs) organized into a hierarchy– Express relevant scenarios as Sequence Diagrams (SDs)
PISATEL
Development
Process
Sequence Diagra...
Sequence Diagram: Desi... Sequence
Diagram: To ...
Sequence Diagram: CheckList...Sequence
Diagram: Kernel / Kernel_Sequ... User Model Design Crit ics T o Do List
CheckListKernel
GEF UML Meta-Model Argo
LibrariesArgoUML
ArgoUML vers.0.7
Designer
(from Logical View)
Page 9
Requirements
• The reference documentation is derived by collecting together and automatically organizing the diagrams developed during the design phase without additional manual effort of formalization.
• Inconsistencies and incompleteness are highlighted (Design weaknesses)
PISATEL
Sequence
Diagram: Kernel
/
Kernel_Sequ... Sequence
Diagram: To ...
To Do List
Sequence
Diagra...
Sequence
Diagram: Desi...
User ModelDesign Critics
Kernel
Sequence Diagram:
CheckList...
CheckList
Argo
GEFUML Meta-Model
LibrariesArgoUML
ArgoUML vers.0.7
Designer
(from Logical View)
Page 10
Cow_Suite approach
The processing of the Cow_Suite approach starts by analysing the project description*, and proceeds in parallel with the design and analysis refinement steps.
[*: the internal description of the project model, given by Rational Rose in a .mdl file]
PISATEL
Cow_Suite consists in the junction of two main components: UIT (Use Interaction Test) and CoWTeSt (Cost Weighted Test Strategy)
Cow_Suite =Cowtest pluS UIT Environment
Page 11
Cow_Suite toolThe Cow_Suite tool consists of three elements working in
combination:
PISATEL
1. the Main Tree: a systematic support to decide a. how to distribute the available test effort between the many
potential test cases
b. how to prioritise among the test cases when a functional coverage is fixed.
2. the UIT Tree: automated generation of abstract test cases from the UML diagrams.
3. the Test specification: a systematic support to generate the test procedures and subsequently the test scripts.
Page 12
Cow_Suite Scheme
Test caseTest case
Test case
Test case
Test caseTest case
Test case
Test case
Test case
Test case
Test case
Test case
Test case
Test case
Test case
Test caseTest Test
Procedures Procedures
listlist
Strategy
StrategySele
ction
Selecti
on
Obj 1 Obj 2 Obj 3
Method1()
Method3()
Method2()
U I TU I T
SDs selection according to the strategy chosen
Automatic UIT application for Test
cases derivation
Test specific
ation
Interaction with the user for deriving executable test procedures
Sequence Diagra...
Sequence Diagram: Desi... Sequence
Diagram: To ...
Sequence Diagram: CheckList...Sequence
Diagram: Kernel / Kernel_Sequ... User Model Design Crit ics To Do List
CheckListKernel
GEF UML Meta-Model Argo
LibrariesArgoUML
ArgoUML vers.0.7
Designer
(from Logical View)
Sequence Diagra...
Sequence Diagram: Desi... Sequence
Diagram: To ...
Sequence Diagram: CheckList...Sequence
Diagram: Kernel / Kernel_Sequ... User Model Design Critics To Do List
CheckListKernel
GEF UML Meta-Model Argo
LibrariesArgoUML
ArgoUML vers.0.7
Designer
(from Logical View)
0.250.25 0.50
0.500.50
0.7 0.3
0.30.35 0.15
Cow_SuiteCow_Suite
PISATEL
Page 13
Main steps
• Derive a Reference Tree • Deduce the Critical Profile• Select the Test Strategy• Derive the Test Cases• Implement the Test Procedures
PISATEL
Page 14
1.1 Tree DerivationUse Cases (UCs) and Sequence Diagrams (SDs) of the .mdl project design realized with Rational Rose are automatically organized by Cow_Suite tool in a sort of hierarchical (pseudo-)tree, starting from the Main Use Case Diagram, UCD.
Sequence Diagra...
Sequence Diagram: Desi... Sequence
Diagram: To ...
Sequence Diagram: CheckList...Sequence
Diagram: Kernel / Kernel_Sequ... User Model Design Crit ics T o Do List
CheckListKernel
GEF UML Meta-Model Argo
LibrariesArgoUML
ArgoUML vers.0.7
Designer
(from Logical View)
Hierarchical TreeHierarchical Tree
The case study is ArgoUML, a tool supporting OO design with UML
PISATEL
Page 15
1.2 Tree Derivation
• Looking at the tree the Cow_Suite users can get a complete view of the status of the specification of the various functionalities;
• The tree represents an ordered and organized documentation, that is continuously updated with the latest changes and progressively completed
Hierarchical TreeHierarchical Tree
PISATEL
Page 16
2.1 Deduce the Critical Profile• Systems are naturally composed of different parts realizing one or more
functionalities, each with different importance for the overall system performance.
• The effort devoted to the test of different functionalities must be planned and scheduled considering their relative “importance” (risk-based testing, reliability guided testing, etc…).
• Following level by level the tree structure of the UCs and SDs, the strategy allows the people in charge of the development of one or more functionalities to simply annotate the corresponding node in the tree with a number (weight).
PISATEL
0.75?0.60?0.65?
0.65!!!Sequence
Diagram: To ...
To Do List
Page 17
2.2 Weighted tree derivation
The user annotates every node with a value, leaf weight, representing the relative “importance” of this node (be it a UC or a SD) with respect to the other nodes at the same level in the tree
Hierarchical TreeHierarchical Tree
PISATEL
Page 18
2.3 Weights managementThe weight value belongs to the [0,1] interval and must be assigned so that the sum of the weights associated to all the children of one node is equal to 1.
The weights are user modifiable as the development proceeds.
PISATEL
Page 19
Integration Testing vs Integration Stage
• To realize a complete system functionality, it is generally necessary to execute several actions realizing lower level functionalities
• A system functionality is realized by the different components interaction.
• Integration Testing: testing interactions between system components (processes, packages, subsystems… ) at an appropriate level of abstraction.
• Integration Stage: is useful because generally the functionalities are not specified at the same level of detail and with the same number of UCs and SDs
PISATEL
Page 20
2.4 Integration Stage Selection
• The first integration stage is represented by the main UC and the SDs (if any), which are children of this node (hence they are at level 2 of the tree).
• The i-th integration stage is represented by the UCs positioned in at i-th level of the tree and every SDs, children of these nodes, situated at i+1-th level.
NOTE: the SDs at level i+1 represent the interaction between the different components that realize the functionalities described in the UCs at i-th level.
PISATEL
Page 21
Example: 4th Integration Stage
The Cow_Suite user selects the integration stage (4) at which the test is conducted.
The nodes that are considered by Cow_Suite tool for the subsequent evaluations are all and only those belonging to the 4th integration stage plus all the tree leaves existing at higher levels (3rd, 2nd, 1st).
SDs at 5th tree level
PISATEL
Page 22
2.5 Final weights assignment• The weights assigned to each node are used to define a relative
importance factor, in terms of how risky is that node and how much effort should be put to test it.
• Considering every node, from the root down to the elements belonging to the integration stage considered, its final weight is computed as the product of all the nodes weights on the complete path from the root to this node.
• The final resulting weight is an element of discrimination for choosing amongst the tests to execute.
PISATEL
GEFUML Meta-Model
Libraries
ArgoUML
ArgoUML vers.0.7
Designer
(from Logical View)
0.60.40.3
FW=0.12
FW=0.6
Page 23
3. Cow_Test Strategy
Two different situations are considered in test cases planning:
1: Cowtest_ing with fixed number of tests
• A certain budget is available, which is translated in terms of number of tests, NT, to execute.
• In this case the tool select the most adequate selection of NT tests from among the (many) various test cases that could be potentially conceived.
PISATEL
Test Case
Test Case Test Case
Tes
t Cas
e
Test Case
Tes
t Cas
e
Test Case
Test C
ase
Test Case Test Case
Test C
ase
Test Case Test Case
Test Case Test Case
Test Case Test Case
Test Case
Test Case
Test Case
Test Case Test Case
Test Case Test Case
Test Case Test Case
Test Case
Test Case Test Case
Test Case Test Case
Test Case Test Case
Test Case Test Case
Test Case Test Case
Test Case Test Case
Test Case Test Case
Test Case Test Case
Test Case Test Case
Test Case Test Case
Test Case Test Case
Test Case Test Case
NTNT
Page 24
3. Cow_Test Strategy
2: Cowtest_ing with fixed functional coverage• A certain percentage of functionalities must be covered.
• In this case by the tool is possible to define in advance which are the functionalities to be covered and the minimum number of tests to
execute.
PISATEL
Functional Coverage = 90%
Sequence Diagra...
Sequence Diagram: Desi... Sequence
Diagram: To ...
Sequence Diagram: CheckList...Sequence
Diagram: Kernel / Kernel_Sequ... User Model Design Critics To Do List
CheckListKernel
GEF UML Meta-Model Argo
LibrariesArgoUML
ArgoUML vers.0.7
Designer
(from Logical View)
0.250.25 0.50
0.500.50
0.7 0.3
0.30.35 0.15
10
2 2
11
Minimum Number of tests to execute = 16
Page 25
3.1 Select the Test Strategy
Cowtest_ing with fixed number of tests.
The selection of the most suitable distribution of Test Procedures is developed on the basis of leaves weights.
PISATEL
Page 26
Cowtest_ing with fixed number of testsUsing the resulting final weight at the chosen integration stage, nw, for each SD the number of Test Procedures, NTtest,
can be easily derived
Int-Stage Leaves names Critical profile
2nd Stage/NTest
3rd Stage/NTest
4th Stage/NTest
5th Stage/NTest
1st Stage ArgoUML version 7
1
Libraries 0.5 0.5 250 0.5 0.5 0.5 2nd Stage ArgoUML 0.5 0.5 250 0.5 250 0.5 250 0.5 250 GEF 0.25 0.125 63 0.125 63 63 UML Meta-Model 0.25 0.125 63 0.125 63 63
3rd Stage
Argo 0.5 0.25 125 0.25 0.25 Kernel , 0.7 0.175 0.175 Kernel_SD 0.2 0.035 17 0.035 17 Kernel_NDchild 0.8 0.14 70 Checklist, 0.3 0.075 0.075
4th Stage
Checklist_SD 1 0.075 38 0.075 38 User Model 0.3 0.0525 UserModel_SD, 1 0.0525 26 Design Critics 0.35 0.0613 DesignCritics_SD 1 0.0613 31 To Do List 0.15 0.0263
5th Stage
ToDoList_SD 1 0.0263 13
PISATEL
Page 27
3.2 Select the Test Strategy
Cowtest_ing with fixed functional coverage.
The most critical system functionalities are
selected for reaching the fixed coverage.
Test Procedures are distributed accordingly.
PISATEL
Page 28
Cowtest_ing with fixed functional coverageConsidering the final weights, nw, of every node the tool can: • select the functional test cases to be run, by ordering in decreasing
manner the nw*100 values and summing them, starting from the heaviest one, until the fixed coverage, C, is reached.
• Calculate the minimum number of tests with respect to the fixed coverage percentage.
Leaves names 5th Stage
weights
70%coverage
nwf /NTest
80%coverage/
nwf /NTest t
90%coverage/
nwf /NTest
100%coverage/
nwf /NTest
ArgoUML 0.5 0.667 4 0.61 6 0.533 10 0.5 19
GEF 0.125 0.167 1 0.152 2 0.133 2 0.125 5
UML Meta-
Model
0.125 0.167 1 0.152 2 0.133 2 0.125 5
Checklist_SD 0.075 0.1 1 0.08 1 0.075 3
DesignCritics_SD 0.0613 0.065 1 0.0613 2
UserModel_SD 0.0525 0.056 1 0.0525 2
Kernel_SD 0.035 0.035 1
ToDoList_SD 0.0263 0.0263 1
PISATEL
Page 29
Notes
• To make a more practical prediction in terms of man/hours, or required budget for testing, it would be necessary to be able to estimate the cost of the different test cases.
• An alternative application of Cowtest is considering a certain budget, B, planned for the testing instead of the fixed number of test NT. In this case is possible to evaluate the “relative cost”, b, for the testing of every leaf of the fixed integration stage.
PISATEL
Page 30
Cow_Suite Scheme
Test caseTest case
Test case
Test case
Test caseTest case
Test case
Test case
Test case
Test case
Test case
Test case
Test case
Test case
Test case
Test case
Strategy
StrategySele
ction
Selecti
on
Obj 1 Obj 2 Obj 3
Method1()
Method3()
Method2()
U I TU I T
SDs selection according to the strategy chosen
Automatic UIT application for Test
cases derivation
Sequence Diagra...
Sequence Diagram: Desi... Sequence
Diagram: To ...
Sequence Diagram: CheckList...Sequence
Diagram: Kernel / Kernel_Sequ... User Model Design Crit ics To Do List
CheckListKernel
GEF UML Meta-Model Argo
LibrariesArgoUML
ArgoUML vers.0.7
Designer
(from Logical View)
Sequence Diagra...
Sequence Diagram: Desi... Sequence
Diagram: To ...
Sequence Diagram: CheckList...Sequence
Diagram: Kernel / Kernel_Sequ... User Model Design Critics To Do List
CheckListKernel
GEF UML Meta-Model Argo
LibrariesArgoUML
ArgoUML vers.0.7
Designer
(from Logical View)
0.250.25 0.50
0.500.50
0.7 0.3
0.30.35 0.15
Cow_SuiteCow_Suite
PISATEL
Page 31
For each fixed integration stage, a weighted subtree is derived according to the choosen test criterion. Only the relative SDs are considered for Test Case generation.
SDs selectionSDs selection
3.3 Sequence Diagrams Selection
PISATEL
Final weightN. Test Procedures
Page 32
4. Test Case Derivation
The tool uses the UIT method, Use Interaction Test, to derive the Test Cases.
The method is mainly based on the SDs (particularly objects, messages and parameters)
SDs SelectionSDs Selection
PISATEL
Page 33
4.1 Use Interaction Test methodology
• A method to systematically derive Integration Test suites from UML diagrams.
• Inspired at large by the Category Partition method: it defines classes of functional equivalence by selecting significant values (choices) of relevant categories (parameters or data)
• Incremental test strategy (over the Use Case organizational structure)
PISATEL
Page 34
4.1.1 Sequence Diagram analysis
: Designer
: Decision...
: Decision : GoalModel
: Goal
1. DecisionModel( ) 1.1. Decision(name, priority)
2. GoalModel( ) 2.1. Goal(name, priority)
3. GetDecisions( ) 3.1. getName( )
3.2. getPriority( )
4. getGoals( ) 4.1. getName( )
4.2. getPriority( )
Horizontal axis shows a set of objects, Test Units, that interact through messages.Those are system units separately testable (class testing) for exercising a possible use of system.Example:
DecisionModel, Decision, GoalModel, Goal.
PISATEL
Page 35
4.1.2 Sequence Diagram analysis
: Designer
: Decision...
: Decision : GoalModel
: Goal
1. DecisionModel( ) 1.1. Decision(name, priority)
2. GoalModel( ) 2.1. Goal(name, priority)
3. GetDecisions( ) 3.1. getName( )
3.2. getPriority( )
4. getGoals( ) 4.1. getName( )
4.2. getPriority( )
Vertical axis shows:• the exchanged messages, Interactions Categories, involved in object interactions;• their parameters and relevant inputs, Settings Categories.DecisionModel
Settings: _decisions
Interaction: DecisionModel() getDecisions()
Decision Interactions: Decision(name:String,priority:int)GetName(), GetPriority() ............
PISATEL
Page 36
Additional Information: Class DiagramComplementary analysis of the Class Diagram description (mainly used for searching the Settings Categories).
These are attributes (or a subset of them) of a class (and the corresponding SD’s object) that affect object interactions and are represented by input parameters used in messages or data structures.
PISATEL
DecisionModel
_decision : String
DecisionModel()GetDecisions()
(from Logical View)GoalModel
_goals : String
GoalModel()getGoals()
(from Logical View)
Decision
_name : String_priority : Integer = 0
Decision()getName()getPriority()
(from Logical View)
+contains
Designer
(from Logical Vi...
Goal
_name : String = "a"_priority : Integer = 3
Goal()getName()getPriority()
(from Logical View)
+contains
Page 37
4.1.3 Search of Message Sequences
: Designer
: Decision...
: Decision : GoalModel
: Goal
1. DecisionModel( ) 1.1. Decision(name, priority)
2. GoalModel( ) 2.1. Goal(name, priority)
3. GetDecisions( ) 3.1. getName( )
3.2. getPriority( )
4. getGoals( ) 4.1. getName( )
4.2. getPriority( )
MessageSequences: set of Messages (in temporal order), involved in a SD, used by objects to define and elaborate specific functionalities.
Example:
getDecisions()
getPriority()
PISATEL
Page 38
4.2 Test Case Derivation
For each traceable Message Sequence a Test Case is generated. It contains the list of all its Settings and Interactions Categories and their values.
Detailed Test Case Description
PISATEL
Page 39
Conditions
A Test Case (TC) can be associated to some feasibility conditions formally expressed (notes or specifications tab).
In this case TC is divided in different subcases.
PISATEL
Page 40
Number of Test Procedures
The final weight of every SDs is used to automatically derive the number of Test Procedures associated to each Test Case.
PISATEL
Number of associated Test Procedures
Page 41
Cow_Suite Scheme
Test caseTest case
Test case
Test case
Test caseTest case
Test case
Test case
Test case
Test case
Test case
Test case
Test case
Test case
Test case
Test caseTest Test
Procedures Procedures
listlist
Strategy
StrategySele
ction
Selecti
on
Obj 1 Obj 2 Obj 3
Method1()
Method3()
Method2()
U I TU I T
Test specific
ation
Interaction with the user for deriving executable test procedures
Sequence Diagra...
Sequence Diagram: Desi... Sequence
Diagram: To ...
Sequence Diagram: CheckList...Sequence
Diagram: Kernel / Kernel_Sequ... User Model Design Crit ics To Do List
CheckListKernel
GEF UML Meta-Model Argo
LibrariesArgoUML
ArgoUML vers.0.7
Designer
(from Logical View)
Sequence Diagra...
Sequence Diagram: Desi... Sequence
Diagram: To ...
Sequence Diagram: CheckList...Sequence
Diagram: Kernel / Kernel_Sequ... User Model Design Critics To Do List
CheckListKernel
GEF UML Meta-Model Argo
LibrariesArgoUML
ArgoUML vers.0.7
Designer
(from Logical View)
0.250.25 0.50
0.500.50
0.7 0.3
0.30.35 0.15
Cow_SuiteCow_Suite
PISATEL
Page 42
5.1 Test Procedures Specification
Test Specification:
For each identified category, we consider all its possible values and constraints (“choices”).
PISATEL
The user interacts with the tool for inserting Choices values.
Page 43
5.1.1 InteractionsCategories Specification
Interactions
Categories: objects interactions, exchanged messages.
The tool implements two possible choices for the user:
the default values,
the user values
PISATEL
Page 44
State Diagram
PISATEL
ACTIVE (Priority >0)
INACTIVE (Priority =0)
SNOOZED UNSNOOZED
set Priority to 0, beInactive(), Snooze()set Priority v>0, Wake()
set Priority to 0, beInactive(), RemoveItem
Snooze()
Snooze()
Elapsed Time, Wake()
Unsnooze()RecomputeAllToDoItems[ elapsed time ]
set Priority v>0
Unsnoozed(), Wake()
If a State Diagram is linked to a specific Test Unit (class), further information can be found about the relevant configurations of InteractionCategories
State diagram of the “Critic” class
Page 45
5.1.2 Specification of SettingsCategories
Settings
Categories:
parameters and inputs relevant for the Test Unit.
The tool implements two possible choices for the user:
the default values,
the user values.
PISATEL
Page 46
5.2 Test Procedures Implementation
A Test Procedure is automatically generated from a Test Specification, for each possible combination of choices of every category involved in a Messages Sequence.
PISATEL
Page 47
Test Procedures: an exampleTest Specification:
DecisionModel:
Settings:
_decisions
Naming
Storage
Modularity
Inheritance
Stereotypes
Relationships……..
Interactions:
getDecisions
Opening a new file
Opening a saved file
After a modification and before saving
After a modification and after saving
DecisionModel()Class constructor
Test Procedure
getDecisions()
getName()
Opening a saved file
_decisions
Naming
Note: Such a Test Case must be repeated for all possible values of _decisions
Page 48
Constraints on Choices
Some choices combinations could result contradictory or meaningless. Following the Category Partition methodology, constraints must be added to the SettingCategories choices.
Constraint are specified using: • Properties, that can be assigned to certain choices and testing
for by other choices.• IF Selector, is a conjunction of properties that were previously
assigned to other choices.
PISATEL
Page 49
Example
SettingsCategories:
pattern size:
empty [Property Empty]
single character [Property Not Empty]
many character [Property Not Empty]
quoting
pattern is quoted [Property quoted]
pattern is not quoted [if Not Empty]
pattern is improperly quoted [if Not Empty]
PISATEL
Page 50
SettingsCategories: Specification Constraints
Test Specification:
Decision(_name, _priority):
_name
Naming [if priority<5]
Class Selection [if priority=0]
Storage
Inheritance
Modularity…..
_priority
0 [Property priority=0]
1
2
3
7 [Property not priority<5]
PISATEL
Page 51
SettingsCategories: Specification Constraints
Test Procedure
DecisionModel()
Opening an existing document
Decision( _name, _priority)
_nameNaming [if priority<5]
_priority7 [Property not
priority<5]
Test Procedure
DecisionModel()
Opening an existing document
Decision( _name, _priority)
_name
Naming [if priority<5]
_priority
4
X
Page 53
Note• The Test Procedures, as derived by the tool, are not directly executable
( Test Scripts). The derived Test Procedures have to be executed and logged by a test driver.
CowSuite goal is:– giving automatic and systematic support to construct a meaningful
Test Suite. – It is not to directly execute the tests.
Cow_Suite
MDL Analizer
Test TreeGenerator
Weights Manager
Test CaseGenerator
Test CaseBuilder
Test CaseChecker
REI
Rational Rose
.Tlb library
.mdl Files
Test Driver
Page 54
Remarks
• The degree of detail of the Test Suite depends directly on:– the design detail level– granularity of the inserted values (selection of
the constraints and choices of the UIT categories).
• Currently: the final result is a text document containing the Test Suite.
• In future: different format (e.g. XML), that can be accepted in input by the adopted test driver.
Page 55
Features under Construction
Two different situations can occur during the tree construction:
PISATEL
Sequence
Diagram: Kernel
/
Kernel_Sequ... Sequence
Diagram: To ...
To Do List
Sequence
Diagra...
Sequence
Diagram: Desi...
User ModelDesign Critics
Kernel
Sequence Diagram:
CheckList...
CheckList
Argo
GEFUML Meta-Model
Libraries
ArgoUML
ArgoUML vers.0.7
Designer
(from Logical View)
Reuse of subsystems
Disconnected tree
Page 56
Disconnected tree
• Analysing the .mdl file, the structure derived by the Cow_Suite tool is not a connected tree. Some “holes” in the design could be highlighted if existing pieces of a system have not a logical connection.
• More than one disconnected subtrees are derived by the Cow_Suite tool and visualized to the user.
PISATEL
GEFUML Meta-Model
Libraries
ArgoUML
ArgoUML vers.0.7
Designer
(from Logical View)
Sequence
Diagra...
Sequence
Diagram: Desi...
User Model Design Critics
Kernel
Designer
(from Logical View)
Designer
(from Logical View)
Possible solution: inserting dummy nodes and arcs where necessary (according to the logical relation between the disconnected trees).
Page 57
Subsystems Reuse
The reuse of a same subsystem in different parts of the system design causes anomalous connections, i.e. cycles, in the derived structure.
PISATEL
Sequence Diagra...
Sequence Diagram: Desi... Sequence
Diagram: To ...
Sequence Diagram: CheckList...Sequence
Diagram: Kernel / Kernel_Sequ...
ArgoUML
Designer
(from Logical View)
GEF UML Meta-Model
ArgoUML vers.0.7
Libraries
CheckList
User Model Design Critics To Do List
Kernel
Argo
Page 58
Subsystems Reuse
Possible solution:• the first time a subsystem is encountered: associate to it a node and derive for it a corresponding subtree, as usual;
•for the following occurrences of the same subsystem, add to the pseudo-tree a “dummy node” referring back to the
subtree already derived.
PISATEL
Page 59
Conclusions
PISATEL
Tool integrated with the design tool (e.g. Rational Rose)
Systematic and automated support for test planning and test case generation, based on the same UML diagrams developed for design and analysis
Page 60
Future Work
PISATEL
Test case derivation (UIT) to be validated on ERI case studies
Practicality of a test strategy based on manual weights (COW_test)? How to assign the weights?
How to use Cow_Suite within a multi-phase test process (different test stages)?