xml-based representation of test cases for distributed systems
DESCRIPTION
XML-based Representation of Test Cases for Distributed Systems. Alexander Kraas 7. October 2008. Contents. Motivation Toolchain and Workflow Design Goals Structure and Mapping of XTR Test Cases Conclusion & Future work. Motivation. Actual situation - PowerPoint PPT PresentationTRANSCRIPT
XML-based Representation of Test Cases for Distributed Systems
Alexander Kraas
7. October 2008
Folie 2
Alexander KraasOct 2007
www.esk.fraunhofer.dewww.esk.fraunhofer.de
Contents
1. Motivation
2. Toolchain and Workflow
3. Design Goals
4. Structure and Mapping of XTR Test Cases
5. Conclusion & Future work
Folie 3
Alexander KraasOct 2007
www.esk.fraunhofer.dewww.esk.fraunhofer.de
MotivationActual situation
Many commercial and academic tools for the model-based test generation
Different solutions for test case specification:
- Using the same language for programming the SUT and writing test cases
- Specification of test cases with scripting languages, e.g. TCL
- Utilization of specialized testing languages, for example TTCN-3
Only a small number of test generation tools support several target languages
Motivation for using an abstract representation format
A test representation format can facilitates:
- The exchange of test cases between different tools
- The implementation of adapters for different target languages
Advantages of using XML for test case representation:
- XML documents can be formalized by XML Schemata (XSD)
- XML is human-readable
Folie 4
Alexander KraasOct 2007
www.esk.fraunhofer.dewww.esk.fraunhofer.de
Toolchain and Workflow (1)
Test CaseGenerator 1
Test CaseGenerator N...
Test CaseSelection Module
(optional)
Test Data Generator(optional)
XML Test Representation (XTR)
TTCN-3Language Adapter
JUnitLanguage Adapter
Adapters for other Languages(optional)
Exemplarily architecture of a tool chain which utilizes the XTR format
Different test case generators and other optional modules
Adaptors for different target languages
Folie 5
Alexander KraasOct 2007
www.esk.fraunhofer.dewww.esk.fraunhofer.de
Toolchain and Workflow (2)
Workflow of test case generation with the XTR format
1. Test Case Generator produces test cases in the XTR format
2. Optional modules can be used for post-processing of XTR test cases
3. A Language Adapter transforms XTR test cases into a target language,e.g. TTCN-3 or JUnit
Test CaseGenerator 1
UML Model
XMI
Test CaseSelection Module
Test Data Generator
Language Adapter
XTR XTR XTR
TargetLanguage
XMIXMI
Folie 6
Alexander KraasOct 2007
www.esk.fraunhofer.dewww.esk.fraunhofer.de
Design Goals
The design of the XTR format rests on following main objectives
Representation of black-box test cases for the test of distributed communication systems
Support of concurrency and asynchronous communication
Usage of a simplified test architecture (without communication ports)
Powerful data type concept for representing user-defined as well as object-oriented data types
Providing only a minimal set of test interactions, because only automatic generated test cases should be represented
Folie 7
Alexander KraasOct 2007
www.esk.fraunhofer.dewww.esk.fraunhofer.de
Structure and Mapping of XTR Test Cases (1)
The structure of a XTR test suite consists of four different element groups
<typeDefinitions>
- Containing all data types used by the test cases
- Definitions can consists of simple as well as complex data types
<templateDefinitions>
- Optional template definitions for data types.
- Templates can represent pre-defined values or matching expressions
<testArchitecture>
- Specification of an abstract test architecture
- The architecture have to be used by all test cases of a XTR test suite.
<testCases>
- This element contains all test cases of a XTR test suite
Folie 8
Alexander KraasOct 2007
www.esk.fraunhofer.dewww.esk.fraunhofer.de
Data type definitions and their mapping to TTCN-3
<class name="ExampleClass" baseType="class" parameter="false"> <attributes> <attribute name="name" typeName="string" baseType="string"/> </attributes> <methods> <method name="setName" typeName="void" baseType="void"> <parameter name="parName" typeName="string" baseType="string"/> </method> <exception name="exceptionCause" typeName="string" baseType="string" </methods></class>
Structure and Mapping of XTR Test Cases (2)
Folie 9
Alexander KraasOct 2007
www.esk.fraunhofer.dewww.esk.fraunhofer.de
type record attrib_ExampleClass {
charstring name }
signature setName_ExampleClass (inout charstring par1)
exception (charstring);
Data type definitions and their mapping to TTCN-3
<class name="ExampleClass" baseType="class" parameter="false"> <attributes> <attribute name="name" typeName="string" baseType="string"/> </attributes> <methods> <method name="setName" typeName="void" baseType="void"> <parameter name="parName" typeName="string" baseType="string"/> </method> <exception name="exceptionCause" typeName="string" baseType="string" </methods></class>
Structure and Mapping of XTR Test Cases (2)
Folie 10
Alexander KraasOct 2007
www.esk.fraunhofer.dewww.esk.fraunhofer.de
Test architecture
Structure and Mapping of XTR Test Cases (3)
Abstract Test Architecture
TestComponent 1
SUT
Objects Under Test
Protocolxy
Protocol Under Test
Test Component 2
a b c
A XTR test architecture can consists of:
Test components for protocol tests
Test components for API tests
Folie 11
Alexander KraasOct 2007
www.esk.fraunhofer.dewww.esk.fraunhofer.de
Test architecture
Structure and Mapping of XTR Test Cases (3)
Abstract Test Architecture
TestComponent 1
SUT
Objects Under Test
Protocolxy
Protocol Under Test
Test Component 2
a b c
<testArchitecture> <globalParameters/> <testComponent name="Component_1"> <api> <object name="sutObj_a" type="Class_A"> <object name="sutObj_b" type="Class_B"> <object name="sutObj_c" type="Class_C"> </api> </testComponent> <testComponent name="Component_2"> <protocol protocolName="protocol_xy"/> </testComponent></testArchitecture>
Folie 12
Alexander KraasOct 2007
www.esk.fraunhofer.dewww.esk.fraunhofer.de
Test architecture
Structure and Mapping of XTR Test Cases (3)
<testArchitecture> <globalParameters/> <testComponent name="Component_1"> <api> <object name="sutObj_a" type="Class_A"> <object name="sutObj_b" type="Class_B"> <object name="sutObj_c" type="Class_C"> </api> </testComponent> <testComponent name="Component_2"> <protocol protocolName="protocol_xy"/> </testComponent></testArchitecture>
testcase TC_1() runs on mtcType {
//creating parallel testComponents:
var t_Component_1 Component_1 :=
Component_1.create;
var t_Component_2 Component_2 :=
Component_2.create;
// mapping to System Ports
map (Component_1.pco1:system.pco1);
map (Component_2.pco2:system.pco2);
// starting Components behaviour
Component_1.start(testBehaviour_1());
Component_2.start(testBehaviour_2());
}
Folie 13
Alexander KraasOct 2007
www.esk.fraunhofer.dewww.esk.fraunhofer.de
Specification of a XTR test case
Structure and Mapping of XTR Test Cases (4)
<testCase name="TC_1"> <testBehaviour testComponent="Component_1"> … </testBehaviour> <testBehaviour testComponent="Component_2"> <variables> <variable name="timeOutTimer" type="Timer"/> <variable name="waitTimer" type="Timer"/> </variables> <testInteraction> <send> // send testInteraction with a fixed value <message> <value typeName="aStructure"> ... </value> </message> </send> </testInteraction> <timerOperation type="start" timer="timeOutTimer"/> <testInteraction> <receive verdict="pass"> // testInteraction <message> // using a template <template templateName="aTemplate"/> </message> </receive> <timeout timer="timeOutTimer" verdict="fail"/> </testInteraction> <timerOperation type="stop" timer="timeOutTimer"/> <synchronize/> // => Synchronization of test </testBehaviour> // components</testCase>
Concurrent test behavior:
Expressed by different groups of <testBehavior> elements
Running on different testComponents
Folie 14
Alexander KraasOct 2007
www.esk.fraunhofer.dewww.esk.fraunhofer.de
Specification of a XTR test case
Structure and Mapping of XTR Test Cases (4)
<testCase name="TC_1"> <testBehaviour testComponent="Component_1"> … </testBehaviour> <testBehaviour testComponent="Component_2"> <variables> <variable name="timeOutTimer" type="Timer"/> <variable name="waitTimer" type="Timer"/> </variables> <testInteraction> <send> // send testInteraction with a fixed value <message> <value typeName="aStructure"> ... </value> </message> </send> </testInteraction> <timerOperation type="start" timer="timeOutTimer"/> <testInteraction> <receive verdict="pass"> // testInteraction <message> // using a template <template templateName="aTemplate"/> </message> </receive> <timeout timer="timeOutTimer" verdict="fail"/> </testInteraction> <timerOperation type="stop" timer="timeOutTimer"/> <synchronize/> // => Synchronization of test </testBehaviour> // components</testCase>
<testInteraction> elements are used to specify different interactions with the SUT:
For message-based communication <send> and <receive>
For procedure-based communication <call>, <return> and <catch>
Folie 15
Alexander KraasOct 2007
www.esk.fraunhofer.dewww.esk.fraunhofer.de
Specification of a XTR test case
Structure and Mapping of XTR Test Cases (4)
<testCase name="TC_1"> <testBehaviour testComponent="Component_1"> … </testBehaviour> <testBehaviour testComponent="Component_2"> <variables> <variable name="timeOutTimer" type="Timer"/> <variable name="waitTimer" type="Timer"/> </variables> <testInteraction> <send> // send testInteraction with a fixed value <message> <value typeName="aStructure"> ... </value> </message> </send> </testInteraction> <timerOperation type="start" timer="timeOutTimer"/> <testInteraction> <receive verdict="pass"> // testInteraction <message> // using a template <template templateName="aTemplate"/> </message> </receive> <timeout timer="timeOutTimer" verdict="fail"/> </testInteraction> <timerOperation type="stop" timer="timeOutTimer"/> <synchronize/> // => Synchronization of test </testBehaviour> // components</testCase>
Values for test interactions can be assigned in three different ways:
Fixed inline values
Templates
Dynamic values from local variables
Folie 16
Alexander KraasOct 2007
www.esk.fraunhofer.dewww.esk.fraunhofer.de
Specification of a XTR test case
Structure and Mapping of XTR Test Cases (4)
<testCase name="TC_1"> <testBehaviour testComponent="Component_1"> … </testBehaviour> <testBehaviour testComponent="Component_2"> <variables> <variable name="timeOutTimer" type="Timer"/> <variable name="waitTimer" type="Timer"/> </variables> <testInteraction> <send> // send testInteraction with a fixed value <message> <value typeName="aStructure"> ... </value> </message> </send> </testInteraction> <timerOperation type="start" timer="timeOutTimer"/> <testInteraction> <receive verdict="pass"> // testInteraction <message> // using a template <template templateName="aTemplate"/> </message> </receive> <timeout timer="timeOutTimer" verdict="fail"/> </testInteraction> <timerOperation type="stop" timer="timeOutTimer"/> <synchronize/> // => Synchronization of test </testBehaviour> // components</testCase>
Test verdict is determined by the result of
<receive>,
<return> or
<catch> test interactions.
Folie 17
Alexander KraasOct 2007
www.esk.fraunhofer.dewww.esk.fraunhofer.de
Specification of a XTR test case
Structure and Mapping of XTR Test Cases (4)
<testCase name="TC_1"> <testBehaviour testComponent="Component_1"> … </testBehaviour> <testBehaviour testComponent="Component_2"> <variables> <variable name="timeOutTimer" type="Timer"/> <variable name="waitTimer" type="Timer"/> </variables> <testInteraction> <send> // send testInteraction with a fixed value <message> <value typeName="aStructure"> ... </value> </message> </send> </testInteraction> <timerOperation type="start" timer="timeOutTimer"/> <testInteraction> <receive verdict="pass"> // testInteraction <message> // using a template <template templateName="aTemplate"/> </message> </receive> <timeout timer="timeOutTimer" verdict="fail"/> </testInteraction> <timerOperation type="stop" timer="timeOutTimer"/> <synchronize/> // => Synchronization of test </testBehaviour> // components</testCase>
function testBehaviour_2 () runs on
Component_2 {
timer timeOutTimer := 15.0;
timer waitTimer := 15.0;
{
pco2.send(...) // sending a fixed value
timeOutTimer.start;
alt {
[]pco2.receive(aTemplate) {
setverdict(pass);
timeOutTimer.stop;}
[]pco2.receive(?) {
setverdict(fail);
timeOutTimer.stop;}
[] timeOutTimer.timeout{
setverdict(fail);}
}
synchronize(); // Waiting for other
testcomponents
}
Folie 18
Alexander KraasOct 2007
www.esk.fraunhofer.dewww.esk.fraunhofer.de
Conclusion & Future work
Summary
The XTR format has been designed in order to represent test cases for distributed communicating systems
Support for asynchronous communication and concurrency
Mappings to different target languages
XTR is independent from any test generation tool
Open Issues
Further refinements of the XTR format specification
Applicability of the XTR format for representing large test cases
Integration of different test case generators
Folie 19
Alexander KraasOct 2007
www.esk.fraunhofer.dewww.esk.fraunhofer.de
Any Questions?