ttcn-3 test generation from system models - tu/e · pdf file1 ttcn-3 test generation from...

1

TTCN-3 Test Generation from System Models

Ina SchieferdeckerIna SchieferdeckerTechnicalTechnical University Berlin/Fraunhofer FOKUSUniversity Berlin/Fraunhofer [email protected]@fokus.fraunhofer.de

and and thethe FOKUS FOKUS teamteam::

MaikMaik Busch, Busch, RanganaiRanganai ChaparadzaChaparadza, , Zhen Zhen RuRu Dai, Andreas Hoffmann, Dai, Andreas Hoffmann, LauretteLaurette LacmeneLacmene, Titus , Titus NgwangwenNgwangwen, , Guy Collins Guy Collins NdemNdem, Diana , Diana SerbanescuSerbanescu, , JustynaJustyna ZanderZander--NowickaNowicka

�� !��"# ��

Contents

� Motivation

� The Basics: TTCN-3

� The Approach: MDA++

� Some more details

� Summary

2

�� !��"# ��

Contents

� Motivation




� Summary

�� !��"# ��

Motivation

� Integrated Development and Testing

� Early and continuous consideration of test aspects

��

� � ��

� ��

� � � � � � ��

��

� � ��

��

� � ��

� ��

��

��

� � ��

� � ��

3

�� !��"# ��

Positioning

� Model-based, but practical work

� Not on-the-fly testing, but explicit test specifications1. tests can be generated along different methods and strategies and require

typically selection/configuration and parameterization

in order to make tests repeatable, a separate test specification is needed

2. a test model developed independently from the system model is a good choice

this test model should be based on common test concepts and by that reduce the efforts in developing/generating the tests

�� !��"# ��

Contents

� Motivation




� Summary

4

�� !��"# ��

The Test Technology TTCN-3

� Testing and Test Control Notation

� TTCN-3 is standardized by ETSI (ES 201 873 series) and the ITU-T (Z.140 series)

� Areas of Testing� Regression Testing

� Conformance and Functionality Testing

� Interoperability and Integration Testing

� Load/ Stress Testing

� Applications� Mobile communications (GSM, GPRS, UMTS, TETRA)

� Wireless LANs and MANs (Hiperlan, Hiperaccess). cordless phones (DECT)

� Broadband technologies (B-ISDN, ATM)

� Internet protocols (IPv6, SIP, Voice over IP)

� Middleware platforms (CORBA, CCM, EJB, WebServices)

� Smart Card Testing, MOST, CAN, Powertrain

�� !��"# ��

Main Aspects of TTCN-3

� Central concepts� Configuration: Static and dynamic concurrent test configurations with test

components

� Communication: Various communication mechanisms (synchronous and asynchronous)

� Control: Test case execution and selection mechanisms

� Matching: Expected vs. received data

� Parameterization: Flexibility along test campaigns

� Modules: Structuring and reuse

� Extendibility via attributes, external function, external data

� Well-defined syntax, static and operational semantics

� Different presentation formats

5

�� !��"# ��

The TTCN-3 Folklore

�� !��"# ��

TTCN-3: Test Specification and Execution

� The testing middleware� unifying the

documentation and definition of tests

� unifying the tests in IT, Internet, Telco, and embedded systems (supporting their convergence)

� unifying the test infrastructure

��

��

��$ ��% ��$�&# ��

% �'�$��# ��(��

�� (�")�# ��

��(��(��

� ��

� �(��$��& ��"��

��*��

��(��(�� (��(��

�� (��

��

��

��$ ��% ��$�&# ��

% �'�$��# ��(��

�� (�")�# ��

��(��(��

� ��

� �(��$��& ��"��

��*��

��(��(�� (��(��

�� (��

6

�� !��"# ��

Our tool set – free for education

�� !��"# ��

Contents

� Motivation




� Summary

7

�� !��"# ��

Model Driven Architecture

� Focusing on model-based system development (system centric)

�$��# ��

�&��#+ ��$�

�$��# ��(��(�&��#+ ��$�

�&��#)��

��

)�# � ��

�&��#+ ��$�

� ��

��(��&�&��# �"

��$�(��

� � �

��(��&��

� ��(��&

��# ��

,�$��"+ �� )��- ��

�� !��"# ��

Leveraging Testing into the Model-Based Paradigm

MOF Model

TestMeta Model

UMLModel Test Suite

UMLMeta Model

System

Meta-meta-model

Meta-model

Models

Data(Testable)

8

�� !��"# ��

A MOF-MM for TTCN-3

TTCN-3Metamodel

U2TP

ASN.1 Types& Values

IDLTypes

Other Types& Valuesn

Tabularformat

Graphicalformat

Textualformat

XMLTypes

Improvements in tool interoperation,model exchange, Interface to the user remains the same,but better features can be provided

�� !��"# ��

Test Design with UML 2.0 Testing Profile

� U2TP has been defined to capture all information that would be needed by different test processes within UML

� It is a testing profile based upon UML 2.0� That enables the test definition and test generation based on structural

(static) and behavioral (dynamic) aspects of UML models, and

� That is capable of inter-operation with existing test technologies for black-box testing

� It is an official OMG standard since summer 2005

9

�� !��"# ��

Sys

tem

Dev

elop

men

t

��

� � � ��

� � �Platform1

�� !� �"� ��

��!� �"� ��

��

Platform1

Test

Dev

elop

men

t

Using MDA Concepts for Testing

��

�� !� �"� ��

� � � ��

� � �Platformn

#

#

�� !� �"� ��

� � � �


�� !� �"� ��

� � � �


#

#

�� !� �"� ��

�� !$ !� ��% �

##

#

�� !� �"� ��

��

Platformn

#

�� !� �"� ��

##

�� !� �"� ��

��

Platformn

�� !� �"� ��

��

��!� �"� ��

��

Platform1

�� !��"# ��

��

��

� � �

� � � ��

� � �

��

�� !$ !� ��% �

�� !� �"� ��

Test

Dev

elop

men

tS

yste

mD

evel

opm

ent

��

#

#

��

��!� �"� ��

�� !� �"� �� Platformn

Platform1#

�� !� �"� ��

#

�� !� �"� ��

��

� � � �

� � �

� � � �

� � ��

#

Platformn

Platform1#

�� !� �"� ��

#

�� !� �"� ��

#

#�� !� �"� ��

Platformn

Platform1

Platformn

Platform1

PSM-based Test Derivation

��!�"� �� "��"��

Scenario 1:

10

�� !��"# ��

��

��

� � �

� � � ��

� � �

��

�� !$ !� ��% �

�� !� �"� ��

Test

Dev

elop

men

tS

yste

mD

evel

opm

ent

��

#

#

��

��!� �"� ��

�� !� �"� �� Platformn

Platform1#

�� !� �"� ��

#

�� !� �"� ��

��

� � �PlatformAdapter1 #

�� !� �"� ��

#

�� !� �"� ��

Platformn

Platform1

PIM-based Test Derivation with Single PST

PlatformAdapter1

Platform specificsfrom test model

,�# ��.�� $� �� !� ��+ ��# �� !'� �!��# � �� /��!�� # $��$�� (��&

Scenario 2:

�� !��"# ��

Test

Dev

elop

men

tS

yste

mD

evel

opm

ent

��

� � � ��


��

Platform1

��

� � � ��


#

#

#

#

��

Platformn

��

� � �

� � � �

� � � �

� � �PlatformAdapter1 #

#

PlatformAdaptern

� � � �

��

� !� !�!� � �� !�&

!�� ' ��

Manual PIM-based Test Derivation with Single PSTScenario 3:

11

�� !��"# ��

Integrated System and Test Development

��

� � � ��

��

� � � ��

� � �

Sys

tem

Dev

elop

men

t

EJB JavaUML2

eUML

��

Detailed Approach:

� � � �

� � �

Test

Dev

elop

men

t

Java

��

��

� � � �

��

��

��

� � � �

��

TTCN-3

��+ �� # ��

�� )�� # ��

�� # ��

��

UML2

� � � �

� � �

��

��01�( ��

�� )�� # ��

�� # ��

UML2eTML

� + �23��+ �� # ��

��

��

Java

��

��

��

�� !��"# ��

UML2 for System and Test Development

� � � �

� � �

Test

Dev

elop

men

t

Java

��

� � � ��

��

� � � ��

� � �

Sys

tem

Dev

elop

men

t

EJB Java

��

��

� � � �

��

��

��

� � � �

��

TTCN-3

��+ �� # ��

�� )�� # ��

�� # ��

��

UML2

� � � �

� � �

��

��01�( ��

�� )�� # ��

�� # ��

UML2eTML

� + �23��+ �� # ��

��

UML2eUML

��

��

Java

��

��

��

UML2

Detailed Approach:

12

�� !��"# ��

Contents

� Motivation




� Summary

�� !��"# ��

System Model to Test Model Transformation

��

� � � ��

eUML

��

� � � ��

TTCN-3

�� !� �"� ��

��

� � � ��

eTML

�� !� �"� ��

13

�� !��"# ��

System Model to Test Model Transformation

� Based on the meta-models defined for both eUML and eTML, transformation rules are proposed that allow the automated generation of eTML test model skeletons out of EUML models

� The transformation rules are defined in a model-driven way, i.e. the information contained in a source meta-model is transformed into a target meta-model.

� The advantage of that approach is that the correctness of model instances inside the source repository is ensured by the conformity to the corresponding meta-model and its invariants.

�� !��"# ��

PIM to PIT Transformation

• Derivation of test models from system models• Includes also test behaviour generation• Includes generation of graphical information for

new test model elements

cd Piz zaS hop B ehaviour

PS :Piz z aServ ice

D :D eliverer

C :Customer

getM enu(::)

M enu

A ctiv i tyIn i tia l

selectM enuItem(::)

M enuItem

m ak eSelec tion

show M enu

confir mOr der(::)

Loc alContactD ata

c ustom erId

getCustomerO rder s(::)

c ustom erId

Orders

show CustomerO rders

Ac tiv i tyFina l

getM enu

A cceptCall

cr eateM enuData

M enu

getM enuR eply

M enu

selectM enuItemAcceptCall

M enuItem

M enuItem

M anageBasket

M enuItem

selectM enuItem

Reply

confir mOr der

A cceptCall

Loc alContac tData

LocalC ontactD ata

createOr der

Loc alContac tData

cus tom erId loca lC ontac tData

i tem s ToDel iv er

to ta lPric e

noti fy O nPendingO rder

loca lC ontactD ata

i tem s ToDel iv er

to ta lPric e

confir mOrder

Reply

c ustom erId

noti fy O nPendingO rder

[true]

[fa ls e]

UML2eUML

��

a d Cu s to me r Ord e r

C E :Cu sto m e rEm u la to r

g etM e n u(: : )

M en u

s e le c tM e n uI te m(: : )

M e nu I te mma ke Se l e c ti o n

s h o w Me n u

c o n fi rm Ord e r(: : )

L o ca l Co n ta ctDa ta

c u sto me rId

g e tCu s tom e rO rd e rs(: : )

c us tom e rId

Ord e rs

s h o wC us to m e rOr de r s

Ac t i vi t y F i na l

re tu rn p a ss

« Sto p T i m erA ct i o n »

tim e r1(: : )

Ac t iv i t yIn i t i a l

« S ta rtT i m e rAct i o n »

tim e r1 (2 .0 )(: :)

« S tartT i m e rAc ti o n »

time r2 (2 .0 )(: : )

« S to p T i m e rAct i o n »

tim e r2(: :)

« S ta rtT i m erAc t i o n »

g e n e ra lTim e r(: : )

« S to p T i m erAc t i o n »


o v e ra l lDe fa ul t :Cu s to me r Emu la to r

« T i me O u tA ct i o n »

g e n e ra l T i me r

re tu rn fa i l



«T im e O utAc t i o n »ti m e r1

re tu rn fai l

Act i v i t yF i n a l

« T i m e Ou tAct i o n »t i m er2

r etu rn fa i l


« L i tera l An y OrNu l l »

*

re tu rn

in c on c


[ t ru e ]

[ fa l se ] « De fa ul tAp p l i ca t i o n »

UML2UML2eTML

��

% � ()' !�� !� �"� ��

Medini MTG

14

�� !��"# ��

Transformation Rules for Test Architecture

� All elements of the eUML model become part of the eTML model.

� Create an overall test package. Import the system model package into the test model package. Name the overall test package according to the rule4TestPackage_SystemName5 with SystemName as the name of the system model.

�� !��"# ��


� Create a package inside the TestPackage_SystemName. Name this package according to the rule4SystemNameSuites5 with SystemName as the name of the system model.

� Create a class stereotyped with <<TestContext>> inside the SystemNameSuites. Name this classaccording to the rule 4SystemNameSuite_Nr5 with SystemName as the name of the system model.

<<TestContext>>

15

�� !��"# ��

<<TestContext>>

cd PizzaServiceSuite


� Derive test configuration in the form of composite diagram from the system configuration, specifically from the eUML collaboration diagram. Put it inside the previously created SystemNameSuite_Nr class.

� The SUT is derived both from a class or a part.

� Test components are derived both from a class or a part.

PizzaShop

C :Customer

usePS

PS :PizzaServ ice

providePS

useDN

D :Deliv erer

provideDN

<<SUT>>

<<TestContext>>

Test Component

Test Component

�� !��"# ��


� Traverse the classes of test configuration assigning for each of them <<TestComponent>> or <<SUT>> stereotypes appropriately so that more composite structure diagrams can be obtained (if needed).

� Rotate the role of SUT for each class defined in the system configuration. Only testable classes may be assigned as SUT. Algorithm of assigning the <<SUT>> stereotype is provided by estimation if a class is testable. A class is testable, if it provides an interface with public methods returning values.

<<TestContext>>PizzaShopSuites_1

<<TestContext>>PizzaShopSuites_2

16

�� !��"# ��


� Create additional classes stereotyped with <<TestContext>> inside the SystemNameSuites package. Name these classes according to the rule 4SystemNameSuite_Nr5 with SystemName as the name of the system model. The number of them depends on and is equal to the number of test configurations that can be obtained.

�� !��"# ��


� Create additional classes (for each part stereotyped with <<TestComponent>> in the composite structure Diagram). Name these classes according to the rule 4ClassNameEmulator5 with ClassName as the name of the class coming from system model.

<<TestContext>> <<TestComponent>>

<<TestComponent>>

17

�� !��"# ��

Transformation Rules for Test Behaviour

� A testcase is derived from an existent activity diagram of the eUML model. Herein, the original activity diagram with all its swim-lanes and behavior are adopted. Name the diagramaccording to the rule 4DiagramName _testCase5 with DiagramName as the name of the diagram coming from system model.

�� !��"# ��

PIT to PST Transformation

• Transforms eTML models to TTCN-3 models• Test behaviour transformation rules for activities• Some adjustments to mapping rules to provide

information the test adapter (and codec) needs fortest execution

TTCN-3

�� !� �"� ��

Medini MTG

a d Cu s to me r Ord e r

C E :Cu sto m e rEm u la to r

g etM e n u(: : )

M en u

s e le c tM e n uI te m(: : )

M e nu I te mma ke Se l e c ti o n

s h o w Me n u

c o n fi rm Ord e r(: : )

L o ca l Co n ta ctDa ta

c u sto me rId

g e tCu s tom e rO rd e rs(: : )

c us tom e rId

Ord e rs

s h o wC us to m e rOr de r s


re tu rn p a ss

« Sto p T i m erA ct i o n »

tim e r1(: : )


« S ta rtT i m e rAct i o n »

tim e r1 (2 .0 )(: :)

« S tartT i m e rAc ti o n »

time r2 (2 .0 )(: : )

« S to p T i m e rAct i o n »

tim e r2(: :)

« S ta rtT i m erAc t i o n »


« S to p T i m erAc t i o n »


o v e ra l lDe fa ul t :Cu s to me r Emu la to r

« T i me O u tA ct i o n »

g e n e ra l T i me r

re tu rn fa i l



«T im e O utAc t i o n »ti m e r1

re tu rn fai l

Act i v i t yF i n a l

« T i m e Ou tAct i o n »t i m er2

r etu rn fa i l


« L i tera l An y OrNu l l »

*

re tu rn

in c on c


[ t ru e ]

[ fa l se ] « De fa ul tAp p l i ca t i o n »

UML2UML2eTML

��

��

TTCN-3 Medini Rep.

18

�� !��"# ��

Generation of TTCN-3 Module, SUT and MTC Component Types out of Test Context

cd PizzaShopTest

PizzaShopTest

«TestContext»PizzaServ iceSuite

- verdict: Verdict

+ «TestCase» PizzaShopBehavior_testCase() : Verdict

«TestComponent»CustomerEmulator

- makeSelection: boolean- generalTimer: Timer- getMenu_Timer: Timer- updateSelection_Timer: Timer- confirmOrder_Timer: Timer

usePS

«TestComponent»Deliv ererEmulator

provideDN

PizzaShop

�� !��"# ��

«interface»

PizzaShopInterface

+ confirmOrder(LocalContactData) : Order+ getCustomerOrders(integer) : Order+ getMenu() : Menu+ removeAllMenuItems() : void+ updateMenuItem(string, MenuItem) : void

Generation of Signatures and their Parameters out of Interface Operations

PizzaShopTest


- verdict: Verdict



- makeSelection: boolean- generalTimer: T imer- getMenu_Timer: Timer- updateSelection_Timer: T imer- confi rmOrder_Timer: Timer

usePS


provideDN

Del ivererNotification

PizzaShop

PizzaShopTest


- verdict: Verdict




usePS


provideDN


PizzaShop

PizzaShopTest


- verdict: Verdict




usePS


provideDN


PizzaShop

19

�� !��"# ��

Generation of Test Cases out of Activities stereotyped with <<TestCase>>

ad PizzaShopBehav ior_testCase

customerEmulator :CustomerEmulator

ActivityInitial

«StartTimerAction»

generalTimer(2.0)(::)

customerEmulator_classifierDefault :CustomerEmulator

ActivityInitial


timer1(2.0)(::)

�� !��"# ��

Generation of Test Cases Bodiesout of Activities stereotyped with <<TestCase>>

CustomerEmul

getMenu(::)

menu

initial


getMenu_Timer(::)


generalTimer(::)

«StopTimerAction»

getMenu_Timer(::)

«ControlFlow»

«ControlFlow»

«ControlFlow»

«ControlFlow»

20

�� !��"# ��

Connection to TTworkbench

TTCN-3

� � � ��

��

TTCN-3 Medini Rep.

• Issue:• Transformers are based on Medini rep.• TTworkbench is based on Eclipse EMF

• Approach: • Decoupling of the Medini TTCN-3 repository and TTworkbench

� Serialiser for the Medini TTCN-3 repository writing TTCN-3 files to TTworkbench

�� !��"# ��

Test Code Generation with TTworkbench

• Generating Java test code from TTCN-3 with TTCN-3 compiler of TTworkbench

• Compiling the Java files to jar filesready for test execution

JavaTTCN-3

� � � � � � �

��

��

�� * )+� � � � ��

TTwb��

TTCN-3 Medini Rep.

Compiled.jar File

21

�� !��"# ��

Test System

Test System

SUT: System Under Test

TE: Test Execution

TM: Management

CD

: CoD

ec

CH

: C

ompo

nent

Han

dlin

gTCI

SA: System Adaptor PA : Platform AdaptorTRI

Test System Architecture

TE: Test Execution

Executable test codegenerated from TTCN-3

�� !��"# ��

Test System

Test System


TE: Test Execution

TM: Management

CD

: CoD

ec

CH

: C

ompo

nent

Han

dlin

g

TCI



TM: Management

CH

: C

ompo

nent

Han

dlin

g

PA : Platform Adaptor

Run-timeLibrary

22

�� !��"# ��

Test System

Test System


TE: Test Execution

TM: Management

CD

: CoD

ec

CH

: C

ompo

nent

Han

dlin

gTCI



SA: System Adaptor

CD

: CoD

ec

Test adapter& codec

�� !��"# ��

Test Adapters

• Test adapters (including codec) are needed for test execution• Usually implemented in Java (but also C etc. possible)• Implement the mechanisms and en/decoding needed for the

communication with the SUT• Support for EJB and WSDL• Some influence on the TTCN-3 generation (parameters

and signatures needed for test execution in the test adapter Java

� � � � � � �

��

��

23

�� !��"# ��

Test Execution

TTworkbench - TTman

Compiled.jar File

Test Deployment & Execution

SUT

)�# # ��(��6078�"�9 �% �:

Test Logging, Tracing & Evaluation

TTworkbench – Logged Test Traces

�� !��"# ��

Evaluating the Test Results

24

�� !��"# ��

Contents

� Motivation




� Summary

�� !��"# ��

��

Integrated Modelling and Testing

��)��

+ � ��+ ��$�8 ��

� System & test centric view which is supported by tools

�$��# ��

�&��#+ ��$�

�$��# ��(��(�&��#+ ��$�

�&��#)��

��

)�# � ��

�&��#+ ��$�

� ��

)�# � ��

��+ ��$�

� ��

�$��# ��

��+ ��$�

��

�$��# ��(��( ��

+ ��$�

��

25

�� !��"# ��

Conclusion

� MDA techniques are applicable to the automated derivation of test cases� Application of PIM and PSM abstraction also to test development

� Transformers for both system development and test derivation based on MOF technology

� Seamless integration of test into system development with UML2 easier� Integrated UML2 based tool chain at platform-independent level

� Joint UML2 based editor and joint repository for system and test models

� Automated generation of test models with test behaviour from UML based system models is possible � System and test behaviour based on activity diagrams

� Test derivation rules and transformers for test behaviour based on activity diagrams is quite complex and need some further refinement

� Developed a test framework with seamless tool chain for testing EJB and WS systems� Based on UML2, TTCN-3, MOF infrastructure (applying Medini and EMF) and Eclipse

� New approach for Medini-Eclipse integration (repository serialisation)

� Proof of test concepts and demonstration with PizzaShop example

�� !��"# ��

Experiences

� Derivation of test behaviour� Mapping rules and transformer implementation get complex because of test

behaviour generation

� Behaviour in system model must be well-defined and completely described fortest behaviour generation

� Otherwise, generated test behaviour needs to be completed manually

� Test execution� Aligning the transformer and the adapter for test execution takes some effort,

e.g.

� Mapping between TTCN-3 signatures and „real“ signatures of SUT required

� Common meta-model for system and test development needs to be quitestable� Changes to meta-model require considerable effort for adjusting the transformers

� Using the same language for both system and test development is reallybeneficial� Allows for early and seamless integration of tests into the system development

26

�� !��"# ��

Thank You!

Any Further Questions?

Please consider3rd TTCN-3 User ConferenceBerlin, May 31 – June 2, 2006

�� !��"# ��

TTCN-3 could be the target, however ...

� TTCN-3 with its roots in telecommunication(and Europe)

is hardly acceptedin the software/IT environment

� Hence, idea to develop a test specification approach within UML (U2TP) and

� extend the MDA paradigmwith a test development chain (MDA++)

27

�� !��"# ��

Extend TTCN-3 with an UML access

��# ��

- ��!�($��# �

3 $��# �

�% �

��; �<

� �!�� &��= �> $ ��

% ,��

? + �

�� * )+� � ��(!� $ !$ �

�� !��"# ��

Test Design with UML 2.0 Testing Profile

� U2TP has been defined to capture all information that would be needed by different test processes

� It is a testing profile based upon UML 2.0� That enables the test definition and test generation based on structural

(static) and behavioral (dynamic) aspects of UML models, and

� That is capable of inter-operation with existing test technologies for black-box testing

� It is an official OMG standard since summer 2004

28

�� !��"# ��

Comparing U2TP and TTCN-3

Format

Test Meta Modelling

Transformation

Test Execution

Test Specification

Test Design

TTCN-3U2TP

��

��

��

�� 6��:

6-:

(–)

��6��:

- ��!�($ �1� $��!�($

� � �� ); 2@ ); 2@��

ttcn-3 test generation from system models - tu/e · pdf file1 ttcn-3 test generation from...

Documents