testing framework for platform independent models with...

18/6/2004 1ECOOP 2004, OSLO 1

Testing framework for Platform Independent Models with TTCN-3

Gábor Bá[email protected]

18/6/2004ECOOP 2004, OSLO 2

Topics

• Motivations• xUML and Simulation architecture• TTCN-3 language• Test configuration• Transformations• Test behavior

18/6/2004ECOOP 2004, OSLO 3

Motivations

• Fuzzy textual specifications• Graphical models

– No way of early model verification– The graphical notation set is very rich

• Executable Specifications– Completely describe the model’s dynamic behavior– Verification with simulation– Creating tests independently of the problem modeling– Model transformation for code generation (Model Driven

Architecture)

18/6/2004ECOOP 2004, OSLO 4

Goals

• Executing large number of test cases on Executable Models

• Regression testing on model level• Testing the whole model or only smaller parts• Using special test language UML doesn’t applicable for

this purpose• Early-phase tests help the implementation level test

development

18/6/2004ECOOP 2004, OSLO 5

Model Driven Architecture (MDA) and xUML

• PIM (Platform Independent Model)Expresses only business functionality and behavior

• PSM (Platform Specific Model)Defines also technology-specific details

• Domain Partitioning– Reusable, well-defined description of a problem

• The domains are in client-server relation• Relationships between domains are well defined with

bridges

18/6/2004ECOOP 2004, OSLO 6

Applicationdomain

Service 2domain

Service 1domain

Architecturaldomain

Implementation 1domain

Implementation 2domain

Communication Security

C++

Distributed

TCP / C++

On-line Bank

Bridges

Domains and bridges

18/6/2004ECOOP 2004, OSLO 7

• Subject matter partition:– DOMAINS

Separate, independentsubject matters

– Subsystems

• Relationship:– BRIDGES

Relationships between domains SS2

SS1

SS2SS1

SS3

Domains and bridges

18/6/2004ECOOP 2004, OSLO 8

How does translation work?

Simulator

FunctionalRequirements

OOAModel

TestCases

Characteristicand platform

specificRequirements

SourceCode

TestSuits

ModelTranslator

• OOA models can be simulated before implementation

• OOA models are translated by Model Compilers

18/6/2004ECOOP 2004, OSLO 9

How does simulation work?

• Create a UML Virtual Machine– As input, the Virtual Machine requires an xUML model– A special platform with general capabilities– We trust on the functionality of the VM

• The simulation runs on this Virtual Machine• The simulated model is testable

– Test interface definitions?

18/6/2004ECOOP 2004, OSLO 10

How does simulation work?

Bank Account

Customer

Transfer

UML model

UML Virtual Machine

Simulation Architecture

State Chart

18/6/2004ECOOP 2004, OSLO 11

What do we have?

Kennedy Kennedy –– Carter toolsetCarter toolset

• iUML - modeler- simulator, Simulation architecture

• iCCG - Configurable Code Generatorfor creation new architectures

Test DescriptionTest Description

• TTCN-3 (Testing and Test Control Notation) a standardized test language

18/6/2004ECOOP 2004, OSLO 12

Introduction to TTCN-3

• Flexible Language• Applicable to the specification of all types of reactive

system tests.• The core language has a C like textual syntactic (easy to

read for programmers as well)• Operations for procedure-based and message-based

communication• Specification of dynamic concurrent testing configurations• Data and signature templates with matching mechanism

18/6/2004ECOOP 2004, OSLO 13

TTCN-3 language overview

• TTCN-3 static definitions– Data types, special types– Templates and constants– Test components, test port definitions

• Dynamic behavior descriptions– Functions, test cases– Test configurations– Behavior trees, timers, verdicts– Executing and controlling test cases

18/6/2004ECOOP 2004, OSLO 14

TTCN-3 ExecutorImported ASN.1

Modules

TTCN-3 Abstract

Test Suite

Other Presentation

Formats

Compiler

Base Library

Source Code

Test Port(s)

Compiler

Executable Test Suite

Skeleton

User Code

18/6/2004ECOOP 2004, OSLO 15

Basic language elements• Module header

– Module name, object identifier, module (external) parameters

• Definitions part– Data types and function signatures– Constants– Templates– Port and component types– Functions– Test Cases

• Control part– Control of execution of test cases

18/6/2004ECOOP 2004, OSLO 16

Basic language elements• Module header

– Module name, object identifier, module (external) parameters

• Definitions part– Data types and function signatures– Constants– Templates– Port and component types– Functions– Test Cases

• Control part– Control of execution of test cases

AutomaticGeneration

18/6/2004ECOOP 2004, OSLO 17

A TTCN-3 example

testcase ConnectionTest() runs on MTCTypetimer T1:=2;ttcn_port.send(Connect);T1.start;alt{[]ttcn_port.receive(Connection_Acknowledgement){setverdict(pass);[]T1.timeout{setverdict(fail);}

}

t empl at e Connect _Message Connect : ={connect i on_t ype: =per si st ent

}

t empl at e Connect _Message Connect : ={connect i on_t ype: =per si st ent ,i p_addr ess : = “ 1. 1. 1. 1” ,t cp_por t : = 1000

}

Test Data on PIM level Test Data on implementation level

18/6/2004ECOOP 2004, OSLO 18

Obvious question:Why not connect these technologies?

Modeler Tool

UML VM

PIM

Sim. Architecture

TTCN-3

?

•Interface definition between the two technologies•Hide the communication problems from the testers and analysts

TestersAnalysts

18/6/2004ECOOP 2004, OSLO 19

Testing and debugging executable models

• PIMs suffer from testability problems in the area of– Observability– Controllability

• We need an interface on UML level to reach the inner part of the model

UML VM

PIM

Sim. Architecture

Co

mm

um

icat

ion

Inte

rfac

e

Tester/Debugger

18/6/2004ECOOP 2004, OSLO 20

Structure of the PIM tester

TTCN-3

Test data

Test behavior

TesterDomain Domain

operations

TestedDomain(s)

BridgesPIM tester

Physical layer

Logical layer

xUML

TTCN-3 Executor UML Virtual Machine

Communication Interface

18/6/2004ECOOP 2004, OSLO 21

Transformation of the xUML model

TTCN-3 Static Definition

Model transformation

xUMLPIM model

TTCN-3 Test Port

Test Port forSim. Architecture

Basic testdata

•The Platform Independent model is the source of the test definition•Based on the bridge definition of the tester domain

18/6/2004ECOOP 2004, OSLO 22


• Black-box testing of the model• Message-based communication between the tester and

the model• Using special TAGs to note the border of the model• Creating sending and receiving messages from the UML

model

18/6/2004ECOOP 2004, OSLO 23


• Based on the PIM model of the analysed problem• Code generation for TTCN-3 and UML Virtual Machine• Client-server based communication interface

PIMUML VMCode

Sim. ModelTranslator

TTCN-3Translator

DataDefinitions

PortDefinitions

TestSuite

18/6/2004ECOOP 2004, OSLO 24

Behavior of the PIM tester

TTCN-3

Test data

Test behavior

TTCN-3 Port

TesterDomain

UML Port

Domainoperations

Terminators TestedDomain(s)

Terminators

Bridges

map( mtc:uml_mport, system:uml_mport );uml_mport.send(packet1);alt{

[]uml_mport.receive;}

18/6/2004ECOOP 2004, OSLO 25


TTCN-3

Test data

Test behavior

TTCN-3 Port

TesterDomain

UML Port

Domainoperations

Terminators

Terminators

Bridges

void UML__PORT::outgoing_send(const Packet& send_par){

packet=EncodePacket();sendto(send_sock,packet,…);

}

TestedDomain(s)

18/6/2004ECOOP 2004, OSLO 26


TTCN-3

Test data

Test behavior

TTCN-3 Port

TesterDomain

UML Port

Domainoperations

Terminators

Terminators

Bridges

void DecodegetPacket(char* message){…D6_TERM4_Packet(6,0,NULL,user_name);}

TestedDomain(s)

18/6/2004ECOOP 2004, OSLO 27


TTCN-3

Test data

Test behavior

TTCN-3 Port

TesterDomain

UML Port

Domainoperations

Terminators

Terminators

Bridges

TestedDomain(s)

18/6/2004ECOOP 2004, OSLO 28

Ongoing work

• Model level test of an Alarm Mediation application• Derive static test data definitions automatically• Describing test data manually based on the derived

definitions • Run the implementation level test on the final product

18/6/2004ECOOP 2004, OSLO 29

Summary

• TTCN-3 allows:– Automatic execution of large number of test cases– Regression test on different version of the model

• Compiler provides automatic creation of:– Static data definition for TTCN-3– Port definition for TTCN-3– Port definition for Simulation Architecture

• Test development simultaneously with modeling

18/6/2004ECOOP 2004, OSLO 30

Thank you for the attention!

18/6/2004ECOOP 2004, OSLO 31

Appendix A

TTCN-3 language elements

18/6/2004ECOOP 2004, OSLO 32

An example: “ Hello, World!” in TTCN-3

modul e MyExampl e{t ype por t PCOType message{

i nout char st r i ng;}t ype component MTCType{

por t PCOType MyPCO;}t est case Hel l oW( ) r uns on MTCType{

MyPCO. st ar t ;MYPCO. send( “ Hel l o, Wor l d! ” ) ;MyPCO. st opset ver di ct ( pass) ;

}cont r ol l {

execut e( Hel l oW( ) ) ;}

}

18/6/2004ECOOP 2004, OSLO 33

Structured Types Example – record, record of

/ / r ecor d t ype def i ni t i on

t ype r ecor d MyMessageType1

{

i nt eger f i el d1,

bool ean f i el d2

}

t ype r ecor d MyMessageType2

{

MyMessageType1 f i el d1,

bool ean f i el d2

}

/ / l i s t t ype def i ni t i on

t ype r ecor d of MyMessageType2 MyLi st Type;

18/6/2004ECOOP 2004, OSLO 34

Data Templates

• Special data values for sending and receiving messages• Templates can be created for simple types as well as

compound types

t empl at e MyMessageType1 MyMessage: ={

f i el d1 : = 2,f i el d2 : = t r ue

}

/ / Exampl e use of t hi s t empl at e ( i n t he dynami c par t of t he t est descr i pt i on)PORT1. send( MyMessage) ;

18/6/2004ECOOP 2004, OSLO 35

Appendix B

Translation of the xUML model

18/6/2004ECOOP 2004, OSLO 36

TTCN-3 Static definition

• Module declaration• TTCN-3 structures represent the parameters of terminators

and domain scoped operations• Port declaration for sending and receiving messages• Main component declaration

18/6/2004ECOOP 2004, OSLO 37


type record Tr apv1_r ecei ved_T{

charstring i n_ent er pr i se,

charstring i n_agent _addr ess,

Gener i c_Tr ap_Type i n_gener i c_t r ap_t ype,

integer i n_speci f i c_t r ap_t ype,

Name_Val ue_Pai r _l i s t i n_var i abl e_l i s t

} ;

define bridge SNMP:SNMP_APP1_Trapv1_received

i nput \

i n_ent er pr i se: OI D, \

i n_agent _addr ess: I P_Addr ess, \

i n_gener i c_t r ap_t ype: Gener i c_Tr ap_Type, \

i n_speci f i c_t r ap_t ype: I nt eger , \

{ i n_var i abl e_l i s t } : Name_Val ue_Pai r

Out put

…

18/6/2004ECOOP 2004, OSLO 38


type enumerated Gener i c_Tr ap_Type{ col dSt ar t ( 0) ,

war mSt ar t ( 1) ,

l i nkDown( 2) ,

l i nkUp( 3) ,

…

} ;

def i ne br i dge SNMP: SNMP_APP1_Tr apv1_r ecei ved

i nput \



in_generic_trap_type:Generic_Trap_Type,\


{ i n_var i abl e_l i s t } : Name_Val ue_Pai r

Out put

…

18/6/2004ECOOP 2004, OSLO 39


type record Name_Val ue_Pai r {

charstring name,

charstring val ue_

} ;

type record of Name_Val ue_Pai r

Name_Val ue_Pai r _l i s t ;

def i ne br i dge SNMP: SNMP_APP1_Tr apv1_r ecei ved

i nput \



i n_gener i c_t r ap_t ype: Gener i c_Tr ap_Type, \


{in_variable_list}:Name_Value_Pair

Out put

…

18/6/2004ECOOP 2004, OSLO 40

TTCN-3 Test Port Definition

• Automatically generated from the UML model• Socket based communication with the xUML model• Automatic encoding and decoding of messages

voi d UML__PORT: : out goi ng_send( const User I nf oRequest __T& send_par ) {char out buf [ 1024] ;i nt l en = 0;char t mp_t ext [ 1024] ;st r cpy ( &out buf [ l en] , " User I nf oRequest _T, " ) ;l en += st r l en( " User I nf oRequest _T, " ) ;st r cpy( &out buf [ l en] , send_par . i n__user __name( ) ) ;l en += st r l en( send_par . i n__user __name( ) ) ;nbyt es = sendt o ( send_sock, out buf , …) ;

}

def i ne br i dge CHATAPP: USERD1_User I nf oRequesti nput \

i n_user _name: TextOut put…

18/6/2004ECOOP 2004, OSLO 41

Test Port Definition for UML

• Automatically generated from the UML model• Socket based communication with the xUML model• Automatic encoding and decoding of messages

voi d DecodeUser I nf oRequest _T( char * message) {char t oken[ 256] ;char * i n_user _name;sscanf ( message, " %[ ^ , ] , %s" , t oken, message) ;i n_user _name = ( Text ) cgen_mal l oc( st r l en( t oken) + 1) ;st r cpy( i n_user _name, t oken ) ;D2_USERD1_User I nf oRequest ( 2, 0, NULL, i n_user _name) ;

}

def i ne br i dge CHATAPP: USERD1_User I nf oRequesti nput \

i n_user _name: TextOut put…

testing framework for platform independent models with...

Documents