telematics group university of göttingen, germany integrated application of ttcn dieter hogrefe
TRANSCRIPT
Telematics groupUniversity of Göttingen, Germany
Integrated Application of TTCN
Dieter Hogrefe
2SG17 workshop, Sep. 2003
Telematics groupUniversity of Göttingen, Germany
B A X
Y (0,5ms]
A.send(X); myT.start(5E-3);
alt { [] A.receive(Y); …
[] myT.timeout; …}
...
#delay(4)
state2
Y
''
X
state1
3SG17 workshop, Sep. 2003
Telematics groupUniversity of Göttingen, Germany
Peculiarities of Communication Software (Protocols) vs. other types Software
1) Establishment of states (connection, interface, process) -> state oriented modelling
2) Precisely defined and standardized message formats
3) The software has to interwork with a large amount of (unknown) implementations
4) Distribution
4SG17 workshop, Sep. 2003
Telematics groupUniversity of Göttingen, Germany
Consequences from the peculiarities
1) Existence of detailed specifications (Internet Draft, ETSI Standard, ITU-T Standard, Proprietary, …)Base Standards and Test Specifications
2) Specialized (state oriented) specification languages and methods have been developed
3) Testing against reference implementations (protocol testers
4) Testing by Interop events, e.g.IPv6 Event, Mandelieu, 23-27 Sep.
5SG17 workshop, Sep. 2003
Telematics groupUniversity of Göttingen, Germany
Specialized specification languages, methods and tools
1) Languages:SDL, MSC, TTCN, ASN.1, (UML in the future?)
2) Methods:- State based modelling- State based verification (e.g. state space exploration)- Systematic test case development based on specialized methodology (ISO 9646, UIO, …)
3) Tools:Telelogic TAU, Solinet SAFIRE, DANET, DaVinci, TestingTech, …
6SG17 workshop, Sep. 2003
Telematics groupUniversity of Göttingen, Germany
Testing Language http://www.etsi.org/ptcc/ptccttcn3.htm
• Testing and Test Control Notation • Versions 1 and 2 developed by ISO SC21 WG3
(1984 - 1997) as part of the widely-used ISO/IEC 9646 conformance testing standard– ISO/IEC 9646-3 (edition 2) and ITU-T X.292
• Version 3 developed by ETSI TC MTS (1998 - 2000)– Specialist Task Forces STF 133 & STF 156
7SG17 workshop, Sep. 2003
Telematics groupUniversity of Göttingen, Germany
Testing Language
• One test notation for many testing applications• Cheaper education and training costs• Facilitates the application of a common methodology• Easier maintenance of test suites• Off-the-shelf tools• Universally understood syntax and operational
semantics• Tests concentrate on the meaning of the test• Constant maintenance of the language
8SG17 workshop, Sep. 2003
Telematics groupUniversity of Göttingen, Germany
Motivation for developing
• Modernization – technology has changed since TTCN was first developed
• Wider scope of application– should be applicable to many kinds of test applications
not just conformance (development, system, integration, iop …)
• Harmonization– should be the first choice for test specifiers, implementors
and users both for standardized test suites ...– … and as a generic solution in industrial product
development
9SG17 workshop, Sep. 2003
Telematics groupUniversity of Göttingen, Germany
Testing Language
– Kinds of testing to which TTCN-3 could be applied • - Conformance - Interoperability• - Configuration - Compatibility• - Performance - Stress• - Robustness - Integration• - Functional - Load• - Reliability - Fault tolerance• - Scalability - Degraded mode• - Unit - Product• - Development - Design• - Interface - System
10SG17 workshop, Sep. 2003
Telematics groupUniversity of Göttingen, Germany
Main Capabilities of
• Dynamic concurrent testing configurations• Various communication mechanisms (synch and asynch)• Data and signature templates with powerful matching
mechanisms• Specification of encoding information• Display and user-defined attributes• Test suite parameterization• Test case control and selection mechanisms• Assignment and handling of test verdicts• Harmonized with ASN.1• Different presentation formats• Well-defined syntax, static semantics and operational
semantics
11SG17 workshop, Sep. 2003
Telematics groupUniversity of Göttingen, Germany
Core Language
PresentationFormat3
PresentationFormatn
TTCN-3 Core Language
Text format
Graphical Format
Tabular Format
• Core format is a text based language
• Core can be viewed as text or in various presentation formats
• Tabular format for conformance testing
• Other standardized formats in the future
• Proprietary formats
• Graphical format for visual overview
12SG17 workshop, Sep. 2003
Telematics groupUniversity of Göttingen, Germany
Use with other Languages
TTCN-3 Core Language
Other types & Values2
Other types & Valuesn
ASN.1 Types & Values
• TTCN can be integrated with other 'type and value' systems
• Harmonization possible with other type and value systems (possibly from proprietary languages)
• Fully harmonized with ASN.1 (1997)
13SG17 workshop, Sep. 2003
Telematics groupUniversity of Göttingen, Germany
Major Elements of
Test Behaviour
Test System Architecture
• Built-in and user-defined generic data types (e.g., to define messages, service primitives, information elements, PDUs)
TTCN-3 Core
Test Data
Data Types
• Specification of the dynamic test system behaviour
• Definition of the components and communication ports that are used to build various testing configurations
• Actual test data transmitted/received during testing
14SG17 workshop, Sep. 2003
Telematics groupUniversity of Göttingen, Germany
Example Core (Text)
function PO49901(integer FL) runs on MyMTC {
L0.send(A_RL3(FL,CREF1,16));TAC.start;alt {
[] L0.receive(A_RC1((FL+1) mod 2)) { TAC.cancel;
verdict.set(pass)
}
[] TAC.timeout { verdict.set(inconc)
}
[] any.receive { verdict.set(fail) }
}END_PTC1() // postamble as function call
}
15SG17 workshop, Sep. 2003
Telematics groupUniversity of Göttingen, Germany
Example Tabular
Test Case Definition
Name : MyTestcase
Group :
Purpose : Example Testcase
System I/f :
MTC Type : MyComponentType
Comments :
Name Type Initial Value Comments
MyVar INTEGER 0
Behaviour Definition Comments alt { [ ] MyPort.receive(Msg); [ ] : }
DetailedComments:
16SG17 workshop, Sep. 2003
Telematics groupUniversity of Göttingen, Germany
Example Graphical
17SG17 workshop, Sep. 2003
Telematics groupUniversity of Göttingen, Germany
Use of for OSP
• Open Settlement Protocol (OSP)
– EP TIPHON XML-based protocol
– uses TTCN-3 Core Language
18SG17 workshop, Sep. 2003
Telematics groupUniversity of Göttingen, Germany
Use of for SIP Tests
• IETF Session Initiation Protocol (SIP)
– RFC 3261
– EP TIPHON SIP Profile
• Work was done by STF196 in EP TIPHON– Nokia, Ericsson, GMD Fokus (Testing Tech), ACACIA,
FSCOM, GN Nettest– work due to finish December 2003– executable tests running
• TTCN-3 will be used by 3GPP for SIP tests in 2004
19SG17 workshop, Sep. 2003
Telematics groupUniversity of Göttingen, Germany
Use of for IP Tests
ETSI has work items on developing test suites for
• IPv6 Core
• IPv6 Security
• IPv4-to-v6 Transition
Currently under review by DG Enterprise at the Commission for funds from the eEurope initiative
WIs for SIGTRAN M3UA, SCTP will be opened in October
20SG17 workshop, Sep. 2003
Telematics groupUniversity of Göttingen, Germany
Methodology
21SG17 workshop, Sep. 2003
Telematics groupUniversity of Göttingen, Germany
Methodology
Process P1
Process P2
SDL
Implementation Under Test
(IUT)
ENV1!a
?TIMEOUT Timer
START Timer ENV1?c
ENV1!d ENV2?b
?TIMEOUT Timer
Behaviour Description
fail fail pass
Verdi
ct
TTCN derive
is basis for
test Tester
is basis for
22SG17 workshop, Sep. 2003
Telematics groupUniversity of Göttingen, Germany
Methodology
• UML testing profile
– TTCN-3 compatible
– JUnit compatible
• UML-TTCN-3 profile?
23SG17 workshop, Sep. 2003
Telematics groupUniversity of Göttingen, Germany
Goals of the UML Testing Profile
• Definition of a testing profile to capture all information that would be needed by different test processes
– To allow black-box testing (i.e. at UML interfaces) of computational models in UML
• A testing profile based upon UML 2.0
– That enables the test definition and generation of structural (static) and behavioral (dynamic) aspects of UML models, and
– That is capable of inter-operation with existing test technologies for black-box testing
• Define
– Test purposes for computational UML models, which should be related to relevant system interfaces
– Test components, test configurations and test system interfaces
– Test cases in an implementation independent manner
24SG17 workshop, Sep. 2003
Telematics groupUniversity of Göttingen, Germany
Roots of the Testing Profile P
roto
col T
esti
ng
like
TT
CN
-3
Soft
war
e T
esti
ng
like
JU
nit,
TE
T, e
tc.
MSC-2000 UML 1.x SDL-2000MSC-2000
UML 2.0Graphical Format
of TTCN-3
UML Testing Profile
• Test control• Wildcards• Defaults• Test components
• Arbiter• Validation actions • Data pools
25SG17 workshop, Sep. 2003
Telematics groupUniversity of Göttingen, Germany
Concepts of the Testing Profile
• Test architecture
– Test structure, test components and test configuration
• Test data
– Data and templates used in test procedures
• Test behavior
– Dynamic aspects of test procedures
• Test time
– Time quantified definition of test procedures
26SG17 workshop, Sep. 2003
Telematics groupUniversity of Göttingen, Germany
Implementations under Development
• Eclipse Project Hyades on an Open Source Trace and Test Framework
– The test part is based on the U2TP standalone metamodel
• Telelogic Tau G2
– The test part is based on the U2TP profile
• Microsoft Visual Studio
• ITEA Project on Advanced Test Methods and Tools TTmedal
27SG17 workshop, Sep. 2003
Telematics groupUniversity of Göttingen, Germany
In Summary U2TP
• UML Testing Profile provides specification means for test artifacts of systems from various domains
• Enhances UML with concepts like test configuration, test components, SUT, verdict and default
• Seamlessly integrates into UML: being based on UML metamodel, using UML syntax
• Adopted at OMG Technical Meeting, Paris, June 2003
• http://www.fokus.fraunhofer.de/u2tp/
28SG17 workshop, Sep. 2003
Telematics groupUniversity of Göttingen, Germany
The End
29SG17 workshop, Sep. 2003
Telematics groupUniversity of Göttingen, Germany
Questions and Answers