![Page 1: LO-5: Testing Lecturer: Ricardo Gonçalves. 2 Testing Overview Conformance Testing –Definition –Methodologies –Application Architecture Interoperability](https://reader031.vdocuments.us/reader031/viewer/2022013012/5697c00d1a28abf838cc992f/html5/thumbnails/1.jpg)
LO-5: Testing
Lecturer: Ricardo Gonçalves
![Page 2: LO-5: Testing Lecturer: Ricardo Gonçalves. 2 Testing Overview Conformance Testing –Definition –Methodologies –Application Architecture Interoperability](https://reader031.vdocuments.us/reader031/viewer/2022013012/5697c00d1a28abf838cc992f/html5/thumbnails/2.jpg)
2
• Testing Overview• Conformance Testing
– Definition– Methodologies – Application Architecture
• Interoperability Checking– Definition– Methodologies – Application Architecture
Contents
![Page 3: LO-5: Testing Lecturer: Ricardo Gonçalves. 2 Testing Overview Conformance Testing –Definition –Methodologies –Application Architecture Interoperability](https://reader031.vdocuments.us/reader031/viewer/2022013012/5697c00d1a28abf838cc992f/html5/thumbnails/3.jpg)
3
What is Testing?
Testing is the process used to help identify: – Correctness – Completeness– Security– Quality
… of developed computer software.
Testing is a process of technical investigation, performed on behalf of stakeholders, that is intended to reveal quality-related information about the product with respect to the context in which it is intended to operate.
![Page 4: LO-5: Testing Lecturer: Ricardo Gonçalves. 2 Testing Overview Conformance Testing –Definition –Methodologies –Application Architecture Interoperability](https://reader031.vdocuments.us/reader031/viewer/2022013012/5697c00d1a28abf838cc992f/html5/thumbnails/4.jpg)
4
Testing life cycle
• Software testing life cycle:
Requirement Analysis
Test Planning
Test Development
Test Execution
Test Reporting
Retesting the Defects
![Page 5: LO-5: Testing Lecturer: Ricardo Gonçalves. 2 Testing Overview Conformance Testing –Definition –Methodologies –Application Architecture Interoperability](https://reader031.vdocuments.us/reader031/viewer/2022013012/5697c00d1a28abf838cc992f/html5/thumbnails/5.jpg)
5
Types of Testing
There are many types of tests, like:
• Performance;
• Robustness;
• Behaviour;
• Functions;
• Conformance;
• Interoperability;
![Page 6: LO-5: Testing Lecturer: Ricardo Gonçalves. 2 Testing Overview Conformance Testing –Definition –Methodologies –Application Architecture Interoperability](https://reader031.vdocuments.us/reader031/viewer/2022013012/5697c00d1a28abf838cc992f/html5/thumbnails/6.jpg)
Conformance Testing (CT)
![Page 7: LO-5: Testing Lecturer: Ricardo Gonçalves. 2 Testing Overview Conformance Testing –Definition –Methodologies –Application Architecture Interoperability](https://reader031.vdocuments.us/reader031/viewer/2022013012/5697c00d1a28abf838cc992f/html5/thumbnails/7.jpg)
7
Conformance Testing (CT)
• ISO/IEC TR 13233 defined conformance and conformity as “fulfilment by a product, process or service of all relevant specified conformance requirements”.
• ISO/IEC DIS 10641 defined conformance testing as "test to evaluate the adherence or no adherence of a candidate implementation to a standard”.
![Page 8: LO-5: Testing Lecturer: Ricardo Gonçalves. 2 Testing Overview Conformance Testing –Definition –Methodologies –Application Architecture Interoperability](https://reader031.vdocuments.us/reader031/viewer/2022013012/5697c00d1a28abf838cc992f/html5/thumbnails/8.jpg)
8
Conformance Testing (CT)
To apply the conformance testing to an implementation, the standards or specifications, must specify the requirements or criteria for conformance. In general this are done in a conformance clause or conformance statement.
Also, to help in the conformance testing, some standards describe methodologies and platforms to implement the conformance testing.
![Page 9: LO-5: Testing Lecturer: Ricardo Gonçalves. 2 Testing Overview Conformance Testing –Definition –Methodologies –Application Architecture Interoperability](https://reader031.vdocuments.us/reader031/viewer/2022013012/5697c00d1a28abf838cc992f/html5/thumbnails/9.jpg)
9
CT - Methodologies
• ISO 9646: “Conformance Testing Methodology and Framework”
• ISO10303 30’s parts a Conformance testing methodology and framework
• ETS 300 406 (Methods for Testing and Specification (MTS), Protocol and profile conformance testing specifications, Standardization methodology
![Page 10: LO-5: Testing Lecturer: Ricardo Gonçalves. 2 Testing Overview Conformance Testing –Definition –Methodologies –Application Architecture Interoperability](https://reader031.vdocuments.us/reader031/viewer/2022013012/5697c00d1a28abf838cc992f/html5/thumbnails/10.jpg)
10
CT General concepts
• System Under Test (SUT)
• Implementation Under Test (IUT)
• Abstract Test Suits (ATS).
SUT
IUTIs verified and
certified via
adopts
ATS Standard
![Page 11: LO-5: Testing Lecturer: Ricardo Gonçalves. 2 Testing Overview Conformance Testing –Definition –Methodologies –Application Architecture Interoperability](https://reader031.vdocuments.us/reader031/viewer/2022013012/5697c00d1a28abf838cc992f/html5/thumbnails/11.jpg)
11
CT Abstract Test Suits(ATS)
• ATS is defined by a set of Abstract Test Cases (ATC) describing in a conceptual format
• With the ATS defined for one AP, the Conformance Testing procedures can be executed on the IUT
• ATS needs to be instantiated creating the Executable Test Cases (ETC), which are the data and commands to be used to execute the test described in computer format.
![Page 12: LO-5: Testing Lecturer: Ricardo Gonçalves. 2 Testing Overview Conformance Testing –Definition –Methodologies –Application Architecture Interoperability](https://reader031.vdocuments.us/reader031/viewer/2022013012/5697c00d1a28abf838cc992f/html5/thumbnails/12.jpg)
12
CT Abstract Test Suits(ATS)
APGenerate set ofTest Purposes (TP)
Identify set ofTest Objectives (TO)
AP domainexperts
Derived fromInformation Requirements (IR)
ImplementationRequirements (IR)
+
for PreandPostprocessors
For PostProcessorsonly
Plan Abstract Test Cases (ATC) & Map TP to each ATCPlan Abstract Test Cases (ATC) & Map TP to each ATC
Input Application Semantics (IAS)Test Case (TC) for bothPost and Pre processorsGenerates
TC =(Input Data Specification – IDS,Veredict Criteria - VC)
TCs=(IDS, VC)UsingConstruct simple ATCs for Post and Pre processors
TCs=(IDS, VC)UsingConstruct simple ATCs for Post and Pre processors
Add ATC to ATSAdd ATC to ATS
ATS covers the TPsAs a whole for eachConformance Class ?
No
ATSdone!
TP coverageisadequateYes
![Page 13: LO-5: Testing Lecturer: Ricardo Gonçalves. 2 Testing Overview Conformance Testing –Definition –Methodologies –Application Architecture Interoperability](https://reader031.vdocuments.us/reader031/viewer/2022013012/5697c00d1a28abf838cc992f/html5/thumbnails/13.jpg)
13
CT – “A Methodology”
Test preparation
Analysis of the results
Report production
Test execution
Reports
SUT
ATS
ISO 10303, Conformance testing methodology
![Page 14: LO-5: Testing Lecturer: Ricardo Gonçalves. 2 Testing Overview Conformance Testing –Definition –Methodologies –Application Architecture Interoperability](https://reader031.vdocuments.us/reader031/viewer/2022013012/5697c00d1a28abf838cc992f/html5/thumbnails/14.jpg)
14
CT – Application example
EXPRESSModel
XMLDATA
Is in conformance??
Retailer
Manufacturer
Model of exchanged messages
Conformance testing of messages
Request quotation received well format
<Request_for_quotation> <buyer>Tomas Smith</buyer> <sellller>Furniture Shop</sellller> <product>Blue Chair</product> <quantity>20</quantity></Request_for_quotation>
ENTITY Request_for_quotation buyer: STRING seller: STRING product: STRING quantity: STRINGRequest_for_quotation
<Request_for_quotation> <buyer>Tomas Smith</buyer> <seller>Furniture Shop</seller> <product>Blue Chair</product> <quantity>20</quantity></Request_for_quotation>
![Page 15: LO-5: Testing Lecturer: Ricardo Gonçalves. 2 Testing Overview Conformance Testing –Definition –Methodologies –Application Architecture Interoperability](https://reader031.vdocuments.us/reader031/viewer/2022013012/5697c00d1a28abf838cc992f/html5/thumbnails/15.jpg)
15
CT – Application Architecture
Knowledge Base
Schematron Rules
XSD Model
XSL Math Functions
JAVAFunctions
ApplicationEngine
ApplicationEngine
SAX
JAV
A V
M
XALAN
ReportReport
Conformance Test Service
WSDL InterfaceWSDL Interface
Client Application
Execute Test
Test Result
![Page 16: LO-5: Testing Lecturer: Ricardo Gonçalves. 2 Testing Overview Conformance Testing –Definition –Methodologies –Application Architecture Interoperability](https://reader031.vdocuments.us/reader031/viewer/2022013012/5697c00d1a28abf838cc992f/html5/thumbnails/16.jpg)
Interoperability Checking (IC)
![Page 17: LO-5: Testing Lecturer: Ricardo Gonçalves. 2 Testing Overview Conformance Testing –Definition –Methodologies –Application Architecture Interoperability](https://reader031.vdocuments.us/reader031/viewer/2022013012/5697c00d1a28abf838cc992f/html5/thumbnails/17.jpg)
17
Interoperability Checking
• IEEE define interoperability as "the ability of two or more systems or components to exchange information and to use the information that has been exchanged.”
• Interoperability testing is the act of determining if end-to-end functionality between (at least) two communicating systems is as required by those base systems’ standards. “ETSI”
![Page 18: LO-5: Testing Lecturer: Ricardo Gonçalves. 2 Testing Overview Conformance Testing –Definition –Methodologies –Application Architecture Interoperability](https://reader031.vdocuments.us/reader031/viewer/2022013012/5697c00d1a28abf838cc992f/html5/thumbnails/18.jpg)
18
IC Methodology – General concepts
• System Under Test (SUT) • Implementation Under Test (IUT)• Abstract Test Suits (ATS)• Test Suite Structure and Test Purposes
(TSS&TP) • Interoperable Functions Statement (IFS) • Implementation eXtra Information for
Testing (IXIT) • Executable Test Suite (ETS)
![Page 19: LO-5: Testing Lecturer: Ricardo Gonçalves. 2 Testing Overview Conformance Testing –Definition –Methodologies –Application Architecture Interoperability](https://reader031.vdocuments.us/reader031/viewer/2022013012/5697c00d1a28abf838cc992f/html5/thumbnails/19.jpg)
19
IC – “A Methodology”
Base
(Protocol)
Standards
TSS & TP ATS
IFS and IXIT
Manual TestingImplementationTest selection and
parameterization
Development of Interoperability Test Specifications
ETSI activities
non-ETSI activities
Filled-in IFS
and IXIT
RE EUT
SUT
• TS 102 237-1; Interoperability test methods & approaches; Part 1: Generic approach to interoperability testing
From: In Pursuit of Interoperability (ITJ2582) - Scott Moseley; Steve Randall; Anthony Wiles.
![Page 20: LO-5: Testing Lecturer: Ricardo Gonçalves. 2 Testing Overview Conformance Testing –Definition –Methodologies –Application Architecture Interoperability](https://reader031.vdocuments.us/reader031/viewer/2022013012/5697c00d1a28abf838cc992f/html5/thumbnails/20.jpg)
20
IC – Application Architecture
Control, Logging, Coordination etc.
Encoding/DecodingSystem
TTCN-3 Runtime System
Adaption Layer
TTCN-3 Runtime Interface (TRI)
TTCN-3 Control Interface (TCI)
TTCN-3 Tester
Test Manager
ExecutableTest Suite
Underlying Protocol Stack
TTCN-3 Test Suite
TTCN-3Compiler
Completed ICS
From: In Pursuit of Interoperability (ITJ2582) - Scott Moseley; Steve Randall; Anthony Wiles.