1

Model Based Conformance Testing for Extensible Internet Protocols

Anastasia Tugaenko

Scientific Adviser: Nikolay Pakulin, PhD

2/19

Internet Protocols Features

Internet protocols are underspecified

Internet protocols are nondeterministic

Internet protocols requirements differ in the level of obligations (MUST, SHOULD, MAY, …)

Protocol’s architecture is extensible

3/19

Architecture of Extensible Protocols

Protocol Functions

Basic

• are mandatory for all protocol implementations;

• specifies the basic protocol functionality, functions that are the basis for the protocol

Extensions• are optional, may be unsupported by implementations;

• add new functions without changing the basic protocol’s model;

• alter format of the protocol’s messages;

• alter basic protocol model, alter functionality of other extensions;

• alter protocol’s modus operandi

4/19

Conformance Testing

Many implementations of protocols from different developers are functioning in the contemporary Internet

The total reliability of data transfer substantially

depends on implementations compatibility

The basic method of attesting implementations compatibility is conformance testing (testing to conform to the standard)

Protocol’s extensions may alter protocol’s standards

5/19

Traditional Methodology of Conformance Testing

Test suite consists of formal given tests which are not connected to the implementation

Test purposes describe situations to be tested. Test purpose is realized in one or several tests

Connections between requirements and test purposes and between test purposes and their tests are informal

Implementation is considered to conform to the standard if all test purposes have been passed

6/19

A Problem with Test Suites Extensions

Basic Standard+ few extensions Test

Basic Standard+ few extensions

+ one more extension

How can we get a new test

?

assume, that we already got it

• manually change the test suite• constructing new test from a set of test units

7/19

A Problem with Adapting Test Suites to Implementations

IUT1• extension 1• extension 2

IUT2• extension 1• extension 3

IUT3• extension 4

How can we compose tests for each implementation?

Manually adapt the tests

Set a number of options in the configuration fileif (extension1) {…} else {… }

Use specification and test units and automatically compose test suites for target set of extensions from that units

8/19

Requirements for Test Suite for Extensible Protocols Testing

For basic standards: formal relation between requirements and tests automated verdict assignment about the correctness of IUT behavior automated tests generation depending on IUT responses

For extensions: easily changing of a protocol model the ability to combine specifications and tests from separate units

9/19

A Method for Extensible Protocols Testing

Constructing specification from units Specification for implementation is a collection of specifications units (basic specification, extensions specifications).

Developing tests as units The comprehensive test is a composition of units of tests for basic functionality and supported implementations.

Automated construction of model and tests for an arbitrary set of extensions

10/19

A Method for Extensible Protocols Testing. Features

Each specification unit do check the postcondition change the test state assign a verdict concerning the correctness of IUT behavior

Test sequence iterator test sequence generates automatically from test model and contract specifications

Automated coverage calculation states and transitions may be labeled with marks and branches, in this case they will be represented in the test report.

Easily changing of a protocol model

11/19

Case Study. SMTP Protocol

SMTP – Simple Mail Transfer Protocol

SMTP protocol has 7 states:

DISCONNECTED

CONNECTED

AFTER HELLO

AFTER MAIL FROM

AFTER RCPT TO

AFTER DOT

AFTER DATA

Transaction commands are allowed only from these states

12/19

Case Study. Changing SMTP Protocol Model

If AUTH extension is supported SMTP protocol has 8 states:

DISCONNECTED

CONNECTED

AFTER HELLO

AFTER AUTH

AFTER MAIL FROM

AFTER RCPT TO

AFTER DOT

AFTER DATA

Several commands are allowed only after AUTH command was successfully handled

13/19

A Method for Extensible Protocols Testing. Protocol Model

Model is an extended finite state machine

command addition

state addition

a с t i o n sstates

14/19

A Method for Extensible Protocols Testing. Protocol Model Case Study

Define 2 mappings <States → Actions>:

allowed

denied

Got 2 types of testing:

conformance (unknown transitions are considered as denied)

robustness (unknown transitions are considered as allowed)

15/19

A Method for Extensible Protocols Testing. Main Steps

Creation of requirements catalogues Designing of extensible model for the protocol Developing units with specifications and units with tests Formalization of requirements Developing test scenarios Constructing a comprehensive test for target implementation, executing it and analyzing the results

16/19

Test Suite for SMTP Protocol

Basic Specification model ~1250 strings test scenario ~250 strings

DSN extension (Delivery Status Notifications)Adds new functionality

Brings in new parameters into existing commands MAIL and RCPT, doesn’t change the protocol model.

model ~100 strings test scenario ~ 50 strings

AUTH Extension (Authentication) Alter basic functionality

Adds new state into the protocol model, adds new AUTH command or parameter for existing command MAIL, brings in new response codes.

model ~250 strings test scenario ~120 strings

17/19

Future Work

Extend the method to develop tests for other types of protocols extensions

Dynamically construct the model of target implementation taking into account the profile of supported extensions

Extend test traces and improve test report generator

18/19

Conclusion

The method for model based conformance test suite development for extensible protocols is presented

method is model based uses unit approach for modeling and specifying tests for protocols extensions uses pure Java without any extensions

The prototype of test suite for SMTP protocol with extensions was developed using presented method. The developed test suite allows to find the following types of incompliances:

protocol rules violation absence of supporting of required commands wrong reply codes to protocol commands

19/19

Thank You!