Academic Year 2013/2014
Warsaw University of Technology
Faculty of Electronics and Information Technology
Electrical and Computer Engineering
BACHELOR OF SCIENCE THESIS
Michał Kurek
Małgorzata Purwin
Automatic Testing of Web Services Based on WSDL
Document
Supervisor
dr inż.Ilona Bluemke
Evaluation: .............................................
................................................................
Signature of the Head
of Examination Committee
2
Electrical and Computer Engineering
Date of Birth: 1990.08.24
Starting Date of Studies: 2010.02.22
Curriculum Vitae
I was born in Warsaw on 24 August. In 2006 I started studying at XV LO im. Narcyzy
Żmichowskiej in Warsaw at class with extended curriculum in mathematics, geography and
English. I got my high school certificate in 2009. I applied successfully to Warsaw University
of Technology to faculty of Electronics and Information Technology. I began my studies at
summer 2010. During my studies, I choose the specialization of Computer Systems and
Networks in Institute of Computer Science.
.......................................................
Signature of the Student
Bachelor of Science Examination
Examination was held on: ................................................................................................... 2014
With the result: ............................................................................................................................
Final Result of the Studies: .........................................................................................................
Suggestions and Remarks of the B.Sc. Examination Committee: ..............................................
......................................................................................................................................................
......................................................................................................................................................
3
Electrical and Computer Engineering
Date of Birth: 1990.08.24
Starting Date of Studies: 2010.10.22
Curriculum Vitae
I was born on 24 August 1990 in Warsaw. In 2006, I started studying at XLVII LO im.
Stanisława Wyspiańskiego in Warsaw at class with extended curriculum in mathematics,
physics and computer science. I got my high school-leaving certificate in 2009. I applied to
Warsaw University of Technology to faculty of Electronics and Information Technology and I
was accepted. I began my studies at summer 2010. During my studies, I choose the
specialization of Computer Systems and Networks in Institute of Computer Science.
.......................................................
Signature of the Student
Bachelor of Science Examination
Examination was held on: ................................................................................................... 2014
With the result: ............................................................................................................................
Final Result of the Studies: .........................................................................................................
Suggestions and Remarks of the B.Sc. Examination Committee: ..............................................
......................................................................................................................................................
......................................................................................................................................................
4
SUMMARY
The purpose of this thesis was to determine the possibility and methods of testing web
services based on a WSDL document. An application, named WSDLTest, has been designed
and implemented. This application can test web services for which WSDL 1.1 or WSDL 2.0
document are available. WSDLTest parses the WSDL document, and based on, it tests the
web service by sending automatically generated messages.
The short description of the WSDL standard is given. Available on the market tools
using WSDL documents as a basis for testing are briefly presented. The functional and
nonfunctional requirements of WSDLTest application are described. The architecture and
implementation of the WSDLTest application are presented. Several examples of the usage of
WSDLTest in testing web services are given.
Keywords: web services, WSDL, SOAP, testing, automatic testing
AUTOMATYCZNE TESTOWANIE SERWISÓW WEBOWYCH NA
PODSTAWIE DOKUMENTU WSDL
Celem pracy było zbadanie możliwości i metod testowania serwisów webowych na
podstawie dokumentu WSDL. Została zaprojektowana i wykonana aplikacja WSDLTest.
Aplikacja ta może testować serwisy webowe, dla których jest dostępny dokument WSDL 1.1
albo WSDL 2.0. WSDLTest analizuje i sprawdza poprawność dokumentu WSDL i na jego
podstawie testuje serwis webowy wysyłając automatycznie wygenerowane wiadomości.
Praca zawiera krótki opis standardu WSDL. Przedstawiono także podobne
oprogramowanie dostępne na rynku. Opisano funkcjonalne i niefunkcjonalne wymagania,
architekturę i implementacje WSDLTest. Pokazano także przykłady użycia WSDLTest w
testowaniu serwisów webowych.
Słowa kluczowe:, serwisy webowe, WSDL, SOAP, testowanie
5
Table of Contents
1 Introduction ...................................................................................... 7
2 Web services .................................................................................... 8
2.1 WSDL ......................................................................................... 9
2.1.1 Introduction ......................................................................... 10
2.1.2 Basic information ................................................................ 11
2.1.3 Structure .............................................................................. 12
2.2 Usage of WSDL in testing ........................................................ 16
2.3 Testing tools ............................................................................. 17
2.3.1 HP QuickTest Professional ................................................. 17
2.3.2 Parasoft SOAtest ................................................................. 18
2.3.3 SOAPSonar ......................................................................... 20
2.3.4 SoapUI ................................................................................. 21
3 Web service testing tool “WSDLTest” .......................................... 23
3.1 Requirements of WSDLTest .................................................... 23
3.1.1 Functional requirements ...................................................... 24
3.1.2 User interfaces requirements .............................................. 29
3.1.3 Non functional requirements .............................................. 30
3.2 Design ....................................................................................... 31
3.2.1 System overview ................................................................. 31
3.2.2 Assumptions ........................................................................ 32
3.2.3 Review of existing WSDL parsers...................................... 34
3.2.4 System architecture ............................................................. 37
3.2.5 Policies and tactics .............................................................. 42
3.2.6 Detailed system design ....................................................... 42
6
3.3 Tests .......................................................................................... 47
3.3.1 Currency Converter ............................................................. 47
3.3.2 E-mail Validate ................................................................... 52
3.3.3 Validate Card Number ........................................................ 56
3.3.1 dohwebservice ..................................................................... 60
4 Examples ........................................................................................ 63
5 Conclusion ...................................................................................... 67
6 References ...................................................................................... 68
6.1 Glossary .................................................................................... 72
7
1 Introduction
Nowadays web services are very important in providing software in the World Wide
Web. Web service is an application, which is available over the Internet, and is delivered not
as a full product but as a service only. Service oriented architecture of web services makes
them an alternative to traditional software model: they are cheaper and theirs time to market is
minimized. Web services were designed to support interoperability between different virtual
or physical machines. Web services can communicate via SOAP [1] messages. This allows
them to communicate with any application, written in any language or designed for any
platform, which can understand SOAP. Like all software, such an application should be tested
before it is delivered to the user. Web service can be described by a Web Service Description
Language [2] document. WSDL was established as a standard by the W3C [3]. The WSDL
document describes web service in terms of its interfaces with operations, types of data the
web service takes or returns. WSDL does not provide description of internal operation of web
service. Nevertheless, such a document can be the basis for testing the web services.
The main goal of this thesis was to develop software, which can be used for testing web
services based on WSDL document. The program, later called WSDLTest, uses WSDL
document to get information about the web service. WSDLTest can send SOAP messages and
receive responses. The GUI and SOAPClient were implemented by Michał Kurek. Testing
algorithm was written by Małgorzata Purwin. The remaining functionalities of this application
were co-written.
The second chapter, written by Małgorzata Purwin (excluding section “Usage of WSDL
in testing”), starts with the detailed description and definition of what Web Service
Description Language actually is. Next, the possibilities of the usage of WSDL in testing of
web service are described by Michał Kurek. In next section, Małgorzata Purwin presents
some examples of testing tools, available on the market, that are testing web services based on
WSDL document.
The third chapter was written (except section 3.3) by Michał Kurek, and begins with a
listing of functional requirements of WSDLTest, followed by non-functional requirements
and the specification of user interfaces. The next section – design presents the high and low
level architecture of WSDLTest. In section 3.3 of the third chapter, prepared by Małgorzata
Purwin, several test cases are presented. They show different aspects of WSDL
functionalities.
In the last chapter, examples of how to use the WSDLTest are presented, including use
case and sequence diagram. This chapter, written by Małgorzata Purwin, shows how the
WSDLTest user can successfully use application in different scenarios.
The introduction and conclusion where co-written by Michał Kurek and Małgorzata
Purwin. In conclusion, advantages and disadvantages of WSDLTest are discussed, also some
modifications, which can be introduced in future, are indicated.
8
2 Web services
A web service is an application available over the Internet. Web services use XML
based messaging system; therefore, they are platform independent, which makes them highly
interoperable. Such a property allows various software written in different languages to
communicate (e.g. Windows applications can communicate with UNIX ones or Java written
applications can communicate with C++ ones).
The W3C [4] (World Wide Web Consortium – international organization which is
responsible for developing standards for Word Wide Web) defines web service as “a software
system designed to support interoperable machine-to-machine interaction over a network. It
has an interface described in a machine-processable format (specifically WSDL). Other
systems interact with the Web service in a manner prescribed by its description using SOAP
messages, typically conveyed using HTTP with an XML serialization in conjunction with
other Web-related standards.” [5] .
Web service fulfills three tasks:
1. Service provider – is responsible for the availability over Internet and the
implementation of the service.
2. Service requestor – is responsible for the utilization of web service and sending
XML-based requests.
3. Service registry – is logically centralized directory of services.
Web Service Protocol Stack is a set of protocols that are used to implement web service.
Web Service Protocol consists of four layers:
1. Service transport – is responsible for transport of exchanged messages and
contains Hypertext Transfer Protocol (HTTP) [6], Simple Mail Transfer
Protocol (SMTP) [7], File Transfer Protocol (FTP) [8] and Blocks Extensible
Exchange Protocol (BEEP) [9].
2. XML messaging – is responsible for encoding XML-based messages. It contains
SOAP [1] and XML-RPC [10]. Detailed description of SOAP and XML is given
in section 2.1.1.
3. Service description – describes the public interface of a web service. Web
Services Description Language (WSDL) [11] document contains such
description and is presented in section 2.1.1.
4. Service discovery – is responsible for providing service discovery and is realized
via UDDI [12] (described in section 2.1.1).
Web services are loosely coupled, which means that each of its components has little
knowledge of all the other components. When there are any changes in client interface of the
web service, there is no need to change server interface of the web service. In other words:
changes in interface of web service does not stop client from interacting with web service.
Web services are coarse-grained systems, which imply that they consist of small
number of large components. Web service can have both synchronous and asynchronous
9
connection between client and execution. The asynchronous operations, when invoked, do not
block the client while service is completing its operation. The client will receive result from
asynchronous operations later than from synchronous operations. Web services asynchronous
capabilities are necessary to have loosely coupled systems.
Web services support the Remote Procedure Calls [13]. RPC is a mechanism in which
there are two parts: the caller (client) and server. The client is managing the call and is
waiting for a response. The most important feature of RPC is the possibility that these
processes can be located on different physical or virtual machines.
Web service is composed of four components:
1. XML-RPC – protocol to perform RPC [13],
2. SOAP,
3. WSDL,
4. UDDI.
Fig. 1 shows detailed structure of web service, with its components and their roles.
Fig. 1 – Web Service schema
2.1 WSDL
Below three out of four layers of Web Service Protocol Stack are described i.e.: XML,
UDDI, SOAP and at end WSDL. These protocols are necessary to implement, use and test
web service.
10
2.1.1 Introduction
Extensible Markup Language – XML
XML [14] is self-descriptive markup language designed to carry and store data. In many
ways, XML is similar to Hypertext Markup Language (HTML) [15]. However, there are two
significant differences:
1. XML was designed to transport data with emphasis on their contents while
HTML displays data with emphasis on the way they look.
2. In HTML, all tags are predefined such as: <title>, <body>, on the contrary,
while using XML user creates his own tags (there are no predefined tags) e.g.:
<address>, <name>. HTML and XML complement each other.
XML`s single purpose is to store and transport information, it cannot do anything else.
With appropriate software, XML documents can be sent, received and displayed. Currently
XML is the most common way to transport data between different types of applications and
for World Wide Web is as important as HTML. It became W3C [16] recommendation in
1998. XML was designed to be readable both by human and by computer. Following Code
Snippet 1 presents an example of XML schema, which stores personal data about a client.
1 <?xml version="1.0" encoding="UTF-8" ?>
2 <client clientid="889923"
3
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
4
xsi:noNamespaceSchemaLocation="client.xsd">
5
<clientname>JK</clientname>
6
<address>
7
<name>Jan Kowalski</name>
8
<street>Warszawska 1222</street>
9
<city>00-000 Warsaw</city>
10
<country>Poland</country>
11
</address>
12 </client>
Code Snippet 1 – XML schema example
Universal Description, Discovery and Integration – UDDI
UDDI [12] is a XML-based protocol that defines a method for publishing software
components of Web services. UDDI registry catalogues and classifies Web services; it can be
compared to phone book. In UDDI, user can find data and metadata about Web services such
as WSDL description. UDDI is widely used in business for discovery of already existing
solutions.
11
Simple Object Access Protocol – SOAP
SOAP (Simple Object Access Protocol) [1] is communication protocol used to access
Web Services. As an XML based protocol, SOAP is platform and language independent. It
provides the communication between applications running on different operating systems and
applications written in different programming languages as long as application can formulate
SOAP message.
Any transport protocol can be in use to deliver SOAP messages, but the most popular is
HTTP. SOAP can be used to send one-way messages, but it is very useful in case of Remote
Procedure Call (RPC [13]) which is a concept of communication between programs located in
different address spaces (different computers). SOAP is a recommendation of W3C [16].
Example of SOAP message, which shows how it is divided into sections, is given in the
following Code Snippet 2.
1 <?xml version="1.0"?>
2 <soap:Envelope xmlns:soap="http://www.w3.org/2001/12/soap-envelope"
3
soap:encodingStyle="http://www.w3.org/2001/12/soap-encoding">
4 <soap:Header>
5 ...
6 </soap:Header>
7 <soap:Body>
8 ...
9
<soap:Fault>
10
...
11
</soap:Fault>
12 </soap:Body>
Code Snippet 2 – SOAP Message example
2.1.2 Basic information
WSDL [11] is an acronym created from Web Services Description Language, and
corresponds to abstract, XML-based language, which specifies location and functionalities
offered by web service. A computer can understand such description, which provides
information about web service, for example what kind of parameters it expects, what kind of
data it returns.
XML schema is used for presentation of WSDL description, which consists of messages
that web service sends and receives. To communicate with web service SOAP messages are
exchanged and are described by WSDL as operations, which can be used for testing. WSDL
describes transport format for interfaces, it can also describe interactivity of given service.
WSDL description only shows possibilities but not required interactions. Such a design of
descriptive language allows testing web service through and through.
12
The newest version of WSDL is 2.0, which is recommended by W3C [16], however
WSDL 1.1 is still quite common and in some software only this version is supported. Initially
version 2.0 had number 1.2 but because of very significant changes and improvements that
were introduced in WSDL 2.0 the number of version changed. Fig. 2 illustrates crucial
differences between both versions. Sections <messages> and <portType> were combined to
create new section <interface>. Semantic in WSDL 2.0 is more extent than in version 1.1.
Fig. 2 – Difference between WSDL 1.1 and WSDL 2.0
2.1.3 Structure
WSDL 2.0 is divided into several sections:
“description” section is a container, inside of which the remaining sections
are located,
“types” element describes the data types send and received by a web
service,
“interface” section describes the abstract functionality the web service
provides (what messages it sends and receives, possible fault messages),
“binding” provides information how to access the service,
“service” provides information where to access the service.
There are also two optional sections in WSDL document:
“documentation” provides human-readable description of a web service,
“import” is used to import other XML schemas.
13
The structure of WSDL document is presented in the following Code Snippet 3. Within
the order of defining individual components is shown.
1 <description>
2 <types>
3 In this section data types are presented
4 </types>
5
6 <interface>
7 Here are presented definitions of operations
8 </interface>
9
10 <binding>
11 This section binds web service and message protocol
12 </binding>
13
14 <service>
15 This section provides endpoints for bindings
16 </service>
17 </description>
Code Snippet 3 – Frame of WSDL document
In further sections the detailed description of each of obligatory section of WSDL is
given.
DESCRIPTION
Tag “description” contains all other tag sections and is used for declaring namespaces.
Element targetNamespace, presented inside the Code Snippet 4, should contain link to
WSDL document describing the service. The default value in xmlns (as shown in line 2 of the
Code Snippet 4) is a standard for WSDL files.
1 <description
2
xmlns=" http://www.w3.org/ns/wsdl"
3
targetNamespace= "..."
4
xmlns:tns= "..."
5
xmlns:stns= "..."
6
xmlns:wsoap= "..."
7
xmlns:soap= "..."
8
xmlns:wsdlx= "...">
Code Snippet 4 – Tags and properties inside description element
14
TYPES
The “types” section contains data types that the web service uses (example given in
Code Snippet 5). Types, to be declared, have to be supported by web service API. Each
operation can have its own input type (request) [line 6], output type (response) [line 15]
and fault (error) [line 17]. The types defined in “types” can be a simple type such as string
[line 9] or int [line 10]. Complex type such as response can consist of many elements,
which can be build-in data types (from XML schema [17]). Complex and simple data types
are presented in Code Snippet 5.
1 <types>
2
<xs:schema xmlns:xs=...
3
targetNamespace=...
4
xmlns:tns=...>
5
6
<xs:element name="request" type="request">
7
<xs:complexType name="request">
8
<xs:sequence>
9
<xs:element name="Name" type="xs:string"/>
10
<xs:element name="Age" type="xs:int"/>
11
</xs:sequence>
12
</xs:complexType>
13
</xs:element>
14
15
<xs:element name="response" type="xs:string"/>
16
17
<xs:element name="error" type="xs:string"/>
18
19 </xs:schema>
20 </types>
Code Snippet 5 – Overview of “types” section
INTERFACE
The “interface” section is a set of operations representing the interaction between the
user and the web service. Each of these operations specifies the sequence of messages
exchange that service sends or receives and the type of a given message. Operations are
similar to methods or functions in programming languages.
1 <interface name = "Iname" >
2
3
<fault name = "error" element = "stns:error"/>
4
5
<operation name="example"
6
pattern="http://www.w3.org/ns/wsdl/in-out"
15
7
style="http://www.w3.org/ns/wsdl/style/iri"
8
wsdlx:safe = "true">
9
10
<input messageLabel="in" element="stns:request" />
11
<output messageLabel="out" element="stns:response" />
12
<outfault messageLabel="out" ref ="tns:error" />
13
14
</operation>
15 </interface>
Code Snippet 6 – Overview of “interface” section
Code Snippet 6 presents example of operation named example [line 5]. This element
has inside parameters: input parameter (input) [line 10] return value (output) [line 11] and
error – which can be referenced to global fault. Error definition that can be used by many
operations is defined in <fault name> tag [line 3]. It will be returned to user when en error
occurs.
WSDL defines four types of operations:
One-way – operation can receive a message,
Request-response – operation can receive a request and will return a
response,
Solicit-response – operation can send a request and will wait for a response,
Notification – operation can send a message.
BINDING
The “binding” section is responsible for access to the web service; it describes protocol,
which may be used. This is a way to bind web service with protocol. If there is defined
binding for given interface, then bindings for all operations from that interface must be
defined. Code Snippet 7 presents, in element wsoap:protocol [line 4], binding of SOAP
protocol to previously defined (Code Snippet 6 [line 5]) operation “example” [lines 8, 9].
1 <binding name="Bname"
2
interface="tns:Iname"
3
type="http://www.w3.org/ns/wsdl/soap"
4
wsoap:protocol="http://www.w3.org/2003/05/soap/bindings/HTTP/">
5
6
<fault ref="tns:error" wsoap:code="soap:Sender"/>
7
8
<operation ref="tns:example"
9
wsoap:mep="http://www.w3.org/2003/05/soap/mep/soap-response"/>
10
11 </binding>
Code Snippet 7 – Overview of “binding” section
16
SERVICE
The “service” section contains a set of endpoints at which service is provided. The
endpoints are places at which the service is provided. Code Snippet 8 presents declaration of
endpoint named Ename [line 5] for binding Bname [line 6] (declared in Code Snippet 7 [line
1]).
1 <service
2
name="Sname"
3
interface="tns:Iname">
4
5
<endpoint name ="Ename"
6
binding ="tns:Bname"
7
address ="http://..."/>
8
9 </service>
Code Snippet 8 – Overview of “service” section
2.2 Usage of WSDL in testing
WSDL document is a description of functionalities provided by web service. Inside this
document, all “methods” and data types can be found. It is divided into several sections
directly connected with each other. The section “types” contain data types, which are input
and output parameters for operations. Operations are described in “interface” section. For
each interface, there is a binding with message protocol. The “binding” section of WSDL
document provides knowledge about protocols bonded to given interface. All interfaces using
SOAP protocol should be found. “Service” sections provide information where to find
endpoint for each binding. All those sections allow to determine input and output, as well as
how and where to access the web service. Nonetheless, there is no information about what
exactly web service do produce output from input. To use effectively WSDL inside testing
software, following elements are needed:
WSDL Parser – tool, which allows reading WSDL document and extracting
information. Several parsers are available over the internet; they are described in
section 3.2.3.
Oracle – mechanism, which allows determining if the results of the test are correct.
Such oracle must be provided by user. By comparing results with user provided
expected results it is possible to determine if web service fulfills its function.
SOAP messages generator - Inside each of the interfaces there are listed available
operations with input and output parameters. In order to communicate with web
service SOAP messages must be generated based on WSDL document.
The process of testing based on WSDL document includes several steps. First step is to
parse WSDL document into addressable memory objects. Information from parsing can be
17
used to generate SOAP requests. Such message may be filled by either random parameters
based on data type taken from WSDL document or filled by provided content. Such approach
to web service testing is applicable only when communication is dual – request and response
from web service. When SOAP request is send, after a while web service should send back
response with results of its operation. These results may be later compared with oracle to
determine if web service is working properly.
2.3 Testing tools
There are several testing tools available on the market, which are still being maintained
or developed. The most recognizable are:
1. HP QuickTest Professional,
2. Parasoft SOAtest,
3. SOAPSonar,
4. SoapUI.
All above-mentioned tools are described below. Some of them offer option of validation
of WSDL document provided by WS-I [18]. The OASIS Web Services Interoperability (WS-I
- http://www.oasis-ws-i.org) is an industry consortium, which defined set of rules showing
how standards should be used. It provides testing tools for use by the community to help in
developing and deploying interoperable Web services.
2.3.1 HP QuickTest Professional
Publisher: Hewlett-Packard [19]
Publisher web site: http://www.hp.com/
HP QuickTest Professional tool is an automation testing software suite. It uses both 1.1
and 2.0 version of WSDL. The application performs automated tests on a variety of software
and in many environments. It has graphical user interface, which allows performing
regression and functional testing.
Main testing algorithm identifies objects to be tested in the applications interface and
performs testing interface operations. Important functionality of HP QuickTest Professional is
ability to perform validation of WSDL using the WS-I [18] tool. Application allows easy
modification, insertion, and removal of test parts like presented on Screenshot 1.
18
Screenshot 1 – Test steps records editor page in HP QuickTest Professional
HP QuickTest Professional has a Web Service Testing Wizard Click that allows
defining variety of options e.g. create a checkpoint or select some actions. Everything defined
inside the wizard is later converted and inserted into the test. QTP has functions allowing the
customization of reports with user-defined images and screenshots. Client performance-
related errors (e.g. memory leakage) can be included in the standard reports with a direct link
from the report to the test script. HP QuickTest Professional automates the design of tests and
test cases. It is quite complex and powerful tool.
2.3.2 Parasoft SOAtest
Publisher: Parasoft [20]
Publisher web site: http://www.parasoft.com/
SOAtest is used for testing of applications build in Service-oriented Architecture. It can
automate application testing, message/protocol testing, cloud testing and security testing. Its
graphical user interface is Eclipse based (uses the same GUI library). It allows creating tests,
defining the behavior of the tests and configuring specific tests (e.g. queue managers or
database connections).
19
SOAtest primary function is to test Web services. It automatically generates tests from
key market platforms such as WSDL, WADL, UDDI, WSIL, XML, Schema, BPEL, HTTP
traffic and other. SOAtest can validate WSDL documents and emulate the client or the server.
Tests can be grouped together, to be executed in a sequence, which allows for construction of
more complex testing scenarios. Failed tests are distinctively highlighted in the generated
results. Parameters for the requests can be entered by the user or can be read from a file.
Example of test screen of SOAtest is presented on Screenshot 2.
Screenshot 2 – SOAtest testing environment
SOAtest can trace and visualize how messages and events flow through ESBs
(Enterprise service bus) [21], message brokers, applications and databases, while tests are
executed. Such tracing allows interpreting problems directly from test environment. It is
possible to validate or start regression control at any point of testing.
SOAtest provides regression tests and functional tests. They will cause an alert when
modifications influence application behavior. Functional tests use load testing for
performance monitoring. Security penetration tests are performed at the message layer.
SOAtest is proficient software to perform web services testing.
20
2.3.3 SOAPSonar
Publisher: Crosscheck [22]
Publisher web site: http://www.crosschecknet.com/
SOAPSonar is a testing and analysis tool specifically designed for Web Services. It can
perform functional regression tests, performance tests; generate compliance reports,
vulnerability checks and identity tests. It has clear graphical user interface. Tests are created
via drag and drop selection. Application has WSDL Region WS-I validation [18]. It accepts
both WSDL 1.1 and 2.0 documents. The professional edition of SOAPSonar has test flow
management options and can create WSDL requests - response chain or data driven test for
the exchange of messages. The navigation of WSDL operations and SOAP request/response
panes are presented on Screenshot 3.
Screenshot 3 – Navigation within WSDL document in SOAPSonar
All versions of SOAPSonar use as test inputs and response analysis data from e.g.
database tables or Microsoft®Excel files. Application can automatically change variables in
message headers, message body, tasks and can change global variables and automation
variables. SOAPSonar has vulnerability mode, which allows associating each test request
with a set of attack. Examining responses helps to find any existing vulnerabilities and
leakage of information. It can parse the WSDL documents and generate a list of the
operations available on interface described by it. It can be used to send SOAP request
messages to the target Web service and capture the responses.
21
2.3.4 SoapUI
Publisher: Smart-Bear [23]
Publisher web site: http://www.soapui.org/
SoapUI is an open source web service testing application for service-oriented
architectures. It provides complete test coverage and supports all the standard protocols and
technologies. SoapUI has clear and easy to use graphical user interface, which allows for
quick creation of any test scenarios. It requires WSDL files to generate tests, messages,
validations and MockServices. SoapUI supports only 1.1 version of WSDL. SoapUI can
easily create functional, regression, compliance, and load tests. SoapUI provides code free test
environment, all tests are created by drag and drop actions. View of functional test created
from a request in WSDL is presented below (Screenshot 4).
Screenshot 4 – Functional test created from WSDL
22
SoapUI Pro Test Debugging is an option, which allows following the flow of test,
variables, properties requests and context. Data for data-driven tests can be provided from
external editors such as Microsoft ® Excel.
SoapUI MockServices is a feature, which allows mimicking Web Services before they
are implemented. It eliminates the expense of building web service. MockServices can
simulate any desired behavior of web service. No matter how complex, the service responses
can be completely configured. SoapUI uses WS-I profile [18] for 1.1 version of WSDL. It can
validate WSDL from pop-up menu or WS-I Compliance tab in the WSDL Service window.
SoapUI allows the code generation from a WSDL, in order to have easy comparison
between different frameworks (e.g. .NET framework [24]). SoapUI is open source software
with a feature, which eliminates code from test designing process and simulates behavior of
web services. Unfortunately, it provides support only for WSDL 1.1.
23
3 Web service testing tool “WSDLTest”
In the section below, the detailed requirements of WSDLTest project are presented, as
well as better overview of the project. Moreover, section includes written project
specification, design and testing final application, use cases and final summary focusing on
realization of the project.
3.1 Requirements of WSDLTest
The result of this project should present full and functional version of WSDLTest tool.
In specific application should fulfill following conditions to be:
1. WSDLTest allows the testing of web services with the usage of WSDL
document attached to web services.
2. WSDL document is downloaded to WSDLTest where is it parsed.
3. User Interface presents the general information regarding web service and its
operations described in WSDL document.
4. User can choose, which operations should be tested and in what specific way.
5. User can modify testing parameters and provide custom data set for testing.
6. WSDLTest should perform tests and write the tests results into file or present
results in the user interface.
The WSDLTest shall support testing of web services based on web service description
language (WSDL). User interface consists of two vertical and separate box panes.
The Right pane of user interface (marked with number 2) should present two tab
components: info tab containing information about web service and WSDL viewer tab
showing WSDL document in interactive tree diagram view. On the left half there should be
input field filled with URL address of WSDL document describing specific web service.
WSDLTest should download and parse such document, when user presses apply button. The
bottom pane (marked as 3) includes textual output of log file, which consists of log entries
informing about various application actions like currently performed action, current process
undergoing, warning or severe error.
The content of center part of the left (marked as 2) pane should depend on test process
stage; it allows user to choose operations to be tested, provide custom parameters for each
operation and present final test results both preview and summary.
24
WSDLTest
InfoWSDL Viewer
WSDL Viewer
Service addressApply
LOGGER output
Accordion:- Choosing operation - Providing parametrs- Results
1
2
3
Fig. 3 – Graphical User Interface – main parts
3.1.1 Functional requirements
Functional requirements in this section are graded by their criticality in scale 1 to 5
where:
1. present very low criticality and item that can be removed without any visible
lose of functionality.
2. indicates low criticality when functionality is an extra item and may be possibly
removed.
3. present medium criticality when item is strongly needed in application.
4. stands for high criticality for item, which is required in application for lower
criticalities to function.
5. present very high criticality for items that are mission critical and application
would not function without them.
Each requirement described below contains textual description, criticality level,
technical issues with pre and post conditions, list of risks if functionality is not provided and
numbers of other requirements that functionality depends on.
25
1. Application should be able to read the WSDL document.
a. Description
With proper coding done, application can read WSDL document from remote
internet location, which address is provided by user.
b. Criticality 5
c. Technical Issues
Precondition: User provided valid remote location of WSDL.
Precondition: Access to internet connection for non-local documents.
Postcondition: WSDL document properly accessed and read by application.
d. Risks
Application fail to read WSDL document.
e. Dependencies with other requirements 4
2. Application should be able to parse WSDL document.
a. Description
With properly accessed and read document application can parse WSDL
document for further processing.
b. Criticality 5
c. Technical Issues
Precondition: WSDL document read by application.
Postcondition: Information parsed from WSDL document accessible within
application.
d. Risks
Cannot access information parsed.
Parsing process has failed.
e. Dependencies with other requirements 1
3. The application shall be able to display graphical user interface (GUI).
a. Description
After proper coding application should display GUI of WSDLTest.
b. Criticality 5
c. Technical Issues Postcondition: Application should properly display GUI.
d. Risks
GUI is displayed improperly or not at all.
e. Dependencies with other requirements
-
4. Application should allow the user to input WSDL address in GUI.
a. Description
With properly working GUI window there should be text input field, which
accepts URL address of WSDL document describing web service to be tested.
b. Criticality 5
c. Technical Issues
Precondition: GUI window is properly displayed.
Postcondition: User inserted WSDL address without errors.
d. Risks
Input field is not enabled or does not save inserted data.
e. Dependencies with other requirements 3
5. Application should be able to establish SOAP connection.
a. Description
Application can establish the SOAP connection over the HTTP protocol.
b. Criticality 5
26
c. Technical Issues
Precondition: Web service accepts SOAP over HTTP connections.
Precondition: Access to the internet connection for non-local web services.
Postcondition: Application is ready to deliver SOAP message over established
connection.
d. Risks
Connection could not be established.
e. Dependencies with other requirements 2
6. Application should be able to send SOAP message.
a. Description
Application can send well-defined SOAP message template with provided
field values over previously established SOAP connection.
b. Criticality 5
c. Technical Issues
Precondition: Established SOAP connection.
Precondition: Well defined SOAP message template.
Postcondition: SOAP message is send and application is ready to accept the
response message.
d. Risks
Message does not reach target web service.
Message is rejected by web service.
e. Dependencies with other requirements 2, 5
7. Application should be able to generate SOAP message template.
a. Description
Based on information parsed from WSDL document application is able to
generate SOAP message with empty value fields.
b. Criticality 5
c. Technical Issues
Precondition: WSDL document parsed.
Postcondition: Properly generated SOAP message template.
d. Risks
Message template is not generated.
e. Dependencies with other requirements 2
8. Application should be able to send set of SOAP messages.
a. Description
Based on parameter set provided by user and SOAP message template
application can send volume of SOAP messages filled with different parameter
values.
b. Criticality
5
c. Technical Issues
Precondition: Ability to send SOAP messages.
Postcondition: Successfully send all SOAP messages.
d. Risks
One or more messages do not reach target web service.
One or more messages are rejected by web service.
e. Dependencies with other requirements 6, 8, 10
9. Application allow user to input parameter set in GUI.
a. Description
User can insert in GUI into text input fields set of parameters, which will fill
27
SOAP message template to generate SOAP message to be sent during the test
phase.
b. Criticality 5
c. Technical Issues
Precondition: GUI window has been properly displayed.
Postcondition: Parameters are correctly read from user input.
d. Risks
Parameters are not read properly or input field is not enabled.
e. Dependencies with other requirements
3
10. Application should be able to display logger messages in GUI.
a. Description
After being properly coded application can display in GUI logger messages
generated by various events in application.
b. Criticality 4
c. Technical Issues
Precondition: GUI window has been properly displayed.
Postcondition: Logger messages are properly displayed.
d. Risks
Text area in GUI does not display logger messages correctly.
e. Dependencies with other requirements 3
11. Application should allow user to set logger display level in GUI.
a. Description
After being properly coded user can choose the minimal level of logger
messages to be displayed.
b. Criticality 2
c. Technical Issues
Precondition: GUI window has been properly displayed.
Precondition: Logger text area in GUI is displayed.
Postcondition: Logger messages from minimal level and higher are properly
displayed.
d. Risks
Text area in GUI does not display logger messages correctly.
e. Dependencies with other requirements 3, 11
12. Application should be able to save logger messages to local file system.
a. Description
After being properly coded application can save all messages from logger to a
file on local system. Moreover, old log files should not be overwritten but
renamed and saved.
b. Criticality 3
c. Technical Issues
Precondition: Properly coded logger messages.
Postcondition: Logger messages are captured and saved in local system file.
d. Risks
Log file is not created or logger messages are not captured properly.
e. Dependencies with other requirements -
13. Application should be able to save test results to local file system.
a. Description
After being properly coded application can save test results to a file on local
28
system. Moreover, old tests result files should not be overwritten but renamed
and saved.
b. Criticality 3
c. Technical Issues
Precondition: Test process completed successfully.
Postcondition: Test results are captured and saved in local system file.
d. Risks
Test file with results is not created or test results are not captured properly.
e. Dependencies with other requirements -
14. Application should be able receive SOAP message.
a. Description
After sending correct SOAP message request to tested web service application
should be able to receive SOAP message response.
b. Criticality 5
c. Technical Issues
Precondition: Properly send SOAP message request.
Postcondition: SOAP message has been received properly.
d. Risks
SOAP message has not been receive or SOAP connection is disrupted.
e. Dependencies with other requirements 9
15. Application should allow user to input regular expression as parameter in GUI.
a. Description
User can insert in GUI into text input fields regular expression, on which base
application will generate parameters, which will match this regular expression
and fill SOAP message template to generate SOAP message to be sent during
the test phase.
b. Criticality 3
c. Technical Issues
Precondition: GUI window has been properly displayed.
Postcondition: Regular expression is correctly read from user input.
Postcondition: Parameters are correctly generated.
d. Risks
Regular expression is not read properly.
Input field is not enabled.
Generated parameters do not match given regular expression.
e. Dependencies with other requirements 3
16. Application should be able to generate automatically random parameters.
a. Description
Application can generate random parameters for tested web service operations
that match their data type.
b. Criticality 3
c. Technical Issues
Precondition: WSDL document has been parsed properly.
Postcondition: Generated parameters have proper data type.
d. Risks
Generated parameters do not match needed data type.
e. Dependencies with other requirements 2
29
17. Application should be able to test web service based on mode chosen by user in
GUI.
a. Description
Application can perform full set of test in one of available modes: automatic,
semi-automatic and manual, which is chosen by user in GUI.
b. Criticality 5
c. Technical Issues
Precondition: WSDL document has been parsed properly.
Precondition: Parameters were generated properly.
Postcondition: Results are printed in GUI.
Postcondition: Results are saved to file.
d. Risks
Current testing process is terminated.
e. Dependencies with other requirements 1,2,3,4,5,6,7,8,9,14,15,16
18. Application should be able to create oracle based on user input in GUI.
a. Description
Application can create oracle from data provided by user in GUI.
b. Criticality
3
c. Technical Issues
Precondition: Application started testing web service.
Postcondition: Results are printed in GUI.
Postcondition: Results are saved to file.
d. Risks
Oracle is not created.
Dependencies with other requirements 17
19. Application should allow user to choose predefined testing mode in GUI.
a. Description
Application should allow user to choose one of predefined testing modes using
GUI component.
b. Criticality
3
c. Technical Issues
Precondition: Web service testing has been started.
Postcondition: Chosen mode has been initialized properly.
d. Risks
Chosen mode has not been initialized.
e. Dependencies with other requirements 17
3.1.2 User interfaces requirements
Graphical user interface
WSDLTest should be GUI type application with event-based architecture with strong
aspect of user interaction with interface. User should be able to perform full process of web
service testing based on WSDL document, therefore all steps of such process should be
visibly divided and presented in GUI. Logger messages should be clearly visible to indicate
that application is working and what operation is currently performed.
30
3.1.3 Non functional requirements
Non-functional requirements of WSDLTest are as follows:
Communication interfaces requirements
User of WSDLTest needs active internet connection for non local area network web
services.
Software interfaces requirements
User need up-to-date Java Runtime Environment (JRE) [25] version 6 or higher
installed. Optionally application can be run thru web browser with Java web plug-in or Java
Web Start.
Performance requirements
Application should response to interface interactions instantly. All operations that are
not instant should be indicated visually within user interface.
Security
Security of WSDLTest depends on interaction with external WSDL document, which is
unknown before reading and therefore is potential security risk. Security is to be ensured by
EasyWSDL Toolbox library, which during parsing looks for non-W3C compliant elements in
target document. Application should not store any secret or vulnerable data therefore extra
security would not be required.
Binary Compatibility
Since Java applications run on Java Virtual Machine and not on native operating system
itself then WSDLTest shall be compatible with all operating systems that can run Java
Runtime Environment.
Maintainability
Application should be both adaptively and correctively maintainable, which means that
the problems can be easily corrected. Modifications should be easily applied to keep
application usable in a long term. With documentation provided modification and adding new
source code by programmer should not be a challenge.
Portability
WSDLTest uses Java Runtime Environment and is both system and processor portable,
which means that the same code can be run without changes on different platforms supporting
JRE.
Extensibility
WSDLTest implementation should allow further modification without major changes in
existing functionality.
Reusability
Applications functions should be implemented in a modular way so it will be easy to
reuse.
31
Compatibility
WSDLTest generally should not interfere with other software working on the system.
Application may be incompatible with JRE incompatible software.
Resource Utilization
Application should not utilize more than minimal requirements for Java Runtime
Environment requirements [26] for graphical based application.
Serviceability
WSDLTest should implement serviceability by logging all possible application events
(for ex.: error and warning messages) and allow software upgrade.
3.2 Design
In this section, the implementation of WSDLTest is presented including the architecture
of application and sub-architecture of associated interfaces.
3.2.1 System overview
Core
Parser (EasyWSDL)
HTTP
WSDL document
GUI (Apache Pivot)
Web Serivce
SOAP over HTTP
Fig. 4 – High-level diagram presenting WSDLTest
32
In the Fig. 4 the high-level structure of WSDLTest is presented. External connections
and internal interfaces are also shown. The frame at the top of figure, which consists of
WSDL document and web service itself, represents remote location that is accessed in
different manner by WSDLTest application. WSDL document is accessed via parser using
HTTP connection, whereas web service is communicated thru SOAP messages transported
over the HTTP connection. Core module consists of classes that support main functionality of
application. WSDLTest can be divided therefore logically into Core, Parser and GUI.
3.2.2 Assumptions
In the design of WSDLTest, following assumptions and considerations were made:
Java Runtime Environment is required to operate.
The system on which WSDLTest run has sufficient operating memory, file
system storage space and enough CPU time.
Working TCP/IP connection is required.
WSDLTest must present means the smooth and clear process of testing WSDL
documents appropriate for a person with basic technical background without deep knowledge
about web services. Application should be interactive and should support various methods of
data input and output.
Application should response in timely manner for all internal operations and user
interactions. User should not wait long time for application response. External communication
with web service and WSDL document remote location should be minimized so that
operations that are variable in time and could take longer time are cut to minimum.
The application should does not store or require any additional local files to execute
properly if not explicitly wanted by user – needed for some advanced operations.
Development methods
For WSDLTest project, the incremental development has been chosen since it allows
high flexibility and reduces the risk of project failure. It is broken into small segments and
provides ability for quick reaction to development issues and quick prototyping. The
incremental development requires initial concept and requirements analysis.
Programming language
Java [27] language was chosen because application functionality specified in section
3.1 does not tie it to any system platform. Therefore, it was best to use programming language
that is platform independent and binary compatible with various systems.
WSDLTest application consists of four modules namely: Core module, Graphical User
Interface, EasyWSDL Toolbox parser and SOAP client.
Graphical user interface
The choice of programming language determines and narrows the scope of applicable
GUI libraries. There are several Java compatible toolkits. The two types of toolkits can be
distinguished: native Java toolkits and cross-platform toolkits, which require their own native
33
libraries attached to the binary file of the application. It has been decided to use one of native
toolkits of Java from which following were considered later:
Abstract Window Toolkit [28] is a basic java toolkit, which uses local system
components and it considered stable and well developed but it has not been updated in
years.
Swing [29] is a basic java toolkit based on Abstract Window Toolkit and is a “rich
internet application” [30] and desktop application toolkit. Similarly, to AWT it has not
been updated recently and is not considered suitable for contemporary GUI building.
SwingX [31] is a java toolkit based on Swing toolkit and is remake of this toolkit,
which tries to reintroduce Swing as a modern GUI toolkit and possible choice for new
projects. However, it does carry inside many components old Swing code and modus
operandi, which requires substantial amount of configuration and coding to build
modern interface.
JGoodies [32] is a java-based toolkit for GUI building. The documentation is limited
so it is difficult to use it in application.
JavaFX [33] is a java toolkit for creating RIA and desktop applications. It does not
contain complex UI components like tree tables.
Apache Pivot [34] is a java-based toolkit for creating RIA and desktop applications. It
has various complex user interface components and is very robust at the same time. It
is cross-platform toolkit and looks very modern. It fulfills all GUI requirements for
this project. Toolkit is very well documented and provides many demos and tutorials,
which allow fast prototyping.
From above list of GUI toolkits, AWT, Swing and SwingX have been eliminated since
their outdated and complex usage. JGoodies look nicely but the documentation is limited.
JavaFX even though it has many components, looks nice and is easy to use, does not contain
all components required for this project. That is why it has been decided to choose Apache
Pivot toolkit to for GUI, which has many GUI components and is easy to use.
Other Libraries
Three additional libraries are used within WSDLTest for various additional tasks not
connected with main functionality.
1. Apache Commons [35] is a library used for the manipulation of Java’s core
classes such as String.
2. Xeger [36] is a library, which allows for generation of strings based on given
regular expression. Xeger support almost all valid Java regular expressions.
Summary of all of Xeger limitations [37] is available over the Internet.
3. dk.brics.automaton [38] is a package which contains a Deterministic Finite
Automaton (shortly DFA) [39] and NFA [40] finite-state automata
implementation. It is used by Xeger library.
34
Control approach
WSDLTest is an application with interactive user interface. Therefore, it suited best to
provide event-based control within application. User actions determine the main flow of
program. However, large parts of application are controlled by automated functions therefore
without user interaction allowing fast processing of many repeating operations.
Error Handling
WSDLTest use based on java logger API [41] common logger instance for all
modules of application. Errors are logged to the log file and can be further investigated. Some
handled errors and warnings are prompted in the alert pop-up display. Most of errors are self-
handled inside code using try and catch block [42].
Communication mechanisms
WSDL document that WSDLTest needs to be retrieve from remote location (in rare
cases local location) is accessed via HTTP protocol connection. This connection is controlled
by EasyWSDL Toolbox library, which uses standard Java methods for HTTP connection and
is described in more details in section 4.2.3.1. WSDL viewer, a part of GUI, uses as well Java
methods from standard library for HTTP connection. Viewer obtains raw WSDL document
and present it in interactive tree diagram.
SOAP protocol has been chosen as messaging protocol instead of HTTP protocol. It
allows communication between WSDLTest and web service. This communication with web
service is based on single WSDL document. It is not required to handle single request node
with s multiple SOAP requests and responses based on more than one WSDL. Therefore,
HTTP protocol loses its primary advantage over SOAP. SOAP ensures common logic within
message whereas HTTP nodes do not.
3.2.3 Review of existing WSDL parsers
Below the information about WSDL parsers is presented.
Introduction
WSDL parser is the library that reads WSDL documents and creates application objects
based on extracted information. Moreover, WSDL parser can additionally allow performing
opposite operation – export information from application object to editable WSDL document.
In this section, freeware WSDL parsers available on the market are described and compared.
All WSDL parsers considered in this section require Oracle Java Development
Environment 1.6 (or later) and Oracle JDK 1.6 (or later). Among limited number of WSDL
parsing libraries, three have been chosen: EasyWSDL Toolbox, Woden and WSDL4J.
EasyWSDL Toolbox
Publisher: Petals Link
35
Publisher web site: http://easywsdl.ow2.org/
EasyWSDL Toolbox [43] is a WSDL parsing library. It can be used to parse both
WSDL 1.1 and WSDL 2.0 descriptions and transforms them in a unified object model (based
on the WSDL 2.0 entities). Library is written in Java language and therefore can be used
within java compatible application only.
EasyWSDL can read, write, edit and create WSDL 1.1, WSDL 2.0 and XML
Schema (following W3C recommendations/submissions) objects. There is a uniform API for
both WSDL versions (1.1 and 2.0). Library is WSDL 2.0 oriented, which includes mechanism
for solving the compatibility issues between both versions of WSDL. The naming convection
of the WSDL 2.0 types and elements is used when elements implementing the same concept
have two different names (ex: PortType in WSDL 1.1 and Interface in WSDL 2.0) in WSDLs
versions (Interface in this case is chosen).
Library provides the capability with the most used bindings, which are described in
details in section 2.1.3, of the two WSDL versions. It does not require casting explicitly
objects within Java and the easy WSDL API provides all necessary methods to retrieve
bindings (ex: getSOAP11Binding4WSDL11). It is based on a plug-in architecture, which
allows the developer to add new functionalities.
Woden
Publisher: The Apache Software Foundation
Publisher web site: http://ws.apache.org/woden/
The Woden [44] project is a subproject of the Apache Web Services Project to develop
a Java class library for reading, manipulating, creating and writing WSDL documents,
initially to support WSDL 2.0 but with the longer-term aim of supporting past, present and
future versions of WSDL. Woden allows users to read, write, edit and create WSDL
2.0 (following W3C recommendations) objects. There is a uniform API for WSDL 2.0
version. WSDL 1.1 documents can be converted to WSDL 2.0 using the W3C standardized
XSL Stylesheet [45]. A factory mechanism used to obtain a WSDL reader (or parser) object.
Optionally WSDL reader may be configured - for example, switching the validation
feature, which checks formal validity of WSDL document. Parser reads (deserialize) a WSDL
document located at a specified URL into the Woden WSDL 2.0 object model.
Two forms of the WSDL 2.0 object model are present: one representing the WSDL 2.0
abstract component model and the one that maps to the XML elements and attributes in the
WSDL namespace. Woden can parse almost all elements and attributes in the WSDL 2.0
namespace (the 'extends' attribute of Interface is the only one not supported element). Partial
WSDL validation is supported using the assertions defined in the WSDL 2.0 spec. Validation
exists for Types, Interface and Binding only. Transforming the WSDL document is performed
via methods of the WSDL 2.0 object model.
Library supports extensibility of elements and attributes (i.e. the XML elements and
attributes that extend standard elements in the WSDL 2.0 namespace). Woden can use this
36
extensibility mechanism for the SOAP binding extensions defined in the WSDL 2.0 spec.
Woden provides a customizable error handling mechanism for reporting warnings, errors or
fatal errors that result from WSDL validation or Woden configuration problems.
WSDL4J
Publisher: The Apache Software Foundation
Publisher web site: http://wsdl4j.sourceforge.net/
The Web Services Description Language for Java Toolkit (WSDL4J) [46] allows the
creation, representation, and manipulation of WSDL documents. It allows reading, writing,
editing and creating WSDL 1.1 (following W3C recommendations/submissions) object. There
is an API for WSDL 1.1 version. WSDL4J uses the reader to parse (deserialize) a WSDL
document at a specified URL into the WSDL4J WSDL 1.1 object model.
Two forms of the WSDL 1.1 object model are available: the one representing the
WSDL 2.0 abstract Component model and the one that maps WSDL structure and element to
the XML elements and attributes in the WSDL namespace. Parsing supports all the elements
and attributes in the WSDL 1.1 namespace. Partial WSDL validation is possible using the
assertions defined in the WSDL 1.1 specification. Manipulating the WSDL document is
realized via methods of the WSDL 1.1 object model.
EasyWSDL Toolbox Woden WSDL4J
WSDL 1.1 support Full Partial (conversion to
2.0)
Full
WSDL 2.0 support Full Full None
Parsing to objects
(object model)
Yes Yes Yes
Object Casting
Needed
No Yes Yes
Uniform WSDL 1.1
and 2.0 API
Yes No No
Conversion from
WSDL 1.1 to 2.0
No Yes No
Plug-in support Yes No No
WSDL validation No Yes Partial
Error handling Yes Yes No
Table 1 – Comparison of WSDL parsers functionality
Summary
In table 1, all three parsers have been compared regarding most important functions. For
purpose of WSDLTest EasyWSDL Toolbox has been chosen. It provides full support for
WSDL 2.0 standard on which library is based. Moreover, it has easy to use and functional
37
API and provides error handling of its functions, meaning that it fulfills all requirements from
program specification. Woden parser provides similar functionality, however its API usage is
more complex. It also fulfills WSDLTest specification. WSDL4J of all three parsers is least
mature project providing less number of functions. Moreover, it does not fulfill WSDLTest
specification.
3.2.4 System architecture
Core
On the Fig. 5, Core module of WSDLTest application is presented with data input and
outputs. It can be noticed that core module holds central role in connecting all the WSDLTest
modules together.
Core
Log File
Error Messages
Formatted data
Data File
GUI Input
EasyWSDL Toolbox
Input
Fig. 5 – Core module
In the Fig. 6, main functions of core module are presented. Based on parsed information
from WSDL document interface bonded to SOAP [1] connection is found. From this interface
all operations are extracted in order to determine what kind of parameters operations require
and how many of them. Depending on chosen test mode, which are described further in 3.2.6:
Given all information about operations and its parameters, GUI creates panel
for input values parameters or input of regular expression.
WSDLTest generates all parameters and performs number of trials given by
the user.
38
Determine web service SOAP
interfaces
Determine web service operations
Interpret operation parameters
GUI input
GUI output
Output results to file
Output FileLoging
Log file
Fig. 6 – WSDLTest level 2
3.2.4.1 Output file
The purpose of output file is to present full set of results from semi-automatic and
automatic modes. The file is saved as .csv[47] file. Such choice of output format allows for
further reuse of data by external tools. The file is human-readable.
Data
Inside of output file, for each web service operation, there are columns with values of
generated input parameters. Their number depends on number of input parameters. The output
file has results of tests. For each returned value there are two obligatory columns and one
optional. The first column, titled using parameter name, contains values taken from returned
SOAP message [1]. The second column is titled the same as first with addition of
“TypeCompatibility:” at the beginning. This column contains Boolean (Yes/No) information
about type compatibility, which indicates compatibility of returned value’s type with type
declared in WSDL document. If the oracle (detailed description is provided in section 3.2.6)
was declared, inside the file the third column appears. This column is titled in similar manner
as previous one, but with addition of “OracleCompatibility:” at the beginning. Third column
contains Boolean (true/false) information about compatibility of returned value with
previously declared oracle.
Structure of data
The data is sorted in groups, according to operation tested. The schema of output file is
presented in Table 2. Each row represents single test for one operation.
39
Operation 1
Input
Parameter 1
Name
Input
Parameter 2
Name
Return Value 1 TypeCompatibility:
Return Value 1
OracleCompatibility:
Return Value 1
Value 1 Value 1 Value 1 Yes/No true/false
Value 2 Value 2 Value 2 Yes/No true/false
Operation 2
Input
Parameter 1
Name
Return Value 1 TypeCompatibility:
Return Value 1
OracleCompatibility:
Return Value 1
Value 1 Value 1 Yes/No true/false
Value 2 Value 2 Yes/No true/false
Table 2 – Structure of data inside results file
Screenshot 5 – Graphical User Interface after application initialization
40
Graphical User Interface
The GUI introduces means of how user interacts with WSDLTest. As presented on
Screenshot 5 the GUI provides the WSDL input address, control pane allowing choosing
testing mode and parameters with results returned at the end of this process. Logger events
and tree diagram of parsed WSDL document are additional elements of GUI. User can browse
interactive tree diagram of WSDL document at any moment after applying WSDL address.
Alternatively, instead of the tab with tree diagram user can switch to Info tab, which provides
general information about web service and will appear in place of tree diagram.
Logger will output information about WSDLTest operations, communication
processes and error messages. Log is written to output file stored in folder called Log by the
name Log_n.txt, where n is order number if more than one log file present, in case of
unexpected application termination. Optionally log can displayed messages narrowed to
certain levels of importance.
User uses control pane component to choose in few steps possible control ways and
provide all necessary data. All program flow steps presented to user are visible on Fig. 7. In
first step user can choose one of three possible testing modes: automatic mode, semi-
automatic mode and manual mode.
The first of them allow full automation of parameter providing and generations on all
possible operations. User can choose how many tests should be performed. Manual mode on
the other hand allows choice of one or more possible operations and providing all operation
parameters by user. Manual mode provides single cycle testing. Semi-automatic mode
combines automatic and manual mode. It allows choosing operations for testing, providing
regular expression masks for generated parameters and number of testing cycles.
StartTesting mode
decision
Manual
Semi-automatic
Automatic
Parameters input
Parameters input
Operations choice
Operations choice
Results presentation
Do not use
Oracle usage decision
Oracle input
End
Use
Fig. 7 – Control flow of GUI panel
41
All modes are finalized by results page showing different summaries depending on the
mode. Semi-automatic and automatic modes produce also output file with formatted test
results.
User can finish whole control cycle by simply quitting whole WSDLTest application by
pressing proper button in title bar of application.
Oracle
OPERATION: nameReturn value: name1, type: type1
Save
Return value: name2, type: type2
Fig. 8 – Oracle dialog
Oracle creation step of control flow is visible in Fig. 7. This step is an option and during
execution of WSDLTest, by default, it is marked as inactive. If user decides to create an
oracle for his test, a dialog screen, presented in Fig. 8, will appear. Inside this popup dialog,
there is some number of TextInput (field that allows input of any string). Each field is
connected to one return value. Oracle as parameters takes only regular expressions,
independently of type returned by web service operation. Against provided regular
expression, results are later mapped to validate them. Definition and more details about oracle
are in available in section 2.2.
SOAPClient
The SOAPClient class provides a basic and simple interface for the rest of WSDLTest
modules to communicate with web service. WSDLTest generates SOAP [1] request via
createSOAPRequest() method. Moreover, WSDLTest can create SOAP connection and call
web service with createSOAPConnection() and callSOAPConnection() methods. SOAPClient
waits and saves web services response after calling it SOAP request. After interface is not
needed by WSDLTest closeSOAPConnection() method might be called to finish whole
process. SOAPClient internal structure is presented on Fig. 9.
42
WSDLTest SOAPClient
CreateSOAP
Connection
CreateSOAP
Request
CallSOAP
Connection
CloseSOAP
Connection
Web services SOAP Server interface
Fig. 9 – Interface structure of SOAPClient class
EasyWSDL Toolbox
EasyWSDL is a library, which implements parser of WSDL and holds several
interfaces and methods to traverse WSDL document and obtain needed information. It is used
to parse WSDL document from the address applied by user in GUI and to access various
objects created while parsing WSDL document.
3.2.5 Policies and tactics
WSDLTest application was designed in such way that each class and module has only
methods that are directly connected to its primary role in the application. Therefore, if it was
possible and reasonable partition of developed code was introduced to define clearly
individual components. For example, SOAPClient relies only on data provided as parameters
and is nearly data independent from other components.
Uniform naming convention was used within source code. Names of methods and
variables begin with small letter and every following word of the name is written beginning
with capital letter without any space between following words. Such convention provide easy
to decipher and very descriptive names.
3.2.6 Detailed system design
In this section, the detailed information about each part of WSDLTest subsystem
(class, package, function etc.) is presented. For each part following information are given:
name, classification, purpose and description of components collaborations.
43
Core
SOAPClient
EasyWSDL toolbox
Graphical User
Interface
No. 2
No. 1
No. 4
No. 3
WSDLTest
Fig. 10 – Structure of WSDLTest
1. Name
WSDLTest core
Classification
The subsystem of the application.
Definition
Set of java classes creating logical core of application: Messager, Logging,
GUIConsoleHandler and Hello classes.
Responsibilities
Implements core functions of WSDL application: automation of testing, parsed
WSDL information manipulation and initial application setup.
User/Interactions
Module has no user interaction but interacts with all other application modules.
Processing
WSDLTest is the event based application and the majority of actions are called
from graphical user interfaces or collaborating components. Core module is control
relay of execution.
2. Name
SOAPClient
Definition
Single java class SOAPClient.
44
Responsibilities
Class implementing SOAP connection and SOAP messages manipulation.
Acts as SOAP interface.
User/Interactions
Module has no user interface and interacts with all other modules.
Processing
Called from Messager class only.
3. Name
Easy WSDL Toolbox
Classification
Subsystem of WSDLTest.
Definition
Java library imported as an external Java Archive.
Responsibilities
Library parses the remote WSDL document into java objects from internal
object model of library.
User/Interactions
Module has no interface with user. It interacts with all other modules.
Processing
Called from Messager class only.
4. Name
Graphical User Interface
Classification
Subsystem of the WSDLTest.
Definition
Java class UserInterface and Apache Pivot library imported as an external Java
Archive.
Responsibilities
GUI is responsible for generating and displaying graphical user interface in
system desktop window. It manages user and local system input and output.
User/Interactions
UserInterface interacts with user and with all other modules.
Processing
It is called from initial class Hello and controls the flow of program execution.
45
UserInterface
-operationCheckboxes: List<Checkbox>
+initialize(namespace: Map<String, Object>, location: URL,
resources: Resources)
SOAPClient
-soapResponse: SOAPMessage
-soapConnection: SOAPConnection
Uses
Messager
-desc: Description
+resolveSimpleType(name: QName): String
+resolveComplexType(name: QName): Map<String,Map<String,Integer>>
+getOut(Op: Operation): Map<String,Map<String,Integer>>
-writer: PrintWriter
+SendMessage(op: Operation, Map<String, Set<String>> args)
+Validate(Type: String, Value: String): boolean
-oracleOn: boolean
+createSOAPRequest(op: Operation,args:
Map<String,Set<String>>):SOAPMessage
+closeSOAPConnection()
+createSOAPConnection()
+callSOAPConnection(request: SOAPMessage, to: String): SOAPBody
-wsdlUrl: String
-Mode: String
-checkWithOracle(Name: String, Op: Operation, Value: String):
Boolean
+OpenOracle()
Uses
Fig. 11 – Simplified class diagram
In Fig. 11 a simplified class diagram is presented. It shows three main classes of
WSDLTest (UserInterface, Messager, SOAPClient). In each class, only significant
attributes and methods are shown.
The first class UserInterface it contains all graphical elements, such as buttons and
text inputs, they are initialized in method initialize(…). The main attributes are:
46
wsdlUrl – stores the URL of WSDL document,
Mode – saves the user choice of testing mode for WSDLTest,
oracleOn – indicates if oracle was declared,
operationCheckboxes – stores information about operations chosen to be
tested.
The important methods are checkWithOracle(…) which checks if return value matches
the declared oracle and OpenOracle() which opens oracle declaration dialog.
The next important class – Messager, is often used by class UserInterface. The main
attributes of class Messager are desc – used for the storage of parsed WSDL document and
writer – used for writing to output file. The main methods of Messager are:
SendMessage – used for transfer information needed to generate SOAP
messages,
Validate – used for the validation of data,
getOut – used for retrieving output from SAOP message,
resolveComplexType and resolveSimpleType are used for managing data
types of WSDL document.
The class, called SOAPClient, is used by class Messager. The main attributes are
soapConnection, which saves information about SOAP connection with web service and
soapResponse, which stores SAOP message returned by the web service. The main methods
of this class are:
createSOAPRequest – method generates SOAP message to be send to the web
service,
createSOAPConnection – used for establishing SOAP connection with the web
service,
callSOAPConnection – method sends generated SOAP message to the web
service,
closeSOAPConnection – used for closing previously opened SOAP connection.
WSDLTest integrated testing modes
WSDLTest has three testing modes available for the user (shown on Screenshot 6):
1. Automatic – all tests are performed automatically. User only specifies the
number of tests to be performed. All possible operations (functions of web
service) defined in WSDL document must be tested. For each operation random
parameters are generated based on theirs defined in XML types.
2. Semi-automatic – parameters of operations are provided as regular expressions.
For these expressions values of parameters, used in tests, are generated. User can
specify number of tests and operations to be tested.
3. Manual – the simplest of modes, parameters of operations are given explicitly
by user.
47
In semi-automatic and automatic mode the oracle (containing expected test results) can
be defined.
Screenshot 6 – Choosing testing mode in WSDLTest
3.3 Tests
Below some test cases are presented. In each section one web service, which was used
as test case for WSDLTest is presented. Following web services were used and tested:
Currency Converter [48],
E-mail validate [49],
Credit Card number validate [50],
dohwebservice [51].
3.3.1 Currency Converter
Web service Currency Converter [48] is a very simple web service with only one
“method” named ConversionRate, used for calculating the ratio of two different currencies.
In Code Snippet 9 a part of WSDL document which presents data types taken by
ConversionRate is shown. There are converter values From [lines 4-5] and To [lines 6-7].
Both of those values are not types from the XML schema but their type is defined within the
WSDL document. This type called Currency [lines 11-17] contains boundaries [lines 13-15]
named in WSDL document enumeration. When enumeration is given for a variable, it can
only take specific values defined by enumeration.
48
1 <s:element name="ConversionRate">
2
<s:complexType>
3
<s:sequence>
4
<s:element minOccurs="1" maxOccurs="1"
5
name="FromCurrency" type="tns:Currency"/>
6
<s:element minOccurs="1" maxOccurs="1"
7
name="ToCurrency" type="tns:Currency"/>
8
</s:sequence>
9
</s:complexType>
10 </s:element>
11 <s:simpleType name="Currency">
12
<s:restriction base="s:string">
13
<s:enumeration value="AFA"/>
14
...
15
<s:enumeration value="TRY"/>
16
</s:restriction>
17 </s:simpleType>
Code Snippet 9 – Snippet of WSDL document of web service Currency Converter
Web service Currency Converter was used in testing of WSDLTest. Below the description of
testing in manual, automatic and semi-automatic mode is presented.
Manual mode
In order to avoid causing intentional faults, values of parameters cannot be entered to
TextInput (field which allows input of any string), as the enumeration is present. Values
have to be chosen from Spinner (field which allows to choose one value from set of values),
as seen in Screenshot 7.
49
Screenshot 7 – Spinner presentation
Screenshot 8 – Result presentation for manual mode of WSDLTest
The result of execution of WSDLTest in manual mode is shown in Screenshot 8; two
tables are presented:
1. First stores initial parameters values, in columns “Parameter’s Name” and
“Parameters Value”.
50
2. Second, named “RESULTS”, stores results of WSDLTest execution.
Semi-automatic mode and automatic mode
WSDLTest execution in semi-automatic and automatic mode for web service Currency
Converter is similar. Because of the enumeration restriction, values of operation
ConversionRate must be chosen, from list of possible values, not generated. Since on
enumeration list each element has its index, choice is made by generation of random number
(within the range of the list size). Corresponding element is taken form list of possible values.
In semi-automatic and automatic mode the test is repeated in the declared number of
trials given by the user, each time with different values of input parameters. On Screenshot 9
the window of WSDLTest, enabling to input the number of trials, is presented.
Screenshot 9 – Trials screen presentation
The results of execution in semi-automatic mode are presented in a tables identical to
the one presented for the manual mode (Screenshot 8). For many number of tests the reading
of results from table can be inconvenient for the user so only first 10 tests for each operation
are written into the table.
For semi-automatic and automatic modes all results and initial parameters are saved into
a WSDLTest-generated file called “results.csv”. This file can be opened using button called
51
“open”, visible on Screenshot 10. The Automatic mode returns overall compatibility test
result also shown on Screenshot 10
Screenshot 10 - Results for automatic mode
Oracle feature testing
Oracle window (detail description is provided in section 3.2.6), is generated based on
the information about types returned by the web service operation. For each return value there
is TextInput field which is shown on Screenshot 11. Oracle was tested with input parameter
given as regular expression: “\d\.\d+”, where \d stands for digit.
Screenshot 11 – Oracle window with input parameter
On Screenshot 12 the result of Oracle test is shown. The result value is 0.8674 which is
a match for previously entered regular expression. As the oracle is declared as regular
52
expression, the result value can be matched to it. The result of this check is presented in
column “Oracle compatibility”.
Screenshot 12 – Result of test of oracle compatibility
3.3.2 E-mail Validate
Web service E-mail Validate [49] is simple web service with only four operations inside
the SOAP interface, which are as follows:
1. VerifyMXRecord
2. AdvancedVerifyEmail
3. VerifyEmail
4. ReturnCodes
Operation named AdvancedVerifyEmail, presented in Code Snippet 10, has input
parameter inside tag AdvancedVerifyEmail [lines 1-12] and output parameter inside tag
AdvancedVerifyEmailResponse [lines 14-22]. Input parameter is complex type with values
that do not have to be provided (when minOccurs="0" [lines 4 and 8]), while output value
named ReturnIndicator [lines 24-35] of this operation is complex type with 4 different
elements: ResponseText [lines 26-27] of type string, ResponseCode [lines 28-29] of type
integer, LastMailServer [lines 30-31] of type string and GoodEmail [lines 32-33] of type
boolean.
53
1 <s:element name="AdvancedVerifyEmail">
2
<s:complexType>
3
<s:sequence>
4
<s:element minOccurs="0" maxOccurs="1"
5
name="email" type="s:string"/>
6
<s:element minOccurs="1" maxOccurs="1"
7
name="timeout" type="s:int"/>
8
<s:element minOccurs="0" maxOccurs="1"
9
name="LicenseKey" type="s:string"/>
10
</s:sequence>
11
</s:complexType>
12 </s:element>
13
14 <s:element name="AdvancedVerifyEmailResponse">
15
<s:complexType>
16
<s:sequence>
17
<s:element minOccurs="0" maxOccurs="1"
18
name="AdvancedVerifyEmailResult"
19
type="tns:ReturnIndicator"/>
20
</s:sequence>
21
</s:complexType>
22 </s:element>
23
24 <s:complexType name="ReturnIndicator">
25
<s:sequence>
26
<s:element minOccurs="0" maxOccurs="1"
27
name="ResponseText" type="s:string"/>
28
<s:element minOccurs="1" maxOccurs="1"
29
name="ResponseCode" type="s:int"/>
30
<s:element minOccurs="0" maxOccurs="1"
31
name="LastMailServer" type="s:string"/>
32
<s:element minOccurs="1" maxOccurs="1"
33
name="GoodEmail" type="s:boolean"/>
34
</s:sequence>
35 </s:complexType>
Code Snippet 10 – Snippet of WSDL document of E-mail Validate web service
Manual mode
For WSDLTest execution in manual mode, user should provide parameters as strings
for web service E-mail Validate. The input window is identical as in case of semi-automatic
mode (Screenshot 14). The results of execution WSDLTest in manual mode are presented
inside two tables: initial parameter values and results. These tables are shown on Screenshot
13.
54
Screenshot 13 – Results tables for manual mode
Semi-automatic mode
Inside the input window of WSDLTest, visible on Screenshot 14, user provided regular
expression “[a-z]{5,10}\@gmail\.com” is designed for generating of random e-mails within
the domain “gmail.com”. In semi-automatic mode, all provided parameters can be regular
expressions and they are tested for validity.
55
Screenshot 14 – Input for semi-automatic mode
Parameters of web service operation VerifyEmail were generated basing on provided
regular expression and send to the web service. The preview of result of execution in semi-
automatic mode is visible in two tables just like in manual mode (Screenshot 13). Full set of
results is available inside generated by WSDLTest file called “results.csv”. The structure of
the file with some sample of results is shown in Table 3.
Fist two columns of Table 3 contain previously generated input parameters. Each return
parameter has two columns describing it. First one contains values of parameter for all tests;
second one named “TypeCompatibility” contains Boolean values Yes/No, which indicates
compatibility of returned value’s type with type declared in WSDL document.
56
email License
Key
Last
Server
Type
Comp
Response
Text
Type
Comp
Response
Code
Type
Comp
Good
Type
Comp
vcuwt@
gmail.com
null Gmail-
smtp-in.l.
google.com
Yes User Not
Found
Yes 4 Yes false Yes
urrdstt@
gmail.com
null null Yes Current
license
key only
allows so
many
checks
Yes 9 Yes false Yes
Table 3 – Snipped of results file
Automatic mode
In automatic mode execution of WSDLTest, parameters are generated randomly. All
results and initial parameters are saved in file. The structure of this file is the same as
presented in Table 3.
3.3.3 Validate Card Number
Web service Validate Card Number [50] is simple web service with only one “method”
named ValidateCardNumber, used for credit card number validation. Code Snippet 11
presents the definition of operation ValidateCardNumber [lines 21-30], input parameter of
operation ValidateCardNumber [lines 1-10] and output of operation parameter
ValidateCardNumberResponse [lines 11-19].
57
1 <s:element name="ValidateCardNumber">
2
<s:complexType>
3
<s:sequence>
4
<s:element minOccurs="0" maxOccurs="1"
5
name="cardType" type="s:string"/>
6
<s:element minOccurs="0" maxOccurs="1"
7
name="cardNumber" type="s:string"/>
8
</s:sequence>
9
</s:complexType>
10 </s:element>
11 <s:element name="ValidateCardNumberResponse">
12
<s:complexType>
13
<s:sequence>
14
<s:element minOccurs="0" maxOccurs="1"
15
name="ValidateCardNumberResult"
16
type="s:string"/>
17
</s:sequence>
18
</s:complexType>
19 </s:element>
20 ...
21 <wsdl:portType name="CCCheckerSoap">
22
<wsdl:operation name="ValidateCardNumber">
23
<wsdl:documentation xmlns:wsdl=
24
"http://schemas.xmlsoap.org/wsdl/">
25
Please enter card type as VISA or
26
MASTERCARD or DINERS or AMEX
27
</wsdl:documentation>
28
<wsdl:input message=
29
"tns:ValidateCardNumberSoapIn"/>
30
</wsdl:operation>
31 </wsdl:portType>
Code Snippet 11 – Web service Credit Card
Semi-automatic mode
Two cases were tested in semi-automatic mode of WSDLTest:
In first case, parameter cardNumber was inputted as regular expression which matches
any random sequence of digits of length equal 16 (typical length of credit card number). Input
window is seen on Screenshot 15.
58
Screenshot 15 – Input window
Results, which are visible in table “RESULTS” on Screenshot 16, indicate whether the
card number generated by WSDLTest is valid or not.
Screenshot 16 – Result for parameters given as digits
In the second case, since an input parameter cardNumber is defined as string [line 11] of
Code Snippet 11, random string consisting of letters was used. Credit card number can only
consist of numbers, so using letters as values caused the web service to return fault and stack
trace. WSDLTest was able to find fault inside returned SOAP message [1] and print it.
Results tables are shown in Screenshot 17.
59
Screenshot 17 – Result for parameters given as random string
In Table 3, full text of web service inner stack trace is presented.
Faultcode:
soap:ServerSystem.Web.Services.Protocols.SoapException: Server was
unable to process request. ---> System.FormatException: Input string
was not in a correct format.
at System.Number.StringToNumber(String str, NumberStyles
options, NumberBuffer& number, NumberFormatInfo info, Boolean
parseDecimal)
at System.Number.ParseInt32(String s, NumberStyles style,
NumberFormatInfo info)
at System.Convert.ToInt32(String value)
at CCChecker.IsMasterCard(String cc)
at CCChecker.ValidateCardNumber(String cardType, String
cardNumber)
--- End of inner exception stack trace ---
Table 4 – Fault code returned by Credit Card
Testing of web service Validate Card Number showed how WSDLTest finds errors
inside web service and how they are indicated.
60
3.3.1 dohwebservice
Web service dohwebservice is a simple web service with two operations:
RequestStatusByCitizenID and RequestSubmitService. The parameter returned by the
first function is declared in RequestStatusByCitizenIDResponse lines [1-9], and it is a
complex type consisting of another complex type named ArrayOfRequestStatusResult
lines [10-16]. The curious about this case is that ArrayOfRequestStatusResult consists of
yet another complex type value RequestStatusResuls lines [17-30]. This parameter has
inside number of elements of type string. This example shows how WSDLTest deals with
recursion in types.
1 <s:element name="RequestStatusByCitizenIDResponse">
2
<s:complexType>
3
<s:sequence>
4
<s:element minOccurs="0" maxOccurs="1"
5
name="RequestStatusByCitizenIDResult"
6
type="tns:ArrayOfRequestStatusResult"/>
7
</s:sequence>
8
</s:complexType>
9 </s:element>
10 <s:complexType name="ArrayOfRequestStatusResult">
11
<s:sequence>
12
<s:element minOccurs="0" maxOccurs="unbounded"
13
name="RequestStatusResult" nillable="true"
14
type="tns:RequestStatusResult"/>
15
</s:sequence>
16 </s:complexType>
17 <s:complexType name="RequestStatusResult">
18
<s:sequence>
19
<s:element minOccurs="0" maxOccurs="1"
20
name="CITIZEN_ID" type="s:string"/>
21
<s:element minOccurs="0" maxOccurs="1"
22
name="REQUEST_NAME" type="s:string"/>
23
<s:element minOccurs="0" maxOccurs="1"
24
name="REQUEST_DATE" type="s:string"/>
25
<s:element minOccurs="0" maxOccurs="1"
26
name="HIGHWAY" type="s:string"/>
27
<s:element minOccurs="0" maxOccurs="1"
28
name="REQUEST_STATUS" type="s:string"/>
29
</s:sequence>
30 </s:complexType>
Code Snippet 12 – Snippet of WSDL document of dohwebservice
61
Semi-automatic mode
For semi-automatic mode parameter were inputted as regular expression. Operation has
only one parameter named citizen_id. Fifteen trials were declared, as can be seen on
Screenshot 18.
Screenshot 18 – Parameter input for semi-automatic mode
WSDLTest were able to find all the names of elements, which should be present inside
returned SOAP message. However, as visible on Screenshot 19, no values were returned by
web service.
62
Screenshot 19 – Results for semi-automatic mode
63
4 Examples
Below the examples of how to use WSDLTest are presented, as well as Use Case
diagram and sequence diagram.
Choose mode
USER
Perform testing
Create Oracle
Provide WSDL address
<<include>>
<<extend>>
Fig. 12 – Use case diagram
Title – Provide WSDL address
Description
User enters address of WSDL document. WSDL is successfully parsed by WSDLTest.
Primary Actor – User
Preconditions – none
Postconditions – WSDL document was successfully parsed.
Main Success Scenario
1. User inserts into TextInput named “WSDL file address” the address of WSDL.
2. User clicks “Apply” button.
3. WSDLTest prompts user with message: “WSDL document applied.”
Extensions
3a. No internet connection.
3a1. WSDLTest displays error message “No internet connection.”
3b. WSDL document address incorrect or not working.
64
3b1. WSDLTest displays error message “WSDL document could not be
applied.”
Title – Perform testing
Description
User chooses testing mode. Than user can specify parameters and number of tests to be
performed. User can provide oracle for his test.
Primary Actor – User
Preconditions –WSDL document was successfully applied.
Postconditions – User can familiarize with results.
Main Success Scenario
1. User selects testing mode.
2. Depending on testing mode WSDLTest displays:
2.1. Manual and Semi-automatic – available web service operations and button
“Next”.
2.2. Automatic – button “Next”.
3. If needed, the user chooses the web service operations to be tested and clicks
“Next” button.
4. Depending on the testing mode WSDLTest displays:
4.1 In manual mode – TextInput for each parameter for all chosen operation,
oracle Checkbox and button “Next”.
4.2 In semi-automatic mode – TextInput for each parameter for all chosen
operation, oracle Checkbox, TextInput for specifying number of tests and
button “Next”.
4.3 In automatic mode – TextInput for specifying number of tests, oracle
Checkbox and button “Next”.
5. User provides necessary data. Additionally user can create oracle, inside oracle
dialog, by clicking on oracle Checkbox. Then user clicks “Next” button.
6. Depending on testing mode WSDLTest displays:
6.1 In manual mode – two tables: one with initial parameters values, second
with results and button “Begin”.
6.2 In semi-automatic mode – two tables: one with initial parameters values,
second with results and buttons: “Begin”, “Open”.
6.3 In automatic mode – information “Results were saved to file” and buttons:
“Begin”, “Open”.
65
Extensions
1a. Internet connection lost.
1a1. WSDLTest will timeout.
3a. Internet connection lost.
3a1. WSDLTest will timeout.
5a. Internet connection lost.
5a1. WSDLTest will timeout.
5b. Parameter type was not valid or parameter was not valid regular expression.
5b1. WSDLTest warns about not successful parameter validity check and
waits new for input.
5c. Oracle parameter was not proper regular expression.
5c1. WSDLTest warns about not successful oracle validity check and waits
for new input.
In Fig. 13 simplified sequence diagram is presented. It shows how program executes
trough successful scenario, when manual mode is chosen.
66
User UserInterface Messager
Apply WSDL
Choose Mode
Choose Operations
Set parameters values
SAOPClient
setWsdlurl
getOperations
getOperationPrameters
SendMessage
createSOAPConnection
callSOAPConnection
createSOAPRequest
getOutput
printResults
Fig. 13 – Simplified sequence diagram
67
5 Conclusion
Web service can be specified in a WSDL [11] document. This document may be used as
a basis in the automatic testing of the web service. We developed an application, called
WSDLTest, which uses the WSDL document to prepare tests of web service. WSDLTest
allows user, with basic technical knowledge, to test a web service. Our application can test
web service based on its WSDL document and provided by user (tester) oracle. Testing can be
performed in fully automatic mode for theoretically infinite number of trials with different
parameters. The automated process of testing includes the generation of parameter values, the
comparison of results with oracle and a basic validity check of the WSDL document. Tester
can also provide parameter values and oracle in form of regular expression. Such form is
more general than the explicit one. Values of parameters are generated in such way that they
match the provided regular expression. Regular expressions used in oracle are used to
generate expected values of results, which are compared with actual results of tests.
WSDLTest can test a variety of web services described by WSDL. This tool is able to
perform the whole testing process. It starts from obtaining information about a web service
from WSDL document and also gets the test’ results from the oracle. Several test cases,
described in chapter 3.3, show that WSDLTest can be very useful in testing web services.
Our tool has some drawbacks compared to other tools available (presented in chapter
2.3). In current version, WSDLTest is not able to read parameters from external files or
mimicking web service. Out tool might have some problems in testing web services with very
complex schema of data. These drawbacks can be eliminated in future.
WSDLTest has also an advantage over other similar tools. It can generate values
of parameters from regular expressions; such functionality is not common in other tools
(chapter 2.3).
Due to the time constraints, some solutions in the WSDLTest are very simple, not
optimal but they could be improved in future versions. WSDLTest has only basic WSDL
document validation. In future, we plan to implement the full validation based on W3C
standard. The method of testing we used in WSDLTest also could be improved. The
generation of parameters’ values should be revisited. The implementation of combinatorial
testing might increase the scope of testing. Complex WSDL documents e.g. with customs’
data schemes should be accepted by WSDLTest.
68
6 References
[1] W3C – World Wide Web Consortium, "SOAP – Simple Object Access Protocol,"
[Online]. Available: http://www.w3.org/TR/soap/. [Accessed October 2013].
[2] W3C – World Wide Web Consortium, "WSDL – Web Services Description Language,"
[Online]. Available: http://www.w3.org/TR/wsdl. [Accessed October 2013].
[3] W3C – World Wide Web Consortium, "Official Website," [Online]. Available:
http://www.w3.org/. [Accessed October 2013].
[4] W3C – World Wide Web Consortium, "Official Website," October 2013. [Online].
Available: http://www.w3.org/.
[5] W3C documentation, "The World Wide Web Consortium," [Online]. Available:
http://www.w3.org/TR/ws-arch/. [Accessed January 2014].
[6] W3C – World Wide Web Consortium, "HTTP – Hypertext Transfer Protocol," [Online].
Available: http://www.w3.org/Protocols/. [Accessed October 2013].
[7] SMTP – Simple Mail Transfer Protocol, [Online]. Available:
http://en.wikipedia.org/wiki/Simple_Mail_Transfer_Protocol. [Accessed October 2013].
[8] FTP – File Transfer Protocol, [Online]. Available:
http://en.wikipedia.org/wiki/File_Transfer_Protocol. [Accessed October 2013].
[9] BEEP – Blocks Extensible Exchange Protocol, [Online]. Available:
http://en.wikipedia.org/wiki/BEEP. [Accessed October 2013].
[10] XML-RPC, "XML-based Remote Procedure Call," [Online]. Available:
http://en.wikipedia.org/wiki/XML-RPC. [Accessed October 2013].
[11] W3C – World Wide Web Consortium, "WSDL – Web Services Description Language,"
[Online]. Available: http://www.w3.org/TR/wsdl. [Accessed October 2013].
[12] UDDI – Universal Description, Discovery and Integration, [Online]. Available:
http://uddi.xml.org/. [Accessed October 2013].
[13] RPC – Remote procedure call, [Online]. Available:
http://en.wikipedia.org/wiki/Remote_procedure_call. [Accessed October 2013].
[14] W3C – World Wide Web Consortium, "XML – Extensible Markup Language," [Online].
Available: http://www.w3.org/XML/. [Accessed October 2013].
[15] W3C – World Wide Web Consortium, "HTML – HyperText Markup Language,"
[Online]. Available: http://www.w3.org/community/webed/wiki/HTML. [Accessed
October 2013].
[16] W3C – World Wide Web Consortium, "Official Website," [Online]. Available:
http://www.w3.org/. [Accessed October 2013].
[17] W3C – World Wide Web Consortium, "XML Schema," [Online]. Available:
http://www.w3.org/TR/xmlschema-2/. [Accessed October 2013].
69
[18] WSI – The OASIS Web Services Interoperability, [Online]. Available: http://www.oasis-
ws-i.org. [Accessed October 2013].
[19] Hewlett-Packard, "HP QuickTest Professional," [Online]. Available:
http://www.hp.com/. [Accessed October 2013].
[20] Parasoft, "Parasoft SOAtest," [Online]. Available: http://www.parasoft.com/. [Accessed
October 2013].
[21] "ESB – Enterprise service bus," [Online]. Available:
http://en.wikipedia.org/wiki/Enterprise_service_bus. [Accessed January 2014].
[22] Crosscheck, "SOAPSonar," [Online]. Available: http://www.crosschecknet.com/.
[Accessed October 2013].
[23] Smart-Bear, "SoapUI," [Online]. Available: http://www.soapui.org/. [Accessed October
2013].
[24] Microsoft, ".NET Framework," [Online]. Available:
http://en.wikipedia.org/wiki/.NET_Framework. [Accessed January 2014].
[25] Oracle, "Java – What is Java?," [Online]. Available:
http://www.java.com/en/download/faq/whatis_java.xml. [Accessed November 2013].
[26] Oracle, "JRE – System Requirements," [Online]. Available:
http://docs.oracle.com/javase/7/docs/webnotes/install/windows/windows-system-
requirements.html. [Accessed October 2013].
[27] Oracle, "Java Language," [Online]. Available: http://docs.oracle.com/javase/specs/.
[Accessed December 2013].
[28] "Abstract Window Toolkit," [Online]. Available:
http://en.wikipedia.org/wiki/Java_AWT. [Accessed December 2013].
[29] "Java Swing," [Online]. Available: http://en.wikipedia.org/wiki/Java_Swing. [Accessed
December 2013].
[30] "Rich Internet application," [Online]. Available:
http://en.wikipedia.org/wiki/Rich_Internet_application. [Accessed December 2013].
[31] "SwingX," [Online]. Available: http://en.wikipedia.org/wiki/SwingLabs#Sub-projects.
[Accessed December 2013].
[32] "JGoodies," [Online]. Available: http://www.jgoodies.com/. [Accessed December 2013].
[33] "JavaFX," [Online]. Available: http://docs.oracle.com/javafx/index.html. [Accessed
December 2013].
[34] "Apache Pivot," [Online]. Available: http://pivot.apache.org. [Accessed December 2013].
[35] "Apache Commons," [Online]. Available: http://commons.apache.org/proper/commons-
lang/. [Accessed December 2013].
[36] "Xeger," [Online]. Available: https://code.google.com/p/xeger/. [Accessed December
2013].
[37] "Xeger Limitations," [Online]. Available:
70
https://code.google.com/p/xeger/wiki/XegerLimitations. [Accessed December 2013].
[38] "dk.brics.automaton," [Online]. Available: http://cs.au.dk/~amoeller/automaton/.
[Accessed December 2013].
[39] "Deterministic finite automaton," [Online]. Available:
http://en.wikipedia.org/wiki/Deterministic_finite_automaton. [Accessed December 2013].
[40] "Nondeterministic finite automaton," [Online]. Available:
http://en.wikipedia.org/wiki/Nondeterministic_finite_automaton. [Accessed December
2013].
[41] "Logger," [Online]. Available:
http://docs.oracle.com/javase/6/docs/api/java/util/logging/Logger.html. [Accessed
December 2013].
[42] "Try and Catch block," [Online]. Available:
http://www.javamex.com/tutorials/exceptions/exceptions_try_catch_block.shtml.
[Accessed December 2013].
[43] P. Link, "Easy WSDL Toolbox," [Online]. Available: http://easywsdl.ow2.org/.
[Accessed January 2014].
[44] Apache Fundation, "Woden," [Online]. Available: http://ws.apache.org/woden/].
[Accessed January 2014].
[45] W3C, "XSL – Extensible Stylesheet Language," [Online]. Available:
http://www.w3.org/Style/XSL/. [Accessed January 2014].
[46] Apache Fundation, "Web Services Description Language for Ja," [Online]. Available:
http://wsdl4j.sourceforge.net/. [Accessed January 2014].
[47] "Comma-separated values," [Online]. Available: http://en.wikipedia.org/wiki/Comma-
separated_values. [Accessed December 2013].
[48] WSDL, "Currency Coverter," [Online]. Available:
http://www.webservicex.com/CurrencyConvertor.asmx?wsdl. [Accessed December
2013].
[49] WSDL, "Verify Email," [Online]. Available:
http://ws.cdyne.com/emailverify/Emailvernotestemail.asmx?wsdl. [Accessed December
2013].
[50] WSDL, "Credit Card," [Online]. Available:
http://www.webservicex.net/CreditCard.asmx?WSDL. [Accessed December 2013].
[51] WSDL, "dohwebservice," [Online]. Available: http://e-
services.doh.go.th/dohweb/dohwebservice.asmx. [Accessed December 2013].
71
Code Snippet 1 – XML schema example ................................................................................. 10
Code Snippet 2 – SOAP Message example ............................................................................. 11
Code Snippet 3 – Frame of WSDL document .......................................................................... 13
Code Snippet 4 – Tags and properties inside description element ........................................... 13
Code Snippet 5 – Overview of “types” section ........................................................................ 14
Code Snippet 6 – Overview of “interface” section .................................................................. 15
Code Snippet 7 – Overview of “binding” section .................................................................... 15
Code Snippet 8 – Overview of “service” section ..................................................................... 16
Code Snippet 9 – Snippet of WSDL document of web service Currency Converter .............. 48
Code Snippet 10 – Snippet of WSDL document of E-mail Validate web service ................... 53
Code Snippet 11 – Web service Credit Card ............................................................................ 57
Code Snippet 12 – Snippet of WSDL document of dohwebservice ........................................ 60
Fig. 1 – Web Service schema
Available: http://www.emeraldinsight.com/content_images/fig/2630210407002.png
[Accessed October 2013] ........................................................................................................... 9
Fig. 2 – Difference between WSDL 1.1 and WSDL 2.0
Available: http://www.emeraldinsight.com/content_images/fig/2630210407002.png
[Accessed October 2013] ......................................................................................................... 12
Fig. 3 – Graphical User Interface – main parts ........................................................................ 24
Fig. 4 – High-level diagram presenting WSDLTest ................................................................ 31
Fig. 5 – Core module ................................................................................................................ 37
Fig. 6 – WSDLTest level 2 ....................................................................................................... 38
Fig. 7 – Control flow of GUI panel .......................................................................................... 40
Fig. 8 – Oracle dialog ............................................................................................................... 41
Fig. 9 – Interface structure of SOAPClient class ..................................................................... 42
Fig. 10 – Structure of WSDLTest ............................................................................................ 43
Fig. 11 – Simplified class diagram ........................................................................................... 45
Fig. 12 – Use case diagram ...................................................................................................... 63
Fig. 13 – Simplified sequence diagram .................................................................................... 66
Screenshot 1 – Test steps records editor page in HP QuickTest Professional
Available: http://h20195.www2.hp.com/V2/GetPDF.aspx%2F4AA3-1748ENW.pdf
[Accessed October 2013] 18
Screenshot 2 – SOAtest testing environment
Available: http://www.parasoft.com/jsp/images/2108.gif [Accessed October 2013] 19
Screenshot 3 – Navigation within WSDL document in SOAPSonar
Available: http://www.parasoft.com/jsp/images/2108.gif [Accessed October 2013] 20
Screenshot 4 – Functional test created from WSDL
Available: http://www.soapui.org/images/stories/cssgallery/images/functional-
testing_665_515_95.png [Accessed October 2013] 21
72
Screenshot 5 – Graphical User Interface after application initialization 39
Screenshot 6 – Choosing testing mode in WSDLTest 47
Screenshot 7 – Spinner presentation 49
Screenshot 8 – Result presentation for manual mode of WSDLTest 49
Screenshot 9 – Trials screen presentation 50
Screenshot 10 - Results for automatic mode 51
Screenshot 11 – Oracle window with input parameter 51
Screenshot 12 – Result of test of oracle compatibility 52
Screenshot 13 – Results tables for manual mode 54
Screenshot 14 – Input for semi-automatic mode 55
Screenshot 15 – Input window 58
Screenshot 16 – Result for parameters given as digits 58
Screenshot 17 – Result for parameters given as random string 59
Screenshot 18 – Parameter input for semi-automatic mode 61
Screenshot 19 – Results for semi-automatic mode 62
Table 1 – Comparison of WSDL parsers functionality ............................................................ 36
Table 2 – Structure of data inside results file ........................................................................... 39
Table 3 – Snipped of results file ............................................................................................... 56
Table 4 – Fault code returned by Credit Card .......................................................................... 59
6.1 Glossary
Black Box – It is a system or object, whose input, output and transfer characteristic can be
obtained without information on internal algorithm or how its function is performed.
Therefore, its implementation is not visible – it is black.
Acronyms and Abbreviations
API – Application Programming Interface
BEEP – Blocks Extensible Exchange Protocol
DFA – Deterministic Finite Automaton
ESB – Enterprise Service Bus
FTP – File Transfer Protocol
GUI – Graphical User Interface
HTML – Hypertext Markup Language
HTTP – Hypertext Transport Protocol
NFA – Nondeterministic Finite Automaton
RPC – Remote Procedure Call
RIA – Rich Internet Application
73
SMTP – Simple Mail Transport Protocol
SOAP – Simple Object Access Protocol
UDDI – Universal Description Discovery and Integration
W3C – World Wide Web Consortium
WSDL – Web Service Description Language
WWW – World Wide Web
XML – Extensible Markup Language
XSL –Extensible Stylesheet Language