How to Use Apache JMeter for Web Services Testing...Recently, we were able to do some clustering related load testing on Apache Axis2 web services engine focusing on state replication property. I thought of sharing things we did in our tests which were carried out using Apache JMeter.

Apache JMeter is a fully open source pure java tool designed for the purpose of load testing and measuring performance of web applications. According to the documentation, it can be used to test both static and dynamic web resources such as files, Servlets, web services, Perl scripts, Java Objects, Data Bases and Queries, FTP Servers etc. JMeter can effectively use to simulate heavy loads on the above resource types and analyze the overall performance under different load  conditions such as concurrent loads. It also provide various types of visualization techniques to graphically analyze the results obtained from the load tests.

Here are some advantages of using JMeter1. Open Source free tool developed by Apache Software Foundation 

2. Has an active community, good documentation, user manuals and plenty of resources which makes easy to learn, get start and running

3. Can load test in variety of server types such as HTTP, FTP, SOAP, Database, LDAP, JMS, Mail servers etc.

4. Support large-scale testing providing concurrency and ability to use various types of loads.

5. Support for distributed stress testing using multiple systems

6. API and plug-ins available for customization

7. Provides variety of test results manipulation and visualization methods

Let's get into the details.

Setting up the Environment JDK 1.5 or later version is required for JMeter latest released version 2.7. Get more details about tested JDK versions and compatible operating systems from here. You can go through my previous post  to get more details about setting up Sun JDK 1.6 in Linux environment.

Download JMeter binaries or checkout and build JMeter source available at repositories. Unpack the archive(zip or tgz) to a local folder. Let [jMeterHome] be the unpacked directory. Run JMeter from the terminal as follows.

   Figure 01. JMeter GUI.

Setting up Apache Axis2 and deploy the service

Here we will be deploying sample service available in Apache Axis2 called StockQuoteService which provides two method calls.

?12345678910111213141516171819

package samples.quickstart.service.pojo; import java.util.HashMap; public class StockQuoteService {    private HashMap map = new HashMap();     public double getPrice(String symbol) {        Double price = (Double) map.get(symbol);        if(price != null){            return price.doubleValue();        }        return 42.00;    }

 public void update(String symbol, double price) {    map.put(symbol, new Double(price));    }}

You need to build the service definition file (StockQuoteService.aar) and place it in "[Axis2_Home]/repository/services" directory in order to deploy it as an axis2 service(Axis2 running in the standalone mode). If axis2 runs inside a servlet engine like Apache Tomcat relevant directory should be "webapps/axis2/WEB-INF/services".

Axis2 quick start guide and installation guide provides extensive details about creating and deploying services in different ways with Apache Axis2.

Since our purpose is to test the state replication in Axis2 cluster, we need to enable clustering from the axis2.xml in "[Axis2_Home]/conf" directory. To deploy a axis2 cluster, two(or any other number) axis2 instances shoud be started, where "domain name" is set to same value and state replication property is enabled. Make sure instances can communicate among each other, depending on the membership scheme used for clustering.

 If we deploy the two instances in the same host, we need to change the "transport listener" and "localMemberPort" as well. In this scenario transport listener ports are set to "8080 and 8081" respectively in two axis2 servers which are running in the same local machine.

Start the axis2 servers as follows after configuring the axis2 cluster.

Then check to make sure that the service has been properly deployed by viewing the list of services at,

Below is an snapshot of the browser in the above URL once you correctly deployed the service.

   Figure 02. Deployed Axis2 services in the browser

Getting started with load Testing

a) Creating a test plan 

Using the JMeter GUI, you can create various types of test plans including, Web service test plan, Database test plan, FTP test plan etc.

Test plan created trough the GUI is saved in .jmx format. Although test plan can be edited by  following the JMeter xml schema, modifying the test plan using GUI is much easier.

In the landing view of JMeter, there are two basic elements. i.e Test plan and Work bench.  

Right click on the Test plan and add a Thread Group element. Create two SOAP/XML-RPC Requests for two method calls associated with the StockQuoteService service.

b) Configuring the test plan

In the Thread Group element, set the thread properties, Number of threads(users) : No. of concurrent users to be simulated.

Ramp - up period(in seconds) :Time taken by the JMeter to invoke the threads

Lop count : No of times the test plan should be repeated.

These properties can be varied depending on the requirements of the load test.Shown below are the steps to follow while adding above elements.

   Figure 03. Adding Thread group

   Figure 04. Adding SOAP/XML-RPC request

In the "updatePrice request", set service URL and the SOAP action properties as shown below. Set SOAP/XML-RPC data as,

?123456789101112131415

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://quickstart.samples/xsd">

    <soapenv:Header/>

    <soapenv:Body>

       <xsd:update>

          <!--Optional:-->

          <xsd:symbol>?</xsd:symbol>

          <!--Optional:-->

          <xsd:price>${__Random(1,1000,)}</xsd:price>

 

161718192021

      </xsd:update>

    </soapenv:Body>

 </soapenv:Envelope>

   Figure 05. updatePrice request view

Here, updatePrice request will be send to a one instance running on port "8080" and the getprice request will be send to the other instance running on "8081" to test the state replication property of the axis2 cluster.

Similarly, set the relevant properties in the "getPrice request" element. Set the XML-RPC Data as,?1234567891011

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://quickstart.samples/xsd">

    <soapenv:Header/>

    <soapenv:Body>

       <xsd:getPrice>

          <!--Optional:-->

          <xsd:symbol>?</xsd:symbol>

121314151617

       </xsd:getPrice>

    </soapenv:Body>

 </soapenv:Envelope>

   Figure 06. getPrice request view

Add a timer to set the "thread delay time". This can be a constant time or random timer such as "Gaussian random timer" to get a more realistic behaviour of the system.

Figure 07. Adding a Timer Element

To visualize the data, we can add Listeners like "Graph results", "View Results in a table", "View results tree" and "Spline Visualizer". In addition we can also send the responses of the service requests to a file by adding a "Write results to a file" in the above element properties. This is an essential feature for debugging and examining errors.

c) Running the test plan

After all the required fields in the test plan filled, save it. Run the test plan from Run-->start or pressing ctrl+r. Use the "Graph results" element to view the chart which is being populated when the load test is running. Graph types like 'Data, average, throughput' can be enabled in the chart to get a clear idea about the test results. Here throughput means the number of requests processed by the server within a minute.

   Figure 08.  Graph results element view.

In the "View results in the table" element, status column illustrates whether service request completed successfully or failed.

   Figure 09. View results in the table

To simulate high load on the server we can increase the no of threads and the loop count.

Once the test is done, "results.jmx" file will be created and this file can be used to visualize the results.loadosophia is one of the sites which offers graph generation services using the XML/CSV results files from JMeter and many other file types such as Apache benchmark tool files etc. You can log into the site using Google account and it provides a workspace to maintain test results related to various projects. Upload the .jmx file and set of graphs will be generated against various parameters such as response time, throughput etc

///////////////////////////////////////////////////

API Testing with JMeterBy Maurice Kelly

Application programming interfaces, or APIs to their friends, have become the

cornerstone of software development. While APIs were traditionally a source

code-based interface, the ever more pervasive nature of the Web has made

Web APIs as prominent as the traditional flavour. Many online services are

keen to open up their services via the use of APIs in an attempt to drive an

ecosystem around third-party applications.

At the basic level an API is a contract - it is a published means of sending and

receiving data between two entities, be they modules within a single binary

application or a client and server communicating across a HTTP connection.

When an API is published so that clients can be written to use it they need the

API to remain consistent.

Testing a Web APITest-driven development has become an important principle in the creation of

software and for good reason. A suite of unit-tests can be run before and after

major code changes to verify that existing behaviour is not negatively

impacted by a regression. A Web API is no different and needs to be

rigorously tested as well. The best way to do so is to make calls against it with

a known set of parameters and verify that the results returned are consistent

between changes. A good practice would be to set up a continuous integration

(CI) server like Jenkins, or Team City. The CI server can detect changes to

the codebase and trigger a build, install and test cycle.

The testing can be performed by sending a series of requests to the API and

inspecting the results. The testing can be simple - send a request, capture the

response and check to see that it matches the expected response. A simple

tool likecurl could easily be used to do this on a small scale, but once you

start ramping up the number of tests - and introduce conditional tests - it will

soon become unmanageable. To perform more complex testing of an API you

need a tool likeJMeter.

JMeter originated from the Apache Foundation as a HTTP load testing tool,

but since its inception has matured into a functional and load testing tool for

numerous protocols. It is an Open Source project with a plugin-based

architecture making it readily extendible if you know your Java. It is useful for

performing intricate functional tests, as well as small to medium scale load

testing.

Setting up JMeterJMeter is based around the concept of a Test Plan that contains a series

ofsamplers, assertions and listeners. When you open JMeter for the first

time you'll be presented with an empty document containing two nodes in the

tree: a test plan, and a Workbench. Your real work will go into the test plan,

the Workbench is like a scratchpad for experimenting - but be warned, its

contents are not saved when you hit save!

Due to its history as a load test tool, JMeter requires that you perform all of

your test work within a thread group. A thread group is a container that

allows you to configure parts of your test plan to run both in parallel and

consecutively. This is important if you want to stress test your API or test for

data integrity across concurrent transactions and can be achieved by

increasing the values of the thread group parameters “Number of Threads”

and “Loop Count”. If you plan to perform more complex functional tests it is

better to keep the parameters at 1 otherwise you may introduce a bottleneck

in JMeter and not thoroughly load test your API.

To capture the output of your tests you'll need to insert some sort of listener.

You can add a listener to the end of a single thread group to capture the

results of just that group, or add at the end of the test plan to capture the

results from all test groups. Basic listeners that are good for capturing simple

test results include the “View Results Tree” or “View Results in Table”

listeners.

Creating TestsThere are two more plugin types that you will need to add to your test plan in

order to perform functional testing. The first is a sampler - a sampler is used

to make a request, read a log file, anything that takes a sample of data. Once

you have a sample of data you need the second type of plugin, an assertion.

When you attach an assertion to a sampler, you are performing a test by

asserting that some property of the sample data is the same as the value you

expect it to be.

To perform a test against a RESTful API you can use a HTTP Request

sampler. This sampler can be configured with a wide array of parameters that

relates to how it will perform the request. Say, for example, you had a

resource that returned data about cars and performing a GET request

against http://example.org/cars/1 returns data about the car that has the

ID of 1. You would configure the HTTP Request sampler with the key values

shown below:

Server Name or IP: example.org Port: 80 Method: GET Path: /cars/1

Performing a test on the data returned requires that you add an assertion to

the sampler. There are a number of different assertion types and the one you

use will often be dependent on the format of the data you are attempting to

test. If the data returned by your API is XML you could use an XML Assertion

or an XPath Assertion. If you want to check the HTTP response code you can

use the Response Assertion.

Unfortunately if the data is JSON - as is often the case with RESTful APIs -

there is no specific assertion for that yet, but there is the BSF (Bean Scripting Framework) Assertion as a fallback. The BSF Assertion gives you

access to a script processor, the data from the previously run sampler, and

the ability to pass parameters to a script. With this knowledge you can write

some Javascript that will process the content of the response, and perform

tests against it. A sample script for testing the make of the car is shown below:

try { eval('var apiResponse = ' + prev.getResponseDataAsString());

var make_of_car = args[0];

if (apiResponse) { var car = apiResponse; if (car.make == make_of_car) { prev.setSuccessful(true); } else { prev.setSuccessful(false); prev.setResponseMessage("Wrong make of car"); } }}catch(e) { prev.setSuccessful(false); prev.setResponseMessage("Invalid JSON response.");}

The BSF Assertion plugin contains a basic script editor but can also accept a

path to a script on your local filesystem. We recommend using this technique -

the scripts are more re-usable if you do, and the built-in script editor is, quite

frankly, abysmal!

Test CoverageThe best way to ensure your API is rock solid is to create a suite of tests that

exercises every potential end point that your API consumers have access to.

Test every operation that can be carried out by your users - for example in a

RESTful interface test every step of the Create-Read-Update-Delete

sequence. JMeter is a little clunky at times and the UI has some “quirks”, but it

is possible to quickly build up a large set of tests to exercise every facet of

your API.

Remember though that creating your tests is only the first step. Re-run the

tests frequently to ensure that everything continues to function as expected. If

any of the code behind your API changes - say to improve performance - the

tests should be run to make sure the API behaves exactly as it did before. If

you add or remove features to the API you need to ensure your test suite is

updated accordingly. In an ideal world your API consumers won't know that

you've put in all this effort, but if they did they would thank you for the reliable

API that comes from it.

////////////////////////////////////////