SOA: What’s in it for Testers?

Graham Bath,

T-Systems,

Germany

Europe’s Premier Software Testing Event

Stockholmsmässan, Sweden

WWW.EUROSTARCONFERENCES.COM

“Testing For Real, Testing For Now”

SOA: What„s in it for testing?

Graham Bath

3

SOA*: What‟s in it for Testing?Contents.

What is SOA?

Why do businesses use SOA?

What does SOA mean for testing?

Key messages

*SOA = Service Oriented Architecture

4

What is SOA? The basics.

SOA is a system architecture which is based on the use of services.

To understand SOA we first need to understand services

Services, in general:

provide businesses with pre-defined “ready to go“ functionality

can be used via the Internet (Web-Services)

may also be defined for access via local networks

5

Services (technically):

are reusable software entities.

share a formal contract (interface).

are loosely coupled.

are discoverable.

are location transparent.

Understanding Services

6

Web-Services communicate via standardized protocols

Web Service Definition Language (WSDL)

Simple Object Access Protocol (SOAP)

Frameworks are commonly used for building Web-Services

Microsoft‟s .NET

Sun‟s Java 2 Enterprise Edition (J2EE)

SOA: The technology.

7

SOA: What‟s in it for Testing?Contents.

What is SOA?

Why do businesses use SOA?

What does SOA mean for testing?

Key messages

8

More flexibility in constructing applications (services = building blocks)

Applications can be constructed quickly (slot the building blocks together)

Costs are reduced by a more agile development process

Easy integration with legacy systems

Web Services: No need to develop own software

Web Services: No ownership and maintenance issues

SOA: The promise.

9

Applications which use SOA are“Ready to go“

Development of systems using SOA is“Kids Stuff“

Attitudes towards testing:

Extreme position: “Who needs it!“

Common position: “Just some basic functional testing needed“

SOA: The expectations of business owners.

10

Achieving the right balance.

Based on the promises of SOA, theexpectations of business ownersare entirely reasonable.

But are they realistic?

Testing has a job to do here…

A balance needs to be achieved between the high expectations created by SOA and the real quality risks to implemented systems.

11

SOA: What‟s in it for Testing?Contents.

What is SOA?

Why do businesses use SOA?

What does SOA mean for testing?

Key messages

12

Overview of the principal risk areas

Performance & Scalability

Reliability

Interoperability

Security

Maintainability

13

Answer:

We will never know unless we test!

User

“here it is“Questions:

Are all these components configured for optimal performance?

How many messages need to be passed to get the data I want?

Are all the servers scaled correctly for my application

db server

“give me the data“

Performance & Scalability.

14

Risks: Performance & Scalability.

Risk drivers

Web services are communication intensive

“Excess baggage“- communications protocols need a lot of it

Messages need to be encrypted / decrypted for security

Dynamically finding services via directories is time-intensive

Middleware is needed to maintain a platform-neutral concept

Synchronized handshake can mean lots of waiting around

SOA design: has it taken performance into account? (e.g. load-balancing)

What this means for testing

Design reviews need to focus on the overall architecture (if there is one)

Performance tests are a “must”

1. Response times may be longer than your user will accept.

2. The load on individual “shared” servers cannot easily be predicted in advance.

3. Easy to build systems which are unusable.

15

Risks: Reliability.

1. The reliability of the entire system is determined by the reliability of the“weakest link in the chain“

2. Finding that weakest link can be especially difficult for complex SOAs

3. Potential for unstable systems

4. Potential for data loss

What this means for testing:

The reliability of the individual services is one of the principal risks where web-services developed by third parties are used.

We need to enquire about service level requirements for reliability (e.g. Mean Time Between Failure) for the business application

Technical and management reviews of the service management concept (“Service-Governance“) are needed

We must determine whether service failures are correctly handled (service switching, monitoring, controlled reduction of functionality)

We must determine whether backup strategies are in place and that the procedures used have been tested.

Performing failover and backup/restore tests may be complex Who actually owns the data we may want backed up and restored? How can we simulate service failures? How can the tests be orchestrated?

16

Risks: Interoperability.

1. Technical interoperability may not be possible due if incompatible protocols are used by collaborating services (not all services may be communicating via WSDL)*.

2. Services may not “understand” each other at the functional (semantic) level due to unpublished add-ons and new versions introduced by Service Providers.

3. Developers may need to create adapters to permit interoperability.

4. Some of the principal benefits of SOA (re-use, easy construction, flexibility) may be lost.

5. The cost of maintaining Service-based systems increases

Automation

What this means for testing:

Technical reviews need to verify that services can cooperate as intended

Need to perform “Alive“-Tests frequently

Need to perform “Smoke“-Tests frequently

Compliance testing becomes a significant testing activity

*Note:The Web Services-Interoperability Organization (WS-I) has been set upto minimize technical interoperability risks.

17

1. The Internet: Security risk Nr.1!

2. Security threats can spread between Web Services quickly.

3. Do we know who we are connecting to?

4. Do we have confidence that the Services we are connecting to are secure?

Risks: Security.

What this means for testing:

Testing needs to provide confidence regarding the authentification of Services.

Any use of Web Services must consider the various security threats presented by using the Internet (e.g. input buffer overflow)

Independent security evaluations (static and dynamic) may be necessary.

SOA Governance must establish a security policy (which Services may be used and what proof of security must be made available).

18

Risks: Maintainability

1. Common view: “Maintenance* - that„s someone else„s responsibility”

2. Failures may be difficult to reproduce, analyze and locate

3. The “testability” of Service-based systems can be technically and (above all) organizationally complex

4. Requirement to repeat testing frequently

5. Overall risk of high “costs of ownership”

* Maintainability consists of

four attributes according

to ISO 9126:

- Analyzability

- Testability

- Changeability

- Stability

What this means for testing:

Automation of test case execution (e.g. “live” tests).

A common defect management must be set upand made available to all participants.

Establish a full-time defect manager with system-wide scope

Acknowledge that not all contributors may want, need or be able to use a common system.

19

SOA: What‟s in it for Testing?Contents.

What is SOA?

Why do businesses use SOA?

What does SOA mean for testing?

Key messages

20

Key message:Use a life-cycle approach to testing SOA systems.

Testing a SOA system is a continuous activity

which applies just as much to the operational

phase of the SOA application as

the development phase.

This requires a consistent and system-wide

test and release management.

An SOA Governance policy needs to be

set up to support these management functions.

The SOA Governance policy should be

implemented using tools, e.g.

HP, Service Test Management

“Service-Governance” plug-in for the HP Quality Center

test management tool.

21

Key Message:Testing costs have become even more significant.

Development costs may be relatively low,

depending on the amount of software re-use involved.

In relation to development, the cost of testing

for systems using SOA may be relatively high,

depending on the risks to be mitigated.

Business owners and project leaders need

to be aware of this shift in relative costs and

adjust existing management practices.

Project approvals based on costs and benefits

need to focus more on testing costs

and quality risks.

Change management needs first to ask

the question “what would this change cost to test?”

Testing effort estimations based on

a percentage of development costs are unlikely to be accurate.

22

Some tests may simply be too expensive

to perform dynamically

Well conducted technical reviews always have

been a cost-effective way to identify and mitigate

risks – this is particularly the case for SOA systems.

Benefit for all stakeholders

(development, business, users, test)

The architecture is the principal area to be checked.

Can business objectives be achieved with SOA?

Is a SO-Architecture technically feasible?

Are high-risk quality attributes addressed?

Are existing (legacy) applications suitable for definition as a Service?

Can we expect an acceptable return on investment?

Key message:Reviews are of critical importance.

23

Key message:Test management can be a significant overhead.

With SOA we are often faced with more complextest management

With multiple suppliers a single, coherent managementstructure for the entire system may simply not exist.

Who is responsible for fixing defects?

Are my points of contact defined and stable?

Many of the essential testing tasks are more complexto organize:

Who owns data? Is it consistent?

How can we orchestrate the tests?

Establishing status (obsolescence, versioning, availability)

Establishing SOA Governance

24

Concluding message: It‟s testing‟s job to make the SOA risks visible.

Are SOA systems really “ready to go” straight from the developer‟s workstation?

As we have seen, there are many different areas of risk which may significantly affect the quality of the operational system.

Businesses might be expecting too muchfrom SOA, in particular the cost of achievingand maintaining an acceptable level of quality.

It‟s the role of testing to make these risks visible to all stakeholders and propose testing measures to reduce them.

Thanks for listening!

Graham.Bath@T-Systems.com

26

Further reading

In English Rami Jaamour, „Ensuring SOA ROI“, Testing Experience, December

2008

In German W. Beinhauer, M.Herr, A. Schmidt, „SOA für agile Unternehmen“, 2008,

Symposion Publishing GmbH, ISBN 978-3-939707-14-1

27

Services are reusable software entities. Regardless of whether immediate reuse opportunities exist, services are designed to support potential reuse.

Services share a formal contract (interface) that defines the terms of information exchange; and that‟s all. The actual implementation is invisible and irrelevant to service requestors.

Services are loosely coupled. They must be designed to interact easily with other services without needing modifications or state-dependent information.

Services are discoverable. They should allow their descriptions to be remotely discovered and understood by humans and service users who may be able to make use of the services‟ logic.

Services are location transparent. Service requestors do not have to explicitly access a service on a specific server. Services can move from one location to another without affecting the requestors.

Understanding Services