software component substitutability through black-box testing andrés flores [email protected]...

Software Component Substitutabilitythrough Black-Box Testing

Andrés [email protected]

Alarcos GroupEscuela Superior de Informática

Universidad de Castilla-La Mancha Ciudad Real, España

Macario [email protected]

GIISCo GroupDepartment of Computer Science

University of ComahueNeuquén, Argentina

Andres Flores - Macario Polo --- STV'07 - December, 5th 2007 2

Motivation

Upgrades of Components Updates of Component Systems

Replacement: upgrades or other OTS

Maintain stability of current systems

Evaluate replacement components

Easy the identification of compatibility


Approach

Complement regular compatibility analysis by means of Component Testing Criteria

based on the Observability testing metric

Given a component C and a candidate upgrade K, they should match in domain and range, and also on each mapping pair (input,output)


Approach – Three Phases


Sample: Java Calculator

C JCalculator K JCalc from http://sourceforge.net


Component Behaviour TS

TS goal: to check that a candidate component K

coincides on behaviour with an original component C.

each test case: a set of calls to C's services, with an

oracle to determine acceptance or refusal

Component Integration Testing Criteria:

all-Interfaces (all-Methods)

all-Events (including all-exceptions)

all–Context-dependence

intra-/inter-component dependence

1st1st


Component Behaviour TS (cont.)

Component Behaviour TS:

intra-component dependence

operational sequences RegEx

Alphabet: names of components services.

RegEx general pattern

protocol of use for a component interface

By reflection Java component interfaces are collected to

automate Test Case generation

1st1st


Sample: TS for JCalculator

We use the testoojtestooj tool some settings:

1st1st

Protocol of use:

Service sequences: length of expressions,

minimum 8, derives 20 Test templates:

4 expressions with 1 math service and

16 with an extra iteration for the '+' operator.


Sample: TS for JCalculator1st1st

ExceptionsExceptions


Sample: TS for JCalculator

test values are combined with 20 test templates (services sequences) and constraints files (pre/postcode).

testoojtestooj provides four algorithms: each choice, antirandom, pairwise, and all-combinations.

In this case we have decided on all-combinations each combination is a test case a testing method

into a test driver file. 468 test cases were generated into a class called

TestJCalculator1

1st1st


Sample: TS for JCalculator1st1st

Pre-code(setAdd)

Pre-code(evalExpression)

Post-code(evalExpression)


Interface Compatibility

To compare interfaces syntactically, three cases for services:

1)1) Exact matchExact match: two services must have identical signature, i.e. return type, service name, and for parameters and exceptions: amount, type and order.

2)2) Near-Exact matchNear-Exact match: similar to previous, butorder is relaxed for parameters and exceptions.

3)3) Soft matchSoft match: similar to previous, butservice name is ignored and for exceptionsit is relaxed to existence of any.

Outcome: a matching list with characterizedservices correspondences

2nd2nd


Sample: JCalculator-JCalc2nd2nd


Test-based Semantic Compatibility

Purpose: to find services from candidate K exposing similar behaviour with the original component C.

Implies to run the Component Behaviour TS for C (1st phase), against K

Syntactic matching list (2nd phase),is used to build a set of wrappers for component K.

Wrappers are adapters (adapter pattern) Interface mutation:

operators to change service invocationsand parameter values

To reduce amount of wrappers, starts with stronger matching cases (exact-match).

3rd3rd


Sample: run JCalculator TS on JCalc

Semantic compatibility: JCalculator–JCalc

Set of Wrappers: would be 286 in size.

we decided to initially analyzed specific cases of Interface Mutation to manually create 30 wrappers:

3rd3rd

1 wrapper with the true services correspondence (like those marked with '*' for putInBuffer service.

2 wrappers with variation on putInBuffer service (the rest of soft-matches).

3 wrappers with variation on the math services and based on the three previous wrappers.

Four groups of 6 wrappers varying the setExpression service and taking the three previous groups.


Conclusions Focused on Maintenance: Component systems are updated by

integrating upgraded components.

Testing criteria used to describe components behaviour to analyse compatibility

Similar to Regression testing: TS reduction strategies to expose faults on targeted pieces.Need to identify changes on versions (upgrades).

We assume only information collected by reflection.

Candidate components could be other OTS

Purpose: not finding faults, but analyse compatibility. TS= valid configurations.


Future Work

Important to get a reduced TS: depends on the combination

algorithm, and alternative strategies, like Mutation.

Other issue: subtyping (built-in) now was added, changing the

cases for Interface compatibility.

Wrappers: need to enhance the syntactic Interface Matching,

we are experimenting with other aspects to reduce the set size.

Wrappers analysis: being adjusted to set a proper threshold,

not always possible 100% successful on tests.

More experimentation to properly validate de whole approach.

Andrés [email protected]

Alarcos GroupEscuela Superior de Informática

Universidad de Castilla-La Mancha Ciudad Real, España

Macario [email protected]

GIISCo GroupDepartment of Computer Science

University of ComahueNeuquén, Argentina

Software Component Substitutabilitythrough Black-Box Testing

software component substitutability through black-box testing andrés flores [email protected]...

Documents

andres flores macario

component c

espaa macario polo

candidate component

original component

component testing criteria

component behaviour

exceptions slide