ASTAAspect Software Testing Assistant

Juha Gustafsson,Juha Taina,Jukka Viljamaa

University of Helsinki

2/21

Motivation

Cross-Cutting Concerns in Software

Implementation modules

Security Logging

Persistence

Cross-cutting concerns clutter the main class model

3/21

Logger

log(s: String)

out: OutputStream

4/21

An Example

Money Transfer: Logical Level

void transfer(Account from, Account to, int amount) { if (from.getBalance() < amount) { throw new InsufficientFundsException(); }

from.withdraw(amount); to.deposit(amount);}

5/21

void transfer(Account from, Account to, int amount) { if (!getCurrentUser().canPerform(OP_TRANSFER)) { throw new SecurityException(); }

Transaction tx = database.newTransaction(); if (from.getBalance() < amount) { tx.rollback(); throw new InsufficientFundsException(); }

from.withdraw(amount); to.deposit(amount);

systemLog.logOperation(OP_TRANSFER, from, to, amount); tx.commit(); }

An Example

Money Transfer: Implementation Details

Security

Logging

Persistence

6/21

Solution

Aspect-Oriented Programming (AOP) Separating cross-cutting concerns from the

core concerns (i.e. from the main class model) Aspect = a definition of a cross-cutting concern

pointcut = a pattern that selects join points(i.e. elements that appear in method bodies)

advice = a method-like construct that is executedin the selected join points (before, after, around)

weaving = the process of attaching advice to the appropriate join points

7/21

An example AOP program

HelloWorld in the AspectJ language

Go to http://www.eclipse.org/ajdt/ … … and watch the demo!

8/21

A new question arises

How to test AOP? Do we test what we intend to test?

How to prioritise our tests? How to pick a good testing strategy?

When should we stop testing? In AOP these are especially difficult questions

because the execution context of an aspect depends on the

(internal implementation of the) class it is woven to relating a failure to the fault that causes it is difficult

the control and data flows are not explicit a particular aspect weave order can cause problems

9/21

A new question arises

How to test AOP? Visualising AOP testing is not straightforward

what kinds of test coverage criteria to use and visualise (is it sufficient to show that we have visited an aspect once, how to show the weaving context)?

current control flow graphs are insufficient to visualise the test execution paths in the presence of aspects

high-level coverage visualisation for AOP testing is a completely open question

Yet, visualisation of AOP testing is (at least) as important as visualisation of traditional (OO) testing

10/21

Testing AOP

Fault Sources Fault in the core concern code Fault in the (context independent) aspect code Fault created by interactions between the aspect

and the core concern code Fault created when more than one aspect is

woven into the core concern code

11/21

Testing AOP [Alexander, Bieman, Andrews, 2004]

Fault Model & Additional Testing Required Incorrect strength in pointcut patterns

test the aspect

Incorrect aspect precedence test all weave orders

Incorrect focus of control flow use a form of condition coverage of pointcut designators

Failure to establish expected postconditions use the original test set to re-test all methods that have the aspect

Failure to preserve state invariants same as in previous

Incorrect changes in control dependencies same as in previous

12/21

AOP Testing Theory

Coverage criteria Coupling metrics Control flow formalism …

13/21

Tool Support

Tool support is essential for systematic testing Not many tools exist for AOP testing

coverage tools visualisation tools scripting languages & environments to automate AOP

testing

14/21

Research Background

Due to project RITA (fRamework Intgration and Testing Application) we have a strong background in white-box test execution and analysis

RITA is intended for product family testing, but similar ideas and issues arise in AOP testing

RITA as a prototype is not suitable for large-scale testing but its ideas are relevant

15/21

RITA Functionality

Rita supports the following: Java program testing Product family or single application testing JUnit and private test execution Full support for data flow diagrams Visualisation and white-box coverages on four levels:

Method level: traditional coverages Object level: object and class coverages Package level: object coupling, reference coverages Application level: multiple application coverages

16/21

Lessons learned from RITA

A White-Box Testing Software (WBTS) must support traditional code RITA: not all applications are in product families ASPECTS: not all applications use aspects

WBTS should be integrated to a software development environment While stand-alone software can have similar functionality than

integrated software, usability issues favor the latter approach Code coverage is not sufficient in WBTS

The higher is the abstraction of testing, the more complex coverages are needed.

Even at code level code coverage gives weak results.

17/21

Aspect-Oriented Testing Tool

Based on our experiences in RITA we propose the following requirements for an Aspect-Oriented Testing Tool (AOTT): It has to support testing of traditional software It must be integrated to a well-known and actively

updated programming environment such as Eclipse It has to support several levels of white-box testing

including program-level, package-level, object-level, aspect-level, and method-level

Aspect-level: testing and visualisation are inside a single aspect.

18/21

AOTT Requirements Continue

More AOTT requirements: It has to calculate static mesurements of the software:

Lines of code (LOC) Aspects of code (AOC) Coupling between objects Depth of inheritance tree Depth of aspect tree (aspects in aspects) Coupling between code and aspects etc.

It has to support integration between dynamic test execution results and static measurements

How? This is an open research question Something else? Definitely! This is a work-in-progress list

19/21

Research directions [Alexander, Bieman, Andrews, 2004]

Open Questions Can we measure test coverage after weaving?

also visualisation How do we test aspects that interact with the core

concern code? coupling between an aspect and core concern code as a

measure of testability of the aspect How do we test aspects that interfere with each other?

inter-aspect coupling How do we test aspects whose effects must span more

than one concern? Are there ways to test aspects on their own?

probably not… ? (Can we reverse engineer the weave process?)

how does this relate to testing

20/21

Validation

How to validate that Theory is correct? Application is effecient?

Alternatives: Formalism Empirical controlled experiments Practical experiments in industrial settings

All alternatives are Relevant Needed.

21/21

Summary

To be written...