ps3: acceptance test

AN EXAMPLE

PS3: Acceptance Test

Based on Testing Extreme Programming

By Lisa Crispin, Tip House

Goal

Check the functionality of the entire system as specified by the project's customer. Acceptance tests should not include specific

knowledge about the internals of the system and are said to be "black box" tests.

These tests touch the system only at predefined APIs or GUIs.

To be effective, system-level tests often require large amounts of domain-specific data.

The customer's involvement in creating this data and specifying system behavior is critical.

2

Software Engineering Spring 2011

Example Application: Seller Reputation

3


Rate the transaction

Rate a transaction

Select a seller to rate from a list of sellers

Accurate description

Select a transaction to rate

Communication

Quick shipping

Reasonable shipping charges

4


Major phases

High level Acceptance Tests Write stories Define some acceptance tests for each story

Find Hidden Assumptions in the stories Customer’s view; User’s view; Programmer’s view Add High level Acceptance Tests based on these

assumptionsObtain Data for testing

Bad, sad, happy scenariosWrite executable tests

Write code for each acceptance test defined Bridge the code to the application

5


High Level Acceptance Test

Story I:Rate Transaction Story The user can provide a rating for a transaction he took

part in, expressing the degree to which he is satisfied with it in the following properties: Accurate description, Communication, Quick shipping, Reasonable shipping charges, and textual comment.

Define high-level acceptance tests for the Story. Action DataExpected Results

Rate the seller by Accurate description, Communication, Quick shipping, Reasonable shipping charges, and add a textual comment.

An unrated transaction that exists in the database

Success: the rating for the transaction is updated

6


Obtaining hidden Assumptions

Story

7


Step1 – Customer view

I'm the customer. We need a way to track performance of sellers in our system in

order to compute the reputation of this seller. For this purpose we need the users that interacted with this

seller to rate the seller by 4 different aspects. We don’t want users that did not have experience with the seller

to rate it – so only user who interacted with the seller may provide ratings for him.

We don’t want the user to provide multiple good ratings or bad ratings – so we allow only one rating per transaction.

What business problem is it solving? How does it solve it? Users would like to know a seller reputation before initiating a

transaction with him. By collecting users rating we use some Trust/Reputation

algorithm to infer the seller’s reputation.

8


Step 2: User view

I'm a user, I want to provide a rating for a transaction I made What are the worst and best things that can happen? What would really irritate me?

Worst thing: I can't locate the transaction I want to rate. It's in the system, but I can't

find it. Two rating properties appear to have a similar meaning —I can't tell the

difference. Best thing:

I can see all the transactions I have performed in the system sorted by date, seller rating –state (rated/unrated).

I can see the rating I gave for the rated ones . I can also select an unrated transaction form my list and rate it.

How can I screw up? How should the system respond? Can I accidentally rate the same transaction more than once? Can I rate a seller that I didn’t interact with? Can I change my opinion – change the rating I already gave a seller upon

completion of a transaction?

9


Step 3: Programmer view

I'm the programmer. What's the simplest implementation that could

possibly work? A list of all transactions each indicated as rated or not. Click on an unrated transaction to select it and to rate it. Click on a rated transaction impossible.

OR?

We need some way to identify a transaction Do we? by a number?

10


Step 4: Tester’s view

How likely is the implementation to solve the business problem? Fairly likely.( under honest and cooperating users

assumptions)How likely is the implementation to solve the

related problems? Fairly likely.

How likely is the implementation to avoid irritating the user? We need a way to indicate the user that

no more than one rating is allowed per transaction There is no option to modify a rating already provided The value they can provide is within a predefined range

11


Hidden assumptions

Identify problematic hidden assumptions in the story The system provides a list of the user’s transactions The user can view rated transactions. The user cannot edit rated transaction. All four properties are required fields. The rating values range within a known scale. Textual comments are optional ?

12


Additional tests (based on the assumptions)

What other acceptance test can be added according to the assumptions?

Action DataExpected Results

1.1.Rate the seller by the four properties.

An unrated transaction that exists in the database

Success: the rating for the transaction is updated

1.2. Rate the seller by the four properties

An unrated transaction, exceed maximum values for the four properties .

Failure: should not be able to enter numbrs out of range

…

2.1. find the List of unrated transactions the user had with a seller.

A user , a seller, existing transactions of the two with “closed” field =No

Success: The list of transactions the user had with the seller is not empty

2.2. find the List of all transactions the user had with a seller.

A user , a seller, no transactions of the two with “closed” field =No

Fail: The list of transactions the user had with the seller is empty

…

13


Writing Executable Acceptance tests

The rateTransStory. High-level acceptance tests for this story:

Rating an unrated transactions using correct values: succeeds – the transaction properties are updated, transaction is closed

Rating an unrated transactions using out-of –range values: fails– the transaction properties are not updated, transaction remains opened.

Data: Users : Bob, Clair Sellers: AutoShop, Buy4Cheap Transactions: Bob, AutoShop - unrated transaction 1223,

Clair, Buy4Cheap– rated transaction 1226 This captures the essence of the test but leaves out the details.

The details can go into an executable test, as follows: assertTrue(rateTrans(1223,1,2,3,4,”bad experience”) ) assertTrue(rateTrans(1223,1,2,3,4,””) ) assertFalse(rateTrans(1223,7,2,3,4,”Called right on time”) ) assertFalse(rateTrans(1226,3,2,3,4,”unreliable”) ) assertFalse(rateTrans (1227,1,2,3,4,”blabla”) - Can blabla cause the test

failure?

14


Sampled Test Data Table 16.3

Defined with the customerHappy, sad, bad…

Characteristics

OpenedTrans ID

Accurate Communications

ShippingCostsCommentExxpected Results

ValidY12231223bad experience

Transaction successfully updated

ValidY12231254Transaction successfully updated

InvalidY12230121THE WORST!!

Rating score is out of range

InvalidY12239241Nice but not proffesional

Rating score is out of range

InvalidN12261253Very politeTransaction is already rated

Invalid12271223Bla blaInvalid Transaction ID

15


Testing Method per Action

For each action of the high level acceptance test– write a testing method:

public class rateTransStoryTest {public void testRateTrans() {// Test data and assumptions for action Rate go here}public void testListTrans() {// Test data and assumptions for action List go here}

}

16


Testing Method per Action

public class rateTransStoryTest extends SellerReputationTest {

public rateTransStoryTest (String name) {super(name); } \\ A pass-thru

constructor

public void testRateTrans() {assertTrue(rateTrans(“1223”,1,2,3,4,”blabla”) ) ;assertTrue(rateTrans(“1223”,1,2,3,4,””) ) ;assertFalse(rateTrans(“1223”,7,2,3,4,”blabla”) ) ;assertFalse(rateTrans(“1226”,3,2,3,4,”blabla”) ) ;assertFalse(rateTrans (“1227”,1,2,3,4,”blabla”);

}public void testListTrans() {…}}

rateTrans(<T_ID ,><score1>, <score2>, <score3>,

<score4><comment>) should be bridged to the real

application

17


import junit.framework.*; //Define an application-test class that extends JUnit' s TestCase

import SellerReputation.*;

public class SellerReputationTest extends TestCase {// bridge classpublic SellerReputationTest(String name) {super(name); }

public void assertFalse( boolean condition) {assertTrue(!condition);

}// Make every story acceptance test a sub-class of the application test class.

Public boolean rateTrans(String tId, int att1, int att2, int att3, int att4, string comment){SellerReputationTrans trans= new SellerReputationTrans();return trans.rate(tId, att1,att2,att3,att4, comment);

}public boolean listTrans( String uId, String sId) {

…….}

}

A Bridge to the rate operation as implemented in the application

18


Bridge by inheritance


The SellerReputationTrans class in the SellerReputation system represents a transaction.

The SellerReputationTrans has a rate method that sets the transaction rating criteria: Accurate description, Communication, Quick shipping, Reasonable shipping charges and comment. If the specified values are all valid, it then attempts to update the transaction,

if the update is successful it returns true otherwise - if the update fails or if any of the specified values is invalid, it

returns false

The SellerReputationUser class in the SellerReputation system represents a user.

The SellerReputationUser has a transList method that searches for a list of transactions the user had with a specified seller If the user had interactions with the specified seller, it produces the list of

transactions, if the list is nonempty it returns true otherwise - if the list is empty or if seller id is invalid, it returns false

19



TestCase (junit )•assertTrue•assertEqual•…

SellerReputationTest•assertFalse•rateTrans •listTrans•…

rateTransStoryTest•testRateTrans•testListTrans•…

Functionality provided by testing

framework, e.g. junit

Project Test Defines the bridge to

the application

Actual tests

20


Bridge by Composition


TestCase (junit )•setUp()•assertTrue•assertEqual•…

SellerReputationTest•setUp()•assertFalse•rateTrans { ..bridge..} •listTrans { ..bridge..}•…

rateTransStoryTest•testRateTrans {..rateTrans..}•testListTrans{..listTrans..}•…

<<Interface >>

SellerReputationBridge

•rateT()•listT()•…….

Real Bridge

•rateT()•listT()•……..

Proxy Bridge


bridge

real

implements implements

21


How to avoid recompile the tests when replacing bridges ?•SellerReputationTest class receives the bridge as parameter (setter and/or in constructor).•SellerReputationTest class reads configuration file to select the right bridge (in setUp()).•SellerReputationTest class uses external class to receive the bridge (next slide).•Any other methods you can come up with.


22



TestCase (junit )•setUp()•assertTrue•assertEqual•…

SellerReputationTest•setUp()•assertFalse•rateTrans { ..bridge..} •listTrans { ..bridge..}•…

rateTransStoryTest•testRateTrans {..rateTrans..}•testListTrans{..listTrans..}•…

<<Interface >>SellerReputationBridge

•rateT()•listT()•…….

Real Bridge


Proxy Bridge


bridge

real

implements implements

23

Driver

getBridge()

<<use>><<use>>

protected void setUp() throws Exception {

super.setUp();…

this.bridge=Driver.getBridge();…

}

protected void setUp() throws Exception {

super.setUp();…

this.bridge=Driver.getBridge();…

}

<<use>>

public static SellerReputationBridge getBridge() {ProxyBridge bridge=new ProxyBridge (); // add when real bridge is readybridge.real = new RealBridge (); return bridge;

}

public static SellerReputationBridge getBridge() {ProxyBridge bridge=new ProxyBridge (); // add when real bridge is readybridge.real = new RealBridge (); return bridge;

}

ps3: acceptance test

Documents