search based test data generation for server-side web...

Search Based Test Data Generation for Server-side Web Application Testing

Nadia Alshahwan and Mark Harman CREST Centre University College London

Automated web application testing using search based software Engineering (ASE 2011) •  Hill Climbing (AVM) •  Maximise branch coverage •  Server-side code (PHP)

Overall strategy

1

2

3 4

9

10

7 6 5 8

D

C. C. Michael, G. McGraw, and M. A. Schatz. Generating software test data by evolution. IEEE Transactions on Software Engineering,2001.

A

B C

D E F G

H I J

K

L

Pros •  Only local distance à

no need for approach level

•  Accidental coverage (highest %)




Cons •  Not suitable for

specific targets •  Distance calculations

affect execution time




Cons •  Not suitable for

specific targets •  Distance calculations

affect execution time à Keep track of covered branches and skip

Web Specific Issues

•  Identifying inputs ($_POST[‘inputname’]) •  Dynamic includes •  Dynamic typing à check type at run-time •  Non determinism

Dynamic Value Seeding

if($x>=$y) { . . } if($class==$result[0]) { . . }

$x = 5 $y = 300

$class = CS $result[0] = English

Dynamic Value Seeding

if (file_exists($lng.’.php’)) { ..}

Evaluation - Coverage

Evaluation - Faults

SBST vs DSE

Bugs found by SBST (SWAT) and DSE (APOLLO)

Results

•  Impact of seeding higher with string predicates •  Constant seeding might mislead the search •  Test suites with the same coverage perform

differently in fault detection

Results

•  In some applications coverage is low (20%), similar results for DSE

•  Branches that are not covered: –  Database dependent –  Environment dependent: time, OS, browser..etc –  Configuration: infeasible? –  Multi-user dependent

Future Directions

•  Easy to apply, better performance •  Different goals not just coverage •  Oracle problem: automated might not be possible

but reduce the cost

search based test data generation for server-side web...

Documents