Chapter 8

Testing the Programs

Integration Testing

Combine individual comp., into a working s/m.

Test strategy gives why & how comp., are

combined to test the working s/m.

Strategy affects not only the integration timing,

coding, cost & thoroughness of the testing.

S/m viewed as hierarchy of comps.Approaches

Bottom-upTop-downBig-bangSandwich testingModified top-downModified sandwich

Bottom-up Integration

Merging comp.,

Lowest level of s/m hierarchy is tested individually first then next comp., are those that call the previously tested comp.,

Component Driver: a routine that calls a particular component and passes a test case to it.

Drawback – functionally decomposed s/m – is that top level comp., are important but last to be tested.

Top level are more general , whereas bottom level are more specific.

Bottom up testing is most sensible for OOpgms.,

System viewed as a hierarchy of components The sequence of tests and their dependencies We need comp., driver for E,F,G

Top-Down Integration

Reverse of bottom-up

Top level – one controlling comp., is tested by itself. Then all comps., called by the tested comp., are

combined & tested

Stub: a special-purpose program to simulate the activity of the missing component

The stub answers the calling seq., & passes back o/p data that lets the testing process continue.

Only A is tested by itself Stubs needed for B,C & D.

Top down allows the test team to exercise one fn., at a time

Test cases defined in terms of fns., being examined. Design faults abt fnality., addressed at the beginning of

testing. Drivers pgms., are not needed. Writing stubs can be difficult, its correctness may affect

the validity of the test.

Drawback with top-down testing is possibility that a very large no., of stubs may be required.

One way to avoid is to alter the strategy slightly Modified Top-Down Integration

Each level’s components individually tested before the merger takes place.

Bing-Bang Integration

Used for small s/m, not practical for large s/m. First, it requires both stubs and drivers to test the independent

components Second, b’coz all comp., are merged at once, it is difficult to find

the cause of failures. Finally, i/f faults cannot be distinguished easily

Sandwich Integration Combines top down with bottom-up Viewed system as three layers: target layer in middle, levels

above the target , levels below the target. Top down in top level & bottom up in lower layer. Testing converges towards target layer – chosen on the basis of s/m

char., & struc., of comp., hierarchy.

Ex., comp., hierarchy- Target layer in the

middle level , comp.,

B, C, D Drawback – it does not

test the indiv., comp

before integration.

Modified Sandwich Integration

Allows upper-level components to be tested before merging them with others

Comparison of Integration Strategies

Bottom-up Top-down

Modified top-down

Bing-bang Sandwich Modified sandwich

Integration Early Early Early Late Early Early

Time to basic working program

Late Early Early Late Early Early

Component drivers needed

Yes No Yes Yes Yes Yes

Stubs needed No Yes Yes Yes Yes Yes

Work parallelism at beginning

Medium Low Medium High Medium High

Ability to test particular paths

Easy Hard Easy Easy Medium Easy

Ability to plan and control sequence

Easy Hard Hard Easy Hard hard

Testing Object-Oriented Systems

Rumbaugh ask several ques.,: Is there a path that generates a unique

result? Is there a way to select a unique result? Are there useful cases that are not handled?

Next check obj., & classes themselves for excesses & deficiencies

Missing obj, useless classes, associations or attributes

Differences Between OO and Traditional Testing

OO comp., - reused, helps minimize testing First, test – base classes having no parents – test each

fn., individually then the interactions among them. Next – provide an alg., to update incrementally the

test history for parent class. OO unit testing is less difficult, but integration testing is

more extensive

The farther the gray line is out, the more the difference

Requires special

treatment

Test Planning Test planning helps us to design & organize

the tests Test steps:

Establish test objectives Design test cases Write test cases Testing test cases Execute tests Evaluate test results

Test obj., - tells us what kind of test cases to generate.

Purpose of the Plan Test plan explains

who does the testing? why the tests are performed? how tests are conducted? when the tests are scheduled?

Contents of the Plan What the test objectives are? How the test will be run? What criteria will be used to determine when the

testing is complete? Statement, branch, & path coverage at the comp.,

level Top-down , bottom-up at the integration level Resulting test plan for merging the comps., into a

whole sometimes called S/m Integration Testing

Automated Testing Tools

Code Analysis Tool Static analysis – src pgm before it is run

Code analyzer Structure checker Data analyzer Sequence checker

Dynamic analysis – pgm is running Program monitors: watch and report program’s behavior

Test execution – automate the planning & running of the test themselves Capture and replay Stubs and drivers Automated testing environments

Test case generators – automate test case generation

When to Stop Testing More faulty?

Probability of finding faults during the development

Stopping Approaches

Fault seeding or error seeding – to estimate no., of faults in a pgm One team – seeds or inserts a known no., of faults Other team – locate as many faults as possible No. of undiscovered faults acts as indicator

The ratio:detected seeded Faults detected nonseeded faults total seeded faults total nonseeded faults

Expressing ratio formally : N = Sn /s N – no., of nonseeded faults S – no., seeded faults placed in a pgm. n – actual no., of nonseeded faults detected during testing s – no., of seeded faults detected during testing

Confidence in the software, - use fault estimate to tell how much confidence we can place in the s/w we are testing.

Confidence expressed in percentage.

= 1, if n >N C = S/(S – N + 1), if n ≤ N

Coverage criteria

Identifying Fault-Prone Code

Track the number of faults found in each component

during the development

Collect measurement (e.g., size, number of decisions)

about each component

Classification trees : a statistical technique that sorts

through large arrays of measurement information and

creates a decision tree to show best predictors

A tree helps in deciding the which components are

likely to have a large number of errors

An Example of a Classification Tree