Software Testing

The process of operating a system or component under specified conditions, observing and recording the results, and making an evaluation of some aspect of the system or component.

IEEE Std. 610.12-1990

Testing versus Other Approaches

• Testing versus Software Quality Management• Testing versus Formal Inspections• Testing versus Proofs of Correctness• Testing versus Formal Verification• Testing versus Debugging• Testing versus Programming

Black-box Testing

• Tests the software without understanding how the internal code is implemented.

• Tests functionality based on requirements• Advantages?• Disadvantages?

White-box Testing

• Tests based on the internal logic and structure of the code

• Advantages?• Disadvantages?

What makes a good test?

• Expected test outcome is known• Has a high probability of exposing an error• Independence from coding• Tests boundary conditions• Automated

Testing can not show the absence of defects, it can only show that software defects are present

Test Levels

• Unit testing• Integration testing

– Finding errors when integrating unit-tested modules• System testing

– Testing the entire system (as a whole) running on the target hardware

• Acceptance testing– Performed by the customer– Beta testing

• Regression testing– Testing changes to make sure the old code still works with the new

changes

Types of Tests

Black-box Testing Techniques

• Equivalence partitioning– Reduces the number of test cases and selects the right

test cases to cover all possible scenarios.• Boundary-value analysis– Test cases chosen at boundaries of equivalence

partitions. Helps with off-by-one errors• Robustness testing– Test values outside the domain

• Ad hoc testing, Exploratory testing– Based on the tester’s skill, intuition, and experience

Equivalence Partitions

)2()1( XX

X < = -2 valid-2 < X < 1 invalidX >= 1 valid

Integer.MIN_VALUE, -2-1, 01, Integer.MAX_VALUE

White-box Testing Techniques

• Basis Path testing1. Convert the unit into a flow graph• A flow graph is a directed graph with a start node and a terminal

node

2. Compute the cyclomatic complexity of the unit's logical complexity

3. Use the measure to derive a basis set of execution paths4. Prepare test cases that will force execution of each path in

the basis set.

Flow Graph Example

1. GET(A); GET(B);2. if A > 15 then3. if B < 10 then4. B := A + 5;5. else6. B := A - 5;7. end if8. else9. A := B + 5;10. end if;

1

3 9

10

4 6

2

7

Cyclomatic Complexity• The cyclomatic complexity gives a quantitative measure of

logical complexity• Its value is the number of independent paths in the basis

set V(G) = E - N + 2

where E = number of edges in G and N = number of nodes in G

• V(G) provides an upper bound on the number of tests needed to ensure that all paths are executed at least once

• Studies have shown:– V(G) is directly related to the number of errors in source

code– V(G) = 10 is a practical upper limit for testing

Draw the Flow Graph

1: WHILE NOT EOF LOOP2: Read Record;2: IF field1 equals 0 THEN3: Add field1 to Total3: Increment Counter4: ELSE4: IF field2 equals 0 THEN5: Print Total, Counter5: Reset Counter6: ELSE6: Subtract field2 from Total7: END IF8: END IF8: Print "End Record"9: END LOOP9: Print Counter

Cyclomatic complexity#edges - #nodes + 211 – 9 + 2 = 4