fall, 2006sw eng standalone progs, univ of colorado boulder 1 wk 11 glass box testing, flow graphs,...

Fall, 2006 SW Eng Standalone Progs, Univ of Colorado Boulder

1

Wk 11 Glass Box Testing, Flow Graphs, Test Coverage

SW Engineering of Standalone Programs


2

Basis set for statement coverage testing

• Draw the flow graph• Count the bounded regions including the “outside

region”– N is the number of linearly independent paths through the

program control structure

– A linearly independent path includes one node of the control flow graph not contained in any other

– Create a basis set of linearly independent paths

• Prepare test cases to force execution of each path in the basis set.

• Execution of these test cases yields ______ coverage


3

Flow Graph:McCabe’s is25


4

Flow Graph Revelations


5

Complexity impact


6

Uses of Test Coverage Measurement

• Measure test suite completeness• Identify risk areas

– Combine execution counts with complexity– Understand complex code segments that are

frequently executed– Understand criticality

• Assist in creating new test cases • Assist in manual detection of faults• Prioritize test cases for regression tests


7

Some More Test Coverage terms


8

Test Coverage MeasuresCompleteness of Test Suite


9

Test Coverage MeasurementMinimize and select regression tests 377 Total Tests

Blocks Decisions P-uses All-uses

75% 65% 38% 44%

160 Total Tests


75% 65% 38% 44% 18 Total Tests


62% 52% 30% 37%

42% of the original tests provide identical coverage

5% of original tests provide 85% of original coverage


10

Unit Test Coverage vsSystem Test Errors


11

System Test Observed & Expected Failures


12

System Test Failure Rate over Active Hrs


13

System Test Case Completion Graph


14

HW 7 start of codegetop(s, lim) /*get next operator or operand */

char all;

int lim;

{

int i, c;

while ((c = getch ( ) ) == ‘ ‘|| c == ‘\t’ || c == ‘\n’)

;

if (c ! = ‘.’ && (c<‘0’ || c>’9’))

return (c);

s[0] = c;

for (i=1; (c=getchar()) >= ‘0’ && c <=‘9’; i++)

if (i < lim)

s[i] = c;

if ...


15

Inspections


16

What are inspections?

• Means of verifying work products

• Manual examination technique

• One piece at a time

• Small group of peers – at least 4


17

Purpose

• Verification of a work product against– established criteria– product specifications


18

How performed?

• Planning

• Overview

• Preparation

• Meeting

• Re-work

• Follow up


19

Roles of participants

• Author

• Moderator

• Reader

• Recorder

• Inspector


20

The meeting

• Each participant reports preparation time• Reader paraphrases

– Pace should be effective

– Not too slow, not too fast

• Inspectors look, listen, and think simultaneously• Author and inspectors speak up if they disagree with

reader’s paraphrase – need to decide which is correct. If Reader, Recorder notes location & brief description of issue

• Moderator can adjust pace and “atmosphere”


21

End of meeting

• Recorder’s list goes to author

• After rework, at a minimum, author reviews changes with one inspector.

• Better – inspect the changed item


22

Benefits

• Errors found at faster rate

• Errors found early are cheaper to fix

• Cross-training

• Learn good and bad techniques by seeing them, e.g. coding techniques


23

A Word about Extreme Programming

• My personal view of extreme programming is that Beck & Cunningham managed to incorporate many effective software engineering tasks into short iterations with high feedback.

• One example: pair programming is ...

fall, 2006sw eng standalone progs, univ of colorado boulder 1 wk 11 glass box testing, flow graphs,...

Documents

sw eng standalone progs

univ of colorado boulder

test coverage terms

new test cases

original coverage

system test failure

complexity impact slide

identical coverage