lecture-6 testing slide 4 print
TRANSCRIPT
-
8/16/2019 Lecture-6 Testing Slide 4 Print
1/12
Testing the Programs
Testing is part of SWE Process – irrespective any model
Testing the programs
In this part we look at
• classification of faults
• the purpose of testing
• unit testing
• integration testing strategies
• when to stop testing
Concept change!!
• Many programmers view testing as ademonstration that their program performsproperly.
• The idea of demonstrating correctness isreally the reverse of what testing is all about.
• We test a program to demonstrate theexistence of a fault! Because our goal is todiscover faults, we consider a test successful
only when a fault is discovered or a failureoccurs as a result of our testing procedures.
Classification of faults
• In an ideal world, we produce programswhere everything works flawlessly every time.Unfortunately this is not the case!
• We say that our software has failed, usuallywhen its behaviour deviates from the onedescribed in the requirements.
• First we identify the fault, i.e. determine whatfault or faults caused the failure. Next we
correct the fault by making changes to thesystem so that the fault is removed.
-
8/16/2019 Lecture-6 Testing Slide 4 Print
2/12
Classification of faults
• Why do we classify faults?
– In order to improve our development
process!
• We would like to match a fault to a
specific area of our development
process.
– In other words, we would like our
classification scheme to be orthogonal.
IBM Orthogonal Defect
ClassificationFault type Meaning
Function Fault that affects capability, end-user interfaces,
product interfaces, interface with hardware
architecture, or global data structure
Interface Fault in interacting with other components or
drivers via calls, macros,control blocks or parameter lists
Checking Fault in program logic that fails to validate data
and values properly before
they are used
Assignment Fault in data structure or code block initialization.
IBM Orthogonal Defect
ClassificationFault type Meaning
Timing/serialization Fault that involves timing of shared and real-
time resources
Build/package/merge Fault that occurs because of problems in
repositories, management changes,
or version control
Documentation Fault that affects publications and
maintenance notes
AlgorithmFault involving efficiency or correctness of
algorithm or data structure butnot design
Hewlett-Packard fault
classification
-
8/16/2019 Lecture-6 Testing Slide 4 Print
3/12
Fault Classification for HP (1970)
Computation
18%
Hardware
4%
Requirements
5%
Documentation
19%
Data Handling
6%
Other Code
11%
Logic
32%
Process/
Interprocess
5%
Testing steps
• Black (opaque) box
– In this type of testing, the test object is
viewed from the outside and its contents
are unknown.
– Testing consists of feeding input to the
object and noting what output is produced.
– The test's goal is to be sure that every kind
of input is submitted and that the observed
output matches the expected output.
Views of Test Objects
• Suppose we have a component that acceptsas input the three numbers a, b, c andoutputs the two roots of the equation ax2 + bx+ c = 0 or the message “no real roots”.
• It is impossible to test the component bysubmitting every possible triple of numbers(a,b,c).
• Representative cases may be chosen so thatwe have all combinations of positive, negativeand zero for each of a, b, and c.
• Additionally we may select values that ensurethat the discriminant, (b2 – 4ac) is positive,zero, or negative.
Black Box Example
-
8/16/2019 Lecture-6 Testing Slide 4 Print
4/12
• If the tests reveal no faults, we have no
guarantee that the component is fault-
free!
• There are other reasons why failure
may occur.
• For some components, it is impossible
to generate a set of test cases to
demonstrate correct functionality for all
cases.
Black Box Example
• White (transparent) box
– In this type of testing, we use the structure
of the test object to test in different ways.
– For example, we can devise test cases that
execute all the statements or all the control
paths within the component(s).
– Sometimes with many branches and loops
it may be impractical to use this kind of
approach.
White Box Testing
Our goal is to find faults in components.
There are several ways to do this:
• Examining the code
– Code walkthroughs
– Code inspections
• Proving code correct
• Testing components
Unit Testing
-
8/16/2019 Lecture-6 Testing Slide 4 Print
5/12
• Code walkthroughs are an informal type
of review.
• Your code and documentation is
presented to a review team and the
team comments on their correctness.
• You lead and control the discussion.
The focus is on the code not the
programmer
Examining the Code
• A code inspection is similar to a walkthroughbut is more formal.
• Here the review team checks the code anddocumentation against a prepared list ofconcerns. – For example, the team may examine the
definition and use of data types and structures tosee if their use is consistent with the design andwith system standards and procedures.
• The team may review algorithms andcomputations for their correctness andefficiency
Examining the Code
• Proof techniques are not widely used. It is difficult
to create proofs, these can sometimes be longer
than the program itself!
• Additionally customers require demonstration that
the program is working correctly.
• Whereas a proof tells us how a program will work
in a hypothetical environment described by the
design and requirements, testing gives us
information on how the program works in its
actual operating environment.
Correcting the Code Testing components
• Choosing test cases: To test a component,we select input data and conditions andobserve the output.
• A test point or case is a particular choice oftest data.
• A test is a finite collection of test cases.
• Create tests that can convince ourselves andour customers that the program workscorrectly, not only for the test cases but for allinput.
– We start by defining test objectives and definetests designed to meet a specific objective.
– One objective can be that all statements shouldexecute correctly another can be that everyfunction performed by the code is done correctly.
-
8/16/2019 Lecture-6 Testing Slide 4 Print
6/12
• As seen before we view the component as
either a “white” or “black” box.
– If we use “black” box testing, we supply all
possible input and compare the output with what
was expected.
– For example, with the quadratic equation seen
earlier we can choose values for the coefficients
that range over combinations of positive, zero and
negative numbers.
– Or select combinations based on the relative sizes
e.g. a > b > c, b > c > a, c > b > a, ...etc
Testing Components
• We can go further and select valuesbased upon the discriminant.
• We even supply non-numeric input todetermine the program's response.
• In total we have four mutually exclusivetypes of test input.
• We thus use the test objective to helpus separate the input into equivalenceclasses.
Testing Components
• Every possible input belongs to one of the
classes. That is, the classes cover the entire
set of input data.
• No input datum belongs to more than one
class. That is, the classes are disjoint.
• If the executing code demonstrates a fault
when using a particular class member is used
as input, then the same fault can be detected
using any other member of the class as input.
That is , any element of the class represents all
elements of that class.
Equivalence Classes
• It is not always easy of feasible to tell if
the third restriction can be met and it is
usually rewritten to say:
– if a class member is used to detect a fault
then the probability is high that the other
elements in the class will reveal the same
fault.
Equivalence Classes
-
8/16/2019 Lecture-6 Testing Slide 4 Print
7/12
Common Practice
• Usually 'white' box and 'black' box testing arecombined.
• Suppose we have a component expects apositive input value. Then, using 'black' boxtesting, we can have a test case for each ofthe following: – a very large positive integer
– a positive integer
– a positive, fixed point decimal
– a number greater than 0 but less than 1
– a negative number
– a non numeric character
Common Practice
• Using 'white' box testing we can chose one or
more of the following:
– Statement testing: Every statement in the
component is executed at least once in some test.
– Branch testing: For every decision point in the the
code, each branch is chosen at least once in
some test.
– Path testing: Every distinct path through the code
is executed at least once in some test.
Check how many
paths are possible?
White box testing• Statement testing
– choose X > K that produces a +ve result
– 1-2-3-4-5-6-7
• Branch testing – choose two test cases to traverse each branch of
the decision points
– 1-2-3-4-5-6-7
– 1-2-4-5-6-1
• Path testing – four test cases needed
– 1-2-3-4-5-6-7 – 1-2-3-4-5-6-1
– 1-2-4-5-6-7
– 1-2-4-5-6-1
-
8/16/2019 Lecture-6 Testing Slide 4 Print
8/12
four test cases needed
1-2-3-4-5-6-7
1-2-3-4-5-6-1
1-2-4-5-6-7
1-2-4-5-6-1
a = 5;
for (b=1; b
-
8/16/2019 Lecture-6 Testing Slide 4 Print
9/12
Integration testing
• When each component has been completed
and tested, we can then combine them into a
working system.
• This integration must be planned and
coordinated so that in the case of a failure,
we would be able to determine what may
have caused it.
• Suppose we view the system as a hierarchy
of components (shown on the following slide).
Integration testing
Bottom-up integration Top-down integration
-
8/16/2019 Lecture-6 Testing Slide 4 Print
10/12
Big-bang integration Sandwich integration
Comparison of integration
strategies
When to Stop Testing?
-
8/16/2019 Lecture-6 Testing Slide 4 Print
11/12
Fault Seeding• We intentionally insert or “seed” a know
number of faults in a program.
• Then another member of the team locate as
many faults as possible.
• The number of undiscovered seeded faults
act as an indicator of the total number of
faults(unseeded and seeded) remaining in
the program.
• We say:
Fault Seeding
Problems:
• It is assumed that the seeded faults are of thesame kind and complexity as the actual faultsin the program. – This is difficult to do since we do not know what
are the typical faults until we have found them.
• We can attempt to overcome this by basingthe seeded faults on historical data aboutprevious faults. – This, however requires that we have built similar
systems before.
Fault Seeding
Solution
• Use two independent groups, Test Group 1and Test Group 2.
• Let x be the number detected by Group 1 andy the number detected by Group 2.
• Some faults will be detected by both groupssay q, such that q
-
8/16/2019 Lecture-6 Testing Slide 4 Print
12/12
Confidence in the Software
• If we seeded a program with S faultsand we claim that the code has only Nactual faults.
• Suppose we tested until all S faultshave been found as well as n non-seeded faults, then a confidence levelcan be calculated as
1 , if n > N
C =
S/(S – N + 1) , if n N
Confidence in the Software
• With that approach we cannot predict the
level of confidence until all the seeded faults
are detected.
• Richards (1974) suggests a modification,
where the confidence level can be estimated
whether or not all the seeded faults have
located.
1 , if n > N
• C= S S + N + 1 ,if n