1 / 38 cs 425/625 software engineering software testing based on chapter 20 of the textbook [somm00]...
Post on 21-Dec-2015
231 Views
Preview:
TRANSCRIPT
1 / 38
CS 425/625 Software Engineering
Software Testing
Based on Chapter 20 of the textbook [Somm00] Ian Sommerville,Software Engineering, 6th Ed., Addison-Wesley, 2000 and on theCh20 PowerPoint presentation available at the book’s web-site:www.comp.lancs.ac.uk/computing/resources/IanS/SE6/Slides/index.html
November 12, 2002
2 / 38
Outline
IntroductionIntroduction Defect TestingDefect Testing
Black Box TestingBlack Box Testing Equivalence Partitioning
Structural (White Box) TestingStructural (White Box) Testing Path Testing
Integration Testing Integration Testing Top-DownTop-Down Bottom-Up Bottom-Up Interface TestingInterface Testing Stress TestingStress Testing
Object Oriented TestingObject Oriented Testing
3 / 38
Introduction..
A very high level view of testing phases [Fig. 20.1, Somm00]
Componenttesting
Integrationtesting
Software developer Independent testing team
4 / 38
.Introduction. Component testing is concerned with checking the
functionality of methods, classes, and modules Integration testing focuses on sub-systems and their
interaction as well as on the operation of the system as a whole
Component testing is individual programmers’ task while integration testing is assigned to an integration team
For critical systems independent testing teams are used during all testing
Integration testing is based on written requirements specifications
5 / 38
..Introduction
In object-oriented testing the boundaries between component and integration testing are blurred because Object-oriented systems do not have the clear
distinction between program units and program modules characteristic to structured (function-oriented) systems
The hierarchy of objects is not necessarily “nested”
6 / 38
Defect Testing
Defect testing is aimed at discovering errors in code
A successful defect test causes the system to perform incorrectly
During the testing process the following are created: Test cases Test data Test reports
7 / 38
Defect Testing
Comprehensive testing is not possible. Subsets of tests cases are designed based on criteria such as: All code statements should be executed at least once All functions accessed through menus should be
tested All combinations of functions accessed through the
same menu should be checked All user input situations should be checked with both
correct and incorrect data
8 / 38
Defect Testing: The Process
The defect testing process [Fig. 20.2, Somm00]
Design testcases
Prepare testdata
Run programwith test data
Compare resultsto test cases
Testcases
Testdata
Testresults
Testreports
9 / 38
Defect Testing: Black Box Testing.
In black box testing (or functional testing) the system is viewed from “outside,” only in terms of its functionality The software is tested against its specification Internal (implementation) details are ignored The behavior of the system is evaluated in terms
of inputs received, outputs expected, and outputs actually produced
A defect is discovered if the actual system output differs from the expected output
Applicable to both function-oriented and object-oriented systems
10 / 38
Defect Testing: .Black Box Testing
The black-box testing model [Fig. 20.3, Somm00]
Ie
Input test data
OeOutput test results
System
Inputs causinganomalousbehaviour
Outputs which revealthe presence ofdefects
11 / 38
Defect Testing: Equivalence Partitioning….
Equivalence partitioning makes use of classes of input test data (e.g., positive numbers, negative numbers, strings, etc.) Generally, systems tend to behave in a comparable way for
all members of a class Classes of input test data are called equivalence partitions or
domains Equivalence partitions are determined based on program
specification and user documentation; Test cases are derived for each equivalence partition Partition mid-point and boundary values provide useful test
data
12 / 38
.Defect Testing: .Equivalence Partitioning…
Equivalence partitioning [Fig. 20.4, Somm00]
System
Outputs
Invalid inputs Valid inputs
13 / 38
..Defect Testing: Equivalence Partitioning..
Equivalence partitions [Fig. 20.5, Somm00]
Between 10000 and 99999Less than 10000 More than 99999
999910000 50000
10000099999
Input values
Between 4 and 10Less than 4 More than 10
34 7
1110
Number of input values
14 / 38
…Defect Testing: Equivalence Partitioning.
Example: A search routine [Fig. 20.6, Somm00]
procedure Search (Key : ELEM ; T: ELEM_ARRAY;
Found : in out BOOLEAN; L: in out ELEM_INDEX) ;
Pre-condition -- the array has at least one element
T’FIRST <= T’LAST
Post-condition -- the element is found and is referenced by L
( Found and T (L) = Key) or
-- the element is not in the array
( not Found and not (exists i, T’FIRST <= i <= T’LAST, T (i) = Key ))
15 / 38
….Defect Testing: Equivalence Partitioning
Equivalence partitions for search routine [Fig. 20.7, Somm00]
16 / 38
Defect Testing: Structural Testing….
In contrast to black box testing structural testing (or white box, clear box, glass box testing) is based on the knowledge the testers have about the structure and the implementation of the software Applied generally to smaller program units The tester analyzes the code to determine test cases Types of structural testing:
Branch coverage: every branch of the unit is tested at least once
Statement coverage: every statement of the unit is executed at least once
Path coverage: every path through the unit is executed at least once
17 / 38
.Defect Testing: Structural Testing…
The structural testing process [Fig. 20.8, Somm00]
Componentcode
Testoutputs
Test data
DerivesTests
19 / 38
Defect Testing: …Structural Testing.
Equivalence classes for binary search [Fig. 20.10, Somm00]
Mid-point
Elements < Mid Elements > Mid
Equivalence class boundaries
21 / 38
Defect Testing: Path Testing…
Path testing is a form of structural testing aimed at exercising each individual execution path of a program unit. It ensures that Each statement of the unit is executed at least once Each branch condition is tested with both true and
false conditions
The program flow graph is used for path testing. In a flow graph decisions are shown as nodes and the flow of control is represented by edges
22 / 38
Defect Testing: .Path Testing..
The cyclomatic complexity CC of a connected graph G is given by the formula:
CC(G) = Number of edges - Number of nodes + 2 If there are no go to statements and no
compound decisions (involving more than one test) then CC(G) = Number of predicate nodes + 1
(predicate nodes contain conditions that determine branching of the execution flow)
The cyclomatic complexity of a graph gives the minimum number of test cases needed to cover all paths
23 / 38
Defect Testing: ..Path TestingDefect Testing: ..Path Testing.
Flow graph for binary search [Fig. 20.12, Somm00]
1
2
3
4
65
7
while bottom <= top
if (elemArray [mid] == key
(if (elemArray [mid]< key8
9
bottom > top
24 / 38
Defect Testing: …Path Testing
Independent paths in previous case
1, 2, 3, 8, 9
1, 2, 3, 4, 6, 7, 2
1, 2, 3, 4, 5, 7, 2
1, 2, 3, 4, 6, 7, 2, 8, 9
In programs with complex branching structure the number of paths is high and it is often difficult to predict the program’s behaviour
A dynamic program analyzer can be used to identify all paths
25 / 38
Integration Testing.…..
Integration involves building sub-systems from program units and the system from subsystems
Integration testing uses software specifications as basis for checking the system
Errors are more difficult to locate than in unit testing The incremental approach is recommended for system
integration and testing Initially, a minimal system configuration can be used Tests need be repeated after each addition to the
system
26 / 38
.Integration Testing…...Integration Testing…..
Incremental integration testing [Fig. 20.13, Somm00]
T3
T2
T1
T4
T5
A
B
C
D
T2
T1
T3
T4
A
B
C
T1
T2
T3
A
B
Test sequence1
Test sequence2
Test sequence3
27 / 38
..Integration Testing….
Strategies for system integration and testing: Top-Down
Starts with high level components and continues at lower levels
Integral part of the top-down development process Stubs are needed to test the system
28 / 38
…Integration Testing…
Strategies for system integration and testing [cont’d]: Bottom-Up
Starts with lower level components and builds up the system by adding & testing higher level components
Can be used when components from previously developed systems are reused or when the system’s functionality relies on critical lower level modules
Driver modules are used to test the system
“Sandwich” Combines the above two strategies by using the Top-Down
approach for logic modules (higher level) and the Bottom-Up approach for operational (lower level) modules
29 / 38
….Integration Testing..
Top-down integration testing [Fig. 20.14, Somm00]
Level 2Level 2Level 2Level 2
Level 1 Level 1Testing
sequence
Level 2stubs
Level 3stubs
. . .
30 / 38
…..Integration Testing.
Bottom-up integration testing [Fig. 20.14, Somm00]
Level NLevel NLevel NLevel NLevel N
Level N–1 Level N–1Level N–1
Testingsequence
Testdrivers
Testdrivers
31 / 38
…...Integration Testing
Top-Down versus Bottom-Up: Architectural validation
Top-Down allows earlier discovery of high-level design errors
System demonstration Top-Down supports better an early demo; Bottom-Up can
support such demo if many reusable components are utilized
Test implementation Strict Top-Down testing is more difficult to implement given
that stubs are needed. It is easier to use drivers in a strict Bottom-Up testing
Test observation Equally challenging, since both stubs and drivers may not
allow complete observations
32 / 38
Interface Testing… Interface testing has the goal of detecting errors related to
interfacing modules and sub-systems Particularly important for object-oriented testing given objects’
collaboration relies on their interfaces Types of interfaces:
Parameter interfaces Shared memory interfaces Procedural interfaces Message passing interfaces
Classes of interface errors: Interface misuse Interface misunderstanding Timing errors
33 / 38
.Interface Testing..
Interface testing [Fig. 20.15, Somm00]: tests directed at the sub-system as a whole, not at particular components
Testcases
BA
C
34 / 38
..Interface Testing.
Guidelines for interface testingGuidelines for interface testing Test all external calls with parameters at extreme Test all external calls with parameters at extreme
rangesranges When pointers are used, test interfaces with null When pointers are used, test interfaces with null
pointerspointers For procedural interfaces try to design tests that would For procedural interfaces try to design tests that would
cause the components to failcause the components to fail Use stress testing for message passing interfacesUse stress testing for message passing interfaces Change activation order of components that share Change activation order of components that share
memorymemory
35 / 38
…Interface Testing
Stress testing exercise the systems with Stress testing exercise the systems with overloadingoverloading (e.g., number of transactions per minute, (e.g., number of transactions per minute, number of users in a distributed applications)number of users in a distributed applications)
The main purposes of stress testing:The main purposes of stress testing: Make sure the system does not lose or corrupt data Make sure the system does not lose or corrupt data
when overloadedwhen overloaded Reveal defects that otherwise would pass unnoticedReveal defects that otherwise would pass unnoticed
36 / 38
Object-Oriented Testing..
Differences from function-oriented systems: Objects are usually larger units, which
encompass several methods Objects are typically loosely coupled and there
may not be an obvious high top of the class hierarchy
Testers may not have access to the code of reused objects
37 / 38
.Object-Oriented Testing.
Levels of testing in object-oriented testing: Testing operations of object classes (sequences
of operations may be needed) Testing object classes (operations, attributes,
and states) Testing clusters of objects (use case or scenarios
based testing; thread testing, etc.); each operation in each class should be checked at least once
Testing the object-oriented system
38 / 38
..Object-Oriented Testing
Compared with testing function-oriented systems, testing of object-oriented system is facilitated by inheritance + reuse (previously tested objects) but specific challenges exist because: Information hiding is a complicating factor Writing additional methods (invoked for testing
purposes only) are often necessary New and overwritten methods must be tested Inherited methods must still be re-tested if they
interact with newly written methods
top related