chapter 13 software testing...
TRANSCRIPT
Software Engineering: A Practitioner’s Approach, 6/eSoftware Engineering: A Practitioner’s Approach, 6/e
Chapter 13Chapter 13Software Testing StrategiesSoftware Testing Strategies
These courseware materials are to be used in conjunction with Software Engineering: A Practitioner’s Approach, 6/e and are provided with permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 1
copyright © 1996, 2001, 2005
R.S. Pressman & Associates, Inc.
For University Use OnlyMay be reproduced ONLY for student use at the university level
when used in conjunction with Software Engineering: A Practitioner's Approach.Any other reproduction or use is expressly prohibited.
Software TestingSoftware Testing
Testing is the process of exercising aTesting is the process of exercising aprogram with the specific intent of findingprogram with the specific intent of findingerrors prior to delivery to the end user.errors prior to delivery to the end user.
These courseware materials are to be used in conjunction with Software Engineering: A Practitioner’s Approach, 6/e and are provided with permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 2
errors prior to delivery to the end user.errors prior to delivery to the end user.
What Testing ShowsWhat Testing Shows
errorserrors
requirements conformancerequirements conformance
performanceperformance
These courseware materials are to be used in conjunction with Software Engineering: A Practitioner’s Approach, 6/e and are provided with permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 3
an indicationan indicationof qualityof quality
Who Tests the Software?Who Tests the Software?
These courseware materials are to be used in conjunction with Software Engineering: A Practitioner’s Approach, 6/e and are provided with permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 4
developerdeveloper independent testerindependent tester
Understands the system Understands the system
but, will test "gently"but, will test "gently"
and, is driven by "delivery"and, is driven by "delivery"
Must learn about the system,Must learn about the system,
but, will attempt to break itbut, will attempt to break it
and, is driven by qualityand, is driven by quality
Testing StrategyTesting Strategy
unit testunit test integrationintegrationtesttest
These courseware materials are to be used in conjunction with Software Engineering: A Practitioner’s Approach, 6/e and are provided with permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 5
validationvalidationtesttest
systemsystemtesttest
Testing StrategyTesting Strategy
�� We begin by ‘We begin by ‘testingtesting--inin--thethe--small’small’ and move toward and move toward ‘‘testingtesting--inin--thethe--large’large’
�� For conventional software:For conventional software:
�� The module (component) is our initial focusThe module (component) is our initial focus
These courseware materials are to be used in conjunction with Software Engineering: A Practitioner’s Approach, 6/e and are provided with permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 6
�� The module (component) is our initial focusThe module (component) is our initial focus
�� Integration of modules followsIntegration of modules follows
�� For OO software:For OO software:
�� our focus when “testing in the small” changes from an our focus when “testing in the small” changes from an individual module (the conventional view) to an OO class individual module (the conventional view) to an OO class that encompasses attributes and operations and implies that encompasses attributes and operations and implies communication and collaborationcommunication and collaboration
Strategic IssuesStrategic Issues
�� State testing objectives explicitly. State testing objectives explicitly.
�� Understand the users of the software and develop a profile for Understand the users of the software and develop a profile for each user category.each user category.
�� Develop a testing plan that emphasizes “rapid cycle testing.”Develop a testing plan that emphasizes “rapid cycle testing.”
�� Build “robust” software that is designed to test itself.Build “robust” software that is designed to test itself.
These courseware materials are to be used in conjunction with Software Engineering: A Practitioner’s Approach, 6/e and are provided with permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 7
�� Build “robust” software that is designed to test itself.Build “robust” software that is designed to test itself.
�� Use effective formal technical reviews as a filter prior to testing.Use effective formal technical reviews as a filter prior to testing.
�� Conduct formal technical reviews to assess the test strategy Conduct formal technical reviews to assess the test strategy and test cases themselves. and test cases themselves.
�� Develop a continuous improvement approach for the testing Develop a continuous improvement approach for the testing process. process.
Unit TestingUnit Testing
modulemoduleto beto be
testedtested
These courseware materials are to be used in conjunction with Software Engineering: A Practitioner’s Approach, 6/e and are provided with permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 8
test casestest cases
resultsresults
softwaresoftwareengineerengineer
Unit TestingUnit Testing
interface interface
local data structureslocal data structures
boundary conditionsboundary conditions
modulemoduleto beto be
testedtested
These courseware materials are to be used in conjunction with Software Engineering: A Practitioner’s Approach, 6/e and are provided with permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 9
boundary conditionsboundary conditions
independent pathsindependent paths
error handling pathserror handling paths
test casestest cases
Unit Test EnvironmentUnit Test Environment
ModuleModule
driverdriver
interface interface
local data structureslocal data structures
boundary conditionsboundary conditions
independent pathsindependent paths
These courseware materials are to be used in conjunction with Software Engineering: A Practitioner’s Approach, 6/e and are provided with permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 10
stubstub stubstub
RESULTSRESULTS
independent pathsindependent paths
error handling pathserror handling paths
test casestest cases
Integration Testing StrategiesIntegration Testing Strategies
Options:Options:
•• the “big bang” approachthe “big bang” approach•• an incremental construction strategyan incremental construction strategy
These courseware materials are to be used in conjunction with Software Engineering: A Practitioner’s Approach, 6/e and are provided with permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 11
Top Down IntegrationTop Down Integration
top module is tested with top module is tested with stubsstubs
stubs are replaced one at stubs are replaced one at
AA
BB FF GG
These courseware materials are to be used in conjunction with Software Engineering: A Practitioner’s Approach, 6/e and are provided with permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 12
stubs are replaced one at stubs are replaced one at a time, "depth first"a time, "depth first"
as new modules are integrated, as new modules are integrated, some subset of tests is resome subset of tests is re--runrun
CC
DD EE
BottomBottom--Up IntegrationUp Integration
drivers are replaced one at a drivers are replaced one at a
AA
BB FF GG
These courseware materials are to be used in conjunction with Software Engineering: A Practitioner’s Approach, 6/e and are provided with permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 13
drivers are replaced one at a drivers are replaced one at a time, "depth first"time, "depth first"
worker modules are grouped into worker modules are grouped into builds and integratedbuilds and integrated
CC
DD EE
clustercluster
Sandwich TestingSandwich Testing
Top modules areTop modules aretested with stubstested with stubs
AA
BB FF GG
These courseware materials are to be used in conjunction with Software Engineering: A Practitioner’s Approach, 6/e and are provided with permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 14
Worker modules are grouped into Worker modules are grouped into builds and integratedbuilds and integrated
CC
DD EE
clustercluster
ObjectObject--Oriented TestingOriented Testing
�� Begins by evaluating the correctness and consistency of Begins by evaluating the correctness and consistency of the OOA and OOD modelsthe OOA and OOD models
�� Testing strategy changesTesting strategy changes
�� the concept of the ‘unit’ broadens due to encapsulationthe concept of the ‘unit’ broadens due to encapsulation
These courseware materials are to be used in conjunction with Software Engineering: A Practitioner’s Approach, 6/e and are provided with permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 15
�� the concept of the ‘unit’ broadens due to encapsulationthe concept of the ‘unit’ broadens due to encapsulation
�� integration focuses on classes and their execution across a integration focuses on classes and their execution across a ‘thread’ or in the context of a usage scenario‘thread’ or in the context of a usage scenario
�� validation uses conventional black box methodsvalidation uses conventional black box methods
�� Test case design draws on conventional methods, but Test case design draws on conventional methods, but also encompasses special featuresalso encompasses special features
Broadening the View of “Testing”Broadening the View of “Testing”
It can be argued that the review of OO analysis and It can be argued that the review of OO analysis and
design models is especially useful because the design models is especially useful because the
same semantic constructs (e.g., classes, attributes, same semantic constructs (e.g., classes, attributes,
operations, messages) appear at the analysis, operations, messages) appear at the analysis,
design, and code level. Therefore, a problem in the design, and code level. Therefore, a problem in the
These courseware materials are to be used in conjunction with Software Engineering: A Practitioner’s Approach, 6/e and are provided with permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 16
design, and code level. Therefore, a problem in the design, and code level. Therefore, a problem in the
definition of class attributes that is uncovered definition of class attributes that is uncovered
during analysis will circumvent side effects that during analysis will circumvent side effects that
might occur if the problem were not discovered might occur if the problem were not discovered
until design or code (or even the next iteration of until design or code (or even the next iteration of
analysis). analysis).
Testing the CRC ModelTesting the CRC Model
1. Revisit the CRC model and the object1. Revisit the CRC model and the object--relationship model.relationship model.
2. Inspect the description of each CRC index card to determine if 2. Inspect the description of each CRC index card to determine if a delegated responsibility is part of the collaborator’s definition.a delegated responsibility is part of the collaborator’s definition.
3. Invert the connection to ensure that each collaborator that is 3. Invert the connection to ensure that each collaborator that is asked for service is receiving requests from a reasonable source.asked for service is receiving requests from a reasonable source.
4. Using the inverted connections examined in step 3, determine 4. Using the inverted connections examined in step 3, determine
These courseware materials are to be used in conjunction with Software Engineering: A Practitioner’s Approach, 6/e and are provided with permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 17
4. Using the inverted connections examined in step 3, determine 4. Using the inverted connections examined in step 3, determine whether other classes might be required or whether whether other classes might be required or whether responsibilities are properly grouped among the classes.responsibilities are properly grouped among the classes.
5. Determine whether widely requested responsibilities might be 5. Determine whether widely requested responsibilities might be combined into a single responsibility.combined into a single responsibility.
6. Steps 1 to 5 are applied iteratively to each class and through 6. Steps 1 to 5 are applied iteratively to each class and through each evolution of the OOA model.each evolution of the OOA model.
OOT StrategyOOT Strategy
�� Class testing is the equivalent of unit testing:Class testing is the equivalent of unit testing:
�� operations within the class are testedoperations within the class are tested
�� the state behavior of the class is examinedthe state behavior of the class is examined
�� Integration applies three different strategies:Integration applies three different strategies:
These courseware materials are to be used in conjunction with Software Engineering: A Practitioner’s Approach, 6/e and are provided with permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 18
�� threadthread--based testingbased testing——integrates the set of classes integrates the set of classes required to respond to one input or eventrequired to respond to one input or event
�� useuse--based testingbased testing——integrates the set of classes required integrates the set of classes required to respond to one use caseto respond to one use case
�� cluster testingcluster testing——integrates the set of classes required to integrates the set of classes required to demonstrate one collaborationdemonstrate one collaboration
Smoke TestingSmoke Testing
�� A common approach for creating “daily builds” for product A common approach for creating “daily builds” for product softwaresoftware
�� Smoke testing steps:Smoke testing steps:�� Software components that have been translated into code are Software components that have been translated into code are
integrated into a “build.” integrated into a “build.” �� A build includes all data files, libraries, reusable modules, and A build includes all data files, libraries, reusable modules, and
engineered components that are required to implement one or engineered components that are required to implement one or more product functions.more product functions.
These courseware materials are to be used in conjunction with Software Engineering: A Practitioner’s Approach, 6/e and are provided with permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 19
more product functions.more product functions.
�� A series of tests is designed to expose errors that will keep A series of tests is designed to expose errors that will keep the build from properly performing its function. the build from properly performing its function.
�� The intent should be to uncover “show stopper” errors that have The intent should be to uncover “show stopper” errors that have the highest likelihood of throwing the software project behind the highest likelihood of throwing the software project behind schedule.schedule.
�� The build is integrated with other builds and the entire product The build is integrated with other builds and the entire product (in its current form) is smoke tested daily. (in its current form) is smoke tested daily.
�� The integration approach may be top down or bottom up.The integration approach may be top down or bottom up.
High Order TestingHigh Order Testing
�� Validation testingValidation testing�� Focus is on software requirementsFocus is on software requirements
�� System testingSystem testing�� Focus is on system integrationFocus is on system integration
�� Alpha/Beta testingAlpha/Beta testing�� Focus is on customer usageFocus is on customer usage
�� Recovery testingRecovery testing�� forces the software to fail in a variety of ways and verifies that forces the software to fail in a variety of ways and verifies that
These courseware materials are to be used in conjunction with Software Engineering: A Practitioner’s Approach, 6/e and are provided with permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 20
�� forces the software to fail in a variety of ways and verifies that forces the software to fail in a variety of ways and verifies that recovery is properly performedrecovery is properly performed
�� Security testingSecurity testing�� verifies that protection mechanisms built into a system will, in fact, verifies that protection mechanisms built into a system will, in fact,
protect it from improper penetrationprotect it from improper penetration
�� Stress testingStress testing�� executes a system in a manner that demands resources in abnormal executes a system in a manner that demands resources in abnormal
quantity, frequency, or volumequantity, frequency, or volume
�� Performance TestingPerformance Testing�� test the runtest the run--time performance of software within the context of an time performance of software within the context of an
integrated systemintegrated system
Debugging: Debugging: A Diagnostic ProcessA Diagnostic Process
These courseware materials are to be used in conjunction with Software Engineering: A Practitioner’s Approach, 6/e and are provided with permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 21
The Debugging ProcessThe Debugging Processtest casestest cases
These courseware materials are to be used in conjunction with Software Engineering: A Practitioner’s Approach, 6/e and are provided with permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 22
resultsresults
DebuggingDebugging
suspectedsuspectedcausescauses
identifiedidentifiedcausescauses
correctionscorrections
regressionregressionteststests
new testnew testcasescases
Debugging EffortDebugging Effort
time requiredtime requiredto diagnose theto diagnose thesymptom andsymptom anddetermine thedetermine thecausecause
time requiredtime requiredto correct the errorto correct the errorand conductand conduct
These courseware materials are to be used in conjunction with Software Engineering: A Practitioner’s Approach, 6/e and are provided with permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 23
causecauseand conductand conductregression testsregression tests
Symptoms & CausesSymptoms & Causes
Symptom and cause may be Symptom and cause may be geographically separated geographically separated
Symptom may disappear when Symptom may disappear when another problem is fixedanother problem is fixed
Cause may be due to a Cause may be due to a combination of noncombination of non--errors errors
These courseware materials are to be used in conjunction with Software Engineering: A Practitioner’s Approach, 6/e and are provided with permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 24
symptomsymptomcausecause
Cause may be due to a system Cause may be due to a system or compiler erroror compiler error
Cause may be due to Cause may be due to assumptions that everyone assumptions that everyone believesbelieves
Symptom may be intermittentSymptom may be intermittent
Consequences of BugsConsequences of Bugs
damagedamage
seriousserious
extremeextremecatastrophiccatastrophic
infectiousinfectious
These courseware materials are to be used in conjunction with Software Engineering: A Practitioner’s Approach, 6/e and are provided with permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 25
mildmildannoyingannoying
disturbingdisturbingseriousserious
Bug TypeBug Type
Bug Categories:Bug Categories: functionfunction--related bugs, related bugs, systemsystem--related bugs, data bugs, coding bugs, related bugs, data bugs, coding bugs, design bugs, documentation bugs, standards design bugs, documentation bugs, standards violations, etc.violations, etc.
Debugging TechniquesDebugging Techniques
Brute force / testingBrute force / testing
BacktrackingBacktracking
These courseware materials are to be used in conjunction with Software Engineering: A Practitioner’s Approach, 6/e and are provided with permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 26
InductionInduction
DeductionDeduction
Debugging: Final ThoughtsDebugging: Final Thoughts
Don't run off halfDon't run off half--cocked, cocked, thinkthink about the about the symptom you're seeing.symptom you're seeing.
Use toolsUse tools (e.g., dynamic debugger) to gain (e.g., dynamic debugger) to gain more insight.more insight.
1.1.
2.2.
These courseware materials are to be used in conjunction with Software Engineering: A Practitioner’s Approach, 6/e and are provided with permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 27
If at an impasse, If at an impasse, get helpget help from someone else.from someone else.
Be absolutely sure to Be absolutely sure to conduct regression testsconduct regression testswhen you do "fix" the bug.when you do "fix" the bug.
3.3.
4.4.