Download - Sunita Bodigutla
![Page 1: Sunita Bodigutla](https://reader034.vdocuments.us/reader034/viewer/2022042512/54bdb67b4a7959110a8b458c/html5/thumbnails/1.jpg)
04/10/2304/10/23 11
Testing: A RoadmapTesting: A Roadmap
Mary Jean HarroldMary Jean Harrold
Sunita BodigutlaSunita Bodigutla
CS 589 Fall 2003CS 589 Fall 2003
![Page 2: Sunita Bodigutla](https://reader034.vdocuments.us/reader034/viewer/2022042512/54bdb67b4a7959110a8b458c/html5/thumbnails/2.jpg)
04/10/2304/10/23 22
DefinitionDefinition
A verification A verification method that applies method that applies a controlled set of a controlled set of conditions and conditions and stimuli for the stimuli for the purpose of finding purpose of finding errors.errors.
The paper says:The paper says: It’s a process that is It’s a process that is
performed to support performed to support quality assurancequality assurance
![Page 3: Sunita Bodigutla](https://reader034.vdocuments.us/reader034/viewer/2022042512/54bdb67b4a7959110a8b458c/html5/thumbnails/3.jpg)
04/10/2304/10/23 33
Quality AssuranceQuality Assurance
ISO 9000:2000 ISO 9000:2000 "Part of quality management focused on "Part of quality management focused on
providing confidence that quality requirements providing confidence that quality requirements will be fulfilled“will be fulfilled“
It is a planned program consisting of: all actions It is a planned program consisting of: all actions necessary to provide adequate confidence that necessary to provide adequate confidence that an item or product an item or product conformsconforms to established to established technical requirements; and, activities designed technical requirements; and, activities designed to evaluate the to evaluate the processprocess by which products are by which products are developed or manufactured.developed or manufactured.
A product does what it is expected to.A product does what it is expected to.
![Page 4: Sunita Bodigutla](https://reader034.vdocuments.us/reader034/viewer/2022042512/54bdb67b4a7959110a8b458c/html5/thumbnails/4.jpg)
04/10/2304/10/23 44
TestingTesting
Designing test casesDesigning test cases executing the software with those executing the software with those
test casestest cases examining the results produced by examining the results produced by
those executionsthose executions
![Page 5: Sunita Bodigutla](https://reader034.vdocuments.us/reader034/viewer/2022042512/54bdb67b4a7959110a8b458c/html5/thumbnails/5.jpg)
04/10/2304/10/23 55
ExpensiveExpensive
Testing accounts Testing accounts for for 50% 50% of the cost of the cost of software of software developmentdevelopment
Higher for critical Higher for critical systemssystems
Efficient ways to Efficient ways to perform testing to perform testing to control the costcontrol the cost
![Page 6: Sunita Bodigutla](https://reader034.vdocuments.us/reader034/viewer/2022042512/54bdb67b4a7959110a8b458c/html5/thumbnails/6.jpg)
04/10/2304/10/23 66
What this paper is aboutWhat this paper is about
A report written in 2000A report written in 2000 assesses the state of the art in assesses the state of the art in
software testingsoftware testing outlines future directions in software outlines future directions in software
testingtesting gives pointers to software testing gives pointers to software testing
resourcesresources
![Page 7: Sunita Bodigutla](https://reader034.vdocuments.us/reader034/viewer/2022042512/54bdb67b4a7959110a8b458c/html5/thumbnails/7.jpg)
04/10/2304/10/23 77
Testing Componentbased Systems
Testing based on Precode artifacts
Demonstrating effectiveness
of testing techniques
Testing Evolving Software
creating effective testing processes
Other testing approaches
Using testing artifacts
Methods and Tools
Empirical Studies
Practical Testing
Methods, Tools, and Processes
forDevelopment of
High-QualitySoftware
Fundamental Research
Status of Testing
Methods,Tools,and
Processes
fig 1. Software Testing Roadmap
![Page 8: Sunita Bodigutla](https://reader034.vdocuments.us/reader034/viewer/2022042512/54bdb67b4a7959110a8b458c/html5/thumbnails/8.jpg)
04/10/2304/10/23 88
Testing Componentbased Systems
Testing based on Precode artifacts
Demonstrating effectiveness
of testing techniques
Testing Evolving Software
creating effective testing processes
Other testing approaches
Using testing artifacts
Methods and Tools
Empirical Studies
Practical Testing
Methods, Tools, and Processes
forDevelopment of
High-QualitySoftware
Fundamental Research
Status of Testing
Methods,Tools,and
Processes
fig 2
![Page 9: Sunita Bodigutla](https://reader034.vdocuments.us/reader034/viewer/2022042512/54bdb67b4a7959110a8b458c/html5/thumbnails/9.jpg)
04/10/2304/10/23 99
Component Based SystemsComponent Based Systems
Increased size and Increased size and complexity of complexity of software systems software systems has led to has led to component based component based systemssystems
Component Component Developer vs. Developer vs. Component userComponent user
![Page 10: Sunita Bodigutla](https://reader034.vdocuments.us/reader034/viewer/2022042512/54bdb67b4a7959110a8b458c/html5/thumbnails/10.jpg)
04/10/2304/10/23 1010
Component UserComponent User
Component
ReusableService
Semantics
Repository
System
System Under Assemble
Select
Composible
Executable
Understand
Assembly
Compile
fig 3
![Page 11: Sunita Bodigutla](https://reader034.vdocuments.us/reader034/viewer/2022042512/54bdb67b4a7959110a8b458c/html5/thumbnails/11.jpg)
04/10/2304/10/23 1111
Component Based Systems Component Based Systems (contd.)(contd.)
COTS COTS : Commercial off the shelf software: Commercial off the shelf software e.g. Microsoft Active X, MFC, Java Applets e.g. Microsoft Active X, MFC, Java Applets
etcetc Why COTS-Why COTS-
saves moneysaves money saves timesaves time component reusecomponent reuse standardizationstandardization simplified designsimplified design
![Page 12: Sunita Bodigutla](https://reader034.vdocuments.us/reader034/viewer/2022042512/54bdb67b4a7959110a8b458c/html5/thumbnails/12.jpg)
04/10/2304/10/23 1212
Problems with Testing COTSProblems with Testing COTS
Unavailability of source codeUnavailability of source code Technical HeterogeneityTechnical Heterogeneity Enterprise HeterogeneityEnterprise Heterogeneity Dynamic evolutionDynamic evolution Component Trust ProblemComponent Trust Problem
![Page 13: Sunita Bodigutla](https://reader034.vdocuments.us/reader034/viewer/2022042512/54bdb67b4a7959110a8b458c/html5/thumbnails/13.jpg)
04/10/2304/10/23 1313
Component Based SoftwareComponent Based Software
Techniques for testing by Component Techniques for testing by Component provider- a number of themprovider- a number of them
Techniques available to component Techniques available to component user-very fewuser-very few
Theory of Theory of test adequacytest adequacy by by RosenblumRosenblum
![Page 14: Sunita Bodigutla](https://reader034.vdocuments.us/reader034/viewer/2022042512/54bdb67b4a7959110a8b458c/html5/thumbnails/14.jpg)
04/10/2304/10/23 1414
Test AdequacyTest Adequacy
A test adequacy criterion is a systematic criterion that is used to determine whether a test suite provides an adequate amount of testing for a component under test
C-adequate-for-P for adequate unit testing of a component
C-adequate-on-M for adequate integration testing of a component-based system
fig 4
![Page 15: Sunita Bodigutla](https://reader034.vdocuments.us/reader034/viewer/2022042512/54bdb67b4a7959110a8b458c/html5/thumbnails/15.jpg)
04/10/2304/10/23 1515
Future DirectionsFuture Directions
Efficient techniques and tools forEfficient techniques and tools for testing aspects like security, dependability and safetytesting aspects like security, dependability and safety representing and computing the types of testing representing and computing the types of testing
information (meta-data) that the component-user needsinformation (meta-data) that the component-user needs Java Beans: BeanInfoJava Beans: BeanInfo
storing the informationstoring the information using the informationusing the information A Framework for component deployment [1] called A Framework for component deployment [1] called
‘component deployment testing (CDT) framework’‘component deployment testing (CDT) framework’ component runtime introspectioncomponent runtime introspection reflection API of Javareflection API of Java
![Page 16: Sunita Bodigutla](https://reader034.vdocuments.us/reader034/viewer/2022042512/54bdb67b4a7959110a8b458c/html5/thumbnails/16.jpg)
04/10/2304/10/23 1616
Testing Componentbased Systems
Testing based on Precode artifacts
Demonstrating effectiveness
of testing techniques
Testing Evolving Software
creating effective testing processes
Other testing approaches
Using testing artifacts
Methods and Tools
Empirical Studies
Practical Testing
Methods, Tools, and Processes
forDevelopment of
High-QualitySoftware
Fundamental Research
Status of Testing
Methods,Tools,and
Processes
fig 5
![Page 17: Sunita Bodigutla](https://reader034.vdocuments.us/reader034/viewer/2022042512/54bdb67b4a7959110a8b458c/html5/thumbnails/17.jpg)
04/10/2304/10/23 1717
Integration test strategies and architectural Integration test strategies and architectural stylesstyles
fig 6
![Page 18: Sunita Bodigutla](https://reader034.vdocuments.us/reader034/viewer/2022042512/54bdb67b4a7959110a8b458c/html5/thumbnails/18.jpg)
04/10/2304/10/23 1818
Fault type and Integration Test StrategyFault type and Integration Test Strategy
fig 7
![Page 19: Sunita Bodigutla](https://reader034.vdocuments.us/reader034/viewer/2022042512/54bdb67b4a7959110a8b458c/html5/thumbnails/19.jpg)
04/10/2304/10/23 1919
Testing based on Architecture Testing based on Architecture specificationsspecifications
Formal notation for architecture specification Formal notation for architecture specification ‘‘CHAMCHAM’ : Chemical Abstract Machine.’ : Chemical Abstract Machine. Architecture: a set of components (molecules)Architecture: a set of components (molecules) states and interactions governed by rules states and interactions governed by rules
(reactions)(reactions) CHAM is specified by defining molecules m1,m2,…, CHAM is specified by defining molecules m1,m2,…,
solutions s1, s2,….. and transformation rules T, T’,solutions s1, s2,….. and transformation rules T, T’,…..…..
Solutions are multisets of molecules interpreted as Solutions are multisets of molecules interpreted as defining the states of CHAMdefining the states of CHAM
Transformation rules describe how the system Transformation rules describe how the system dynamically evolves through reaction stepsdynamically evolves through reaction steps
![Page 20: Sunita Bodigutla](https://reader034.vdocuments.us/reader034/viewer/2022042512/54bdb67b4a7959110a8b458c/html5/thumbnails/20.jpg)
04/10/2304/10/23 2020
Testing based on Architecture specifications Testing based on Architecture specifications (contd.)(contd.)
[4] Using the formal description of SA, a transition [4] Using the formal description of SA, a transition graph depicting all possible behaviors of the graph depicting all possible behaviors of the system is arrived at.system is arrived at.
A suitable set of reduced graphs are identified, A suitable set of reduced graphs are identified, each subgraph specific architectural properties of each subgraph specific architectural properties of the system.the system.
For each subgraph, test cases are derived to For each subgraph, test cases are derived to cover all the transitions in it.cover all the transitions in it.
![Page 21: Sunita Bodigutla](https://reader034.vdocuments.us/reader034/viewer/2022042512/54bdb67b4a7959110a8b458c/html5/thumbnails/21.jpg)
04/10/2304/10/23 2121
Future DirectionsFuture Directions
Techniques for Techniques for automatic generation of test cases from automatic generation of test cases from
formal specification of software formal specification of software architecturearchitecture
evaluating software architectures for evaluating software architectures for testabilitytestability
![Page 22: Sunita Bodigutla](https://reader034.vdocuments.us/reader034/viewer/2022042512/54bdb67b4a7959110a8b458c/html5/thumbnails/22.jpg)
04/10/2304/10/23 2222
Testing Componentbased Systems
Testing based on Precode artifacts
Demonstrating effectiveness
of testing techniques
Testing Evolving Software
creating effective testing processes
Other testing approaches
Using testing artifacts
Methods and Tools
Empirical Studies
Practical Testing
Methods, Tools, and Processes
forDevelopment of
High-QualitySoftware
Fundamental Research
Status of Testing
Methods,Tools,and
Processes
fig 8
![Page 23: Sunita Bodigutla](https://reader034.vdocuments.us/reader034/viewer/2022042512/54bdb67b4a7959110a8b458c/html5/thumbnails/23.jpg)
04/10/2304/10/23 2323
Testing Evolving SoftwareTesting Evolving Software
Regression Testing: Regression Testing: ensuring that no ensuring that no new errors are introduced into new errors are introduced into previously tested codepreviously tested code
ExpensiveExpensive Selective Regression Testing: which Selective Regression Testing: which
subset of a test suite must be rerun subset of a test suite must be rerun to test a new version of the systemto test a new version of the system
![Page 24: Sunita Bodigutla](https://reader034.vdocuments.us/reader034/viewer/2022042512/54bdb67b4a7959110a8b458c/html5/thumbnails/24.jpg)
04/10/2304/10/23 2424
Future DirectionsFuture Directions
Techniques toTechniques to assist in managing the test suitesassist in managing the test suites reduce the size of test suitesreduce the size of test suites prioritize test cases prioritize test cases
![Page 25: Sunita Bodigutla](https://reader034.vdocuments.us/reader034/viewer/2022042512/54bdb67b4a7959110a8b458c/html5/thumbnails/25.jpg)
04/10/2304/10/23 2525
Testing Componentbased Systems
Testing based on Precode artifacts
Demonstrating effectiveness
of testing techniques
Testing Evolving Software
creating effective testing processes
Other testing approaches
Using testing artifacts
Methods and Tools
Empirical Studies
Practical Testing
Methods, Tools, and Processes
forDevelopment of
High-QualitySoftware
Fundamental Research
Status of Testing
Methods,Tools,and
Processes
fig 9
![Page 26: Sunita Bodigutla](https://reader034.vdocuments.us/reader034/viewer/2022042512/54bdb67b4a7959110a8b458c/html5/thumbnails/26.jpg)
04/10/2304/10/23 2626
Test CriteriaTest Criteria
Effective for certain classes of faultsEffective for certain classes of faults data flowdata flow complex boolean expressions: Modified complex boolean expressions: Modified
Condition/decision coverageCondition/decision coverage
![Page 27: Sunita Bodigutla](https://reader034.vdocuments.us/reader034/viewer/2022042512/54bdb67b4a7959110a8b458c/html5/thumbnails/27.jpg)
04/10/2304/10/23 2727
Modified Condition/Decision CoverageModified Condition/Decision Coverage
Definition:Definition: Every point of entry and exit in the Every point of entry and exit in the
program has been invoked atleast onceprogram has been invoked atleast once every condition in a decision in the every condition in a decision in the
program has taken on all possible program has taken on all possible outcomes atleast onceoutcomes atleast once
each condition has been shown to each condition has been shown to independently affect the decisions independently affect the decisions outcomeoutcome
![Page 28: Sunita Bodigutla](https://reader034.vdocuments.us/reader034/viewer/2022042512/54bdb67b4a7959110a8b458c/html5/thumbnails/28.jpg)
04/10/2304/10/23 2828
Modified Condition/Decision Modified Condition/Decision Coverage (contd.)Coverage (contd.)
Simple exampleSimple example AA and and BB Extension of truth Extension of truth
table to pairs tabletable to pairs table The test set { (T T),The test set { (T T),
(T F), (F T)} is (T F), (F T)} is adequate for MC/DC adequate for MC/DC coveragecoverage
NumbNumberer ABAB ResulResul
ttAA BB
11 T TT T TT 33 22
22 T FT F FF 11
33 F TF T FF 11
44 F FF F FF
fig 10
![Page 29: Sunita Bodigutla](https://reader034.vdocuments.us/reader034/viewer/2022042512/54bdb67b4a7959110a8b458c/html5/thumbnails/29.jpg)
04/10/2304/10/23 2929
Future directionsFuture directions
Find ways Find ways to determine interaction between the to determine interaction between the
various test- selection criteria and find various test- selection criteria and find ways to combine them to perform more ways to combine them to perform more effective testingeffective testing
![Page 30: Sunita Bodigutla](https://reader034.vdocuments.us/reader034/viewer/2022042512/54bdb67b4a7959110a8b458c/html5/thumbnails/30.jpg)
04/10/2304/10/23 3030
Testing Componentbased Systems
Testing based on Precode artifacts
Demonstrating effectiveness
of testing techniques
Testing Evolving Software
creating effective testing processes
Other testing approaches
Using testing artifacts
Methods and Tools
Empirical Studies
Practical Testing
Methods, Tools, and Processes
forDevelopment of
High-QualitySoftware
Fundamental Research
Status of Testing
Methods,Tools,and
Processes
fig 11
![Page 31: Sunita Bodigutla](https://reader034.vdocuments.us/reader034/viewer/2022042512/54bdb67b4a7959110a8b458c/html5/thumbnails/31.jpg)
04/10/2304/10/23 3131
Effective Processes for Effective Processes for TestingTesting
Process used for planning and implementing Process used for planning and implementing itit
Test plan Test plan Perpetual testingPerpetual testing Microsoft uses a different modelMicrosoft uses a different model
synchronize and stabilizesynchronize and stabilize Future directionsFuture directions
validate the existing modelsvalidate the existing models integration of various quality techniques and toolsintegration of various quality techniques and tools
![Page 32: Sunita Bodigutla](https://reader034.vdocuments.us/reader034/viewer/2022042512/54bdb67b4a7959110a8b458c/html5/thumbnails/32.jpg)
04/10/2304/10/23 3232
Using Testing ArtifactsUsing Testing Artifacts
Execution TracesExecution Traces Information about whichInformation about which
statements were executedstatements were executed paths were executedpaths were executed values particular variables acquiredvalues particular variables acquired
Artifacts can be stored for use in retesting the Artifacts can be stored for use in retesting the software after it is modifiedsoftware after it is modified dynamic program slicesdynamic program slices heuristic approachesheuristic approaches List of invariantsList of invariants Regression TestingRegression Testing Visualization techniqueVisualization technique
![Page 33: Sunita Bodigutla](https://reader034.vdocuments.us/reader034/viewer/2022042512/54bdb67b4a7959110a8b458c/html5/thumbnails/33.jpg)
04/10/2304/10/23 3333
Other topicsOther topics
Methods and ToolsMethods and Tools ScalabilityScalability Computational tradeoffsComputational tradeoffs Automatic generationAutomatic generation
Empirical StudiesEmpirical Studies Testing ResourcesTesting Resources
![Page 34: Sunita Bodigutla](https://reader034.vdocuments.us/reader034/viewer/2022042512/54bdb67b4a7959110a8b458c/html5/thumbnails/34.jpg)
04/10/2304/10/23 3434
ConclusionConclusion
OverviewOverview List of techniquesList of techniques No comparison No comparison Future directionsFuture directions Testing is a wide areaTesting is a wide area Dynamic fieldDynamic field
![Page 35: Sunita Bodigutla](https://reader034.vdocuments.us/reader034/viewer/2022042512/54bdb67b4a7959110a8b458c/html5/thumbnails/35.jpg)
04/10/2304/10/23 3535
[1] Antonia Bertolino, Andrea Polini, “A Framework for Component Deployment [1] Antonia Bertolino, Andrea Polini, “A Framework for Component Deployment Testing”Testing”
[2] A.Orso, M.J.Harrold and D.Rosenblum, “Component Metadata for Software [2] A.Orso, M.J.Harrold and D.Rosenblum, “Component Metadata for Software Engineering Tasks”, in W.Emmerich and S.Tai(Eds) EDO 2000, LNCS 1999, pp. 129-Engineering Tasks”, in W.Emmerich and S.Tai(Eds) EDO 2000, LNCS 1999, pp. 129-144144
[3] J.A.Stafford and A.L.Wolf, “Annotating components to support component based [3] J.A.Stafford and A.L.Wolf, “Annotating components to support component based static analysis of software systems “, static analysis of software systems “, Proc. the Grace Hopper Celeb. of women in Proc. the Grace Hopper Celeb. of women in computing 2001computing 2001
[4] Antonia Bertolino, Paola Inverardi, Henry Muccini, Andrea Rosetti, “An Approach [4] Antonia Bertolino, Paola Inverardi, Henry Muccini, Andrea Rosetti, “An Approach to Integration Testing based on Architectural Descriptions”to Integration Testing based on Architectural Descriptions”
References
![Page 36: Sunita Bodigutla](https://reader034.vdocuments.us/reader034/viewer/2022042512/54bdb67b4a7959110a8b458c/html5/thumbnails/36.jpg)
04/10/2304/10/23 3636
fig 12. Sequential Multiphase Compiler Architecture
![Page 37: Sunita Bodigutla](https://reader034.vdocuments.us/reader034/viewer/2022042512/54bdb67b4a7959110a8b458c/html5/thumbnails/37.jpg)
04/10/2304/10/23 3737
fig 13.Transition Graph for the Sequential Compile
![Page 38: Sunita Bodigutla](https://reader034.vdocuments.us/reader034/viewer/2022042512/54bdb67b4a7959110a8b458c/html5/thumbnails/38.jpg)
04/10/2304/10/23 3838
fig 14. Concurrent Reduced Graph
![Page 39: Sunita Bodigutla](https://reader034.vdocuments.us/reader034/viewer/2022042512/54bdb67b4a7959110a8b458c/html5/thumbnails/39.jpg)
04/10/2304/10/23 3939