software defect prevention using orthogonal defect classification twin-spin january 6, 2005...
TRANSCRIPT
![Page 1: Software Defect Prevention Using Orthogonal Defect Classification Twin-SPIN January 6, 2005 Presented by: Megan Graham, ASQ CSQE megan@metagraham.net](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e725503460f94b70bfb/html5/thumbnails/1.jpg)
Software Defect PreventionUsing Orthogonal Defect Classification
Twin-SPIN
January 6, 2005
Presented by:
Megan Graham, ASQ CSQE
![Page 2: Software Defect Prevention Using Orthogonal Defect Classification Twin-SPIN January 6, 2005 Presented by: Megan Graham, ASQ CSQE megan@metagraham.net](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e725503460f94b70bfb/html5/thumbnails/2.jpg)
2
Overview
• Defect prevention is a process used to improve software quality
• Orthogonal Defect Classification is a tool that characterizes defect data used in defect analysis
![Page 3: Software Defect Prevention Using Orthogonal Defect Classification Twin-SPIN January 6, 2005 Presented by: Megan Graham, ASQ CSQE megan@metagraham.net](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e725503460f94b70bfb/html5/thumbnails/3.jpg)
3
Agenda
• About Defect Prevention
• ODC Concepts
• Applying ODC to the DP Process
• Summary
![Page 4: Software Defect Prevention Using Orthogonal Defect Classification Twin-SPIN January 6, 2005 Presented by: Megan Graham, ASQ CSQE megan@metagraham.net](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e725503460f94b70bfb/html5/thumbnails/4.jpg)
4
Agenda
• About Defect Prevention
• ODC Concepts
• Applying ODC to the DP Process
• Summary
![Page 5: Software Defect Prevention Using Orthogonal Defect Classification Twin-SPIN January 6, 2005 Presented by: Megan Graham, ASQ CSQE megan@metagraham.net](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e725503460f94b70bfb/html5/thumbnails/5.jpg)
5
Defining Defect Prevention
• What does it mean to you?• Defect prevention is a process whose
purpose is to:– identify the common causes of defects, and– change the relevant process(es) to prevent that
type of defect from recurring. (SEI)
• Take what we already know and apply it to what we think we know to produce quality software.
![Page 6: Software Defect Prevention Using Orthogonal Defect Classification Twin-SPIN January 6, 2005 Presented by: Megan Graham, ASQ CSQE megan@metagraham.net](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e725503460f94b70bfb/html5/thumbnails/6.jpg)
6
A Bug’s LifeRequirementsRequirements
DesignDesign
CodeCode
TestTest
![Page 7: Software Defect Prevention Using Orthogonal Defect Classification Twin-SPIN January 6, 2005 Presented by: Megan Graham, ASQ CSQE megan@metagraham.net](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e725503460f94b70bfb/html5/thumbnails/7.jpg)
7
A Bird’s Eye View of DP
DecreaseDecreaseDefectsDefectsInjectedInjected
IncreasIncreasee
DefectsDefectsDetecteDetecte
dd
DefectDefectPreventionPrevention
HistoricalHistoricalProjectProjectDefectsDefects
CurrentCurrentProjectProjectDefectsDefects
![Page 8: Software Defect Prevention Using Orthogonal Defect Classification Twin-SPIN January 6, 2005 Presented by: Megan Graham, ASQ CSQE megan@metagraham.net](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e725503460f94b70bfb/html5/thumbnails/8.jpg)
8
Applying Defect Prevention
• A defect in the software is also a defect in the process (injection and/or detection)
• For DP to work, we need to turn software defects into actionable process defects
Softwaredefects
Processdefects
DP
![Page 9: Software Defect Prevention Using Orthogonal Defect Classification Twin-SPIN January 6, 2005 Presented by: Megan Graham, ASQ CSQE megan@metagraham.net](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e725503460f94b70bfb/html5/thumbnails/9.jpg)
9
And then there was ODC…
• Orthogonal Defect Classification
• Developed at IBM in the 1990s by Ram Chillarege
• Methodology to characterize software defects and translate into process defects
![Page 10: Software Defect Prevention Using Orthogonal Defect Classification Twin-SPIN January 6, 2005 Presented by: Megan Graham, ASQ CSQE megan@metagraham.net](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e725503460f94b70bfb/html5/thumbnails/10.jpg)
10
Agenda
• About Defect Prevention
• ODC Concepts
• Applying ODC to the DP Process
• Summary
![Page 11: Software Defect Prevention Using Orthogonal Defect Classification Twin-SPIN January 6, 2005 Presented by: Megan Graham, ASQ CSQE megan@metagraham.net](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e725503460f94b70bfb/html5/thumbnails/11.jpg)
11
Simple ODC Classification Scheme
DefectDefect
TypeTypeTypeType
TriggerTriggerTriggerTrigger
![Page 12: Software Defect Prevention Using Orthogonal Defect Classification Twin-SPIN January 6, 2005 Presented by: Megan Graham, ASQ CSQE megan@metagraham.net](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e725503460f94b70bfb/html5/thumbnails/12.jpg)
12
Defect Types• FUNCTION: Affects significant capability, end-user interfaces, product
interfaces, interface with hardware architecture or global data structure(s).
• LOGIC: Affects the functionality of a code module and can be fixed by re-implementing an algorithm or local data structure without a need for requesting a high level design change.
• INTERFACE: Affects the interaction of components via macros, call statements and/or parameters.
• CHECKING: Affects program logic that would properly validate data and values before they are stored or used in computation.
• ASSIGNMENT: Requires change in a few lines of code, such as initialization of control blocks or data structures.
• TIMING/SERIALIZATION: Affects the proper management of shared and real-time resources.
• BUILD/PACKAGE/MERGE: Affects product version or configuration; requires a formal changes to reconfigure or rebuild the product.
![Page 13: Software Defect Prevention Using Orthogonal Defect Classification Twin-SPIN January 6, 2005 Presented by: Megan Graham, ASQ CSQE megan@metagraham.net](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e725503460f94b70bfb/html5/thumbnails/13.jpg)
13
Defect Triggers
Fault Failure
Trigger 1
Trigger 2
Trigger n
![Page 14: Software Defect Prevention Using Orthogonal Defect Classification Twin-SPIN January 6, 2005 Presented by: Megan Graham, ASQ CSQE megan@metagraham.net](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e725503460f94b70bfb/html5/thumbnails/14.jpg)
14
Review/Inspection Triggers• DESIGN CONFORMANCE: Comparing the implemented design against a
reference –design document, pattern, or guideline.• LOGIC/FLOW: Checking for correctness or flaws using knowledge of the
practice.• BACKWARD COMPATIBILITY: Examining compatibility with prior version of
the product.• LATERAL COMPATIBILITY: Examining for compatibility with other products
and platforms that need to work with this release.• CONCURRENCY: Serialization, shared resources, multi-threaded tasks, timing,
etc.• INTERNAL DOCUMENT: Inconsistencies in prologs, and sections in the same
work product.• LANGUAGE DEPENDENCY: Programming standards, specific implementation
considerations, environment restrictions, execution modes, etc.• SIDE EFFECTS: Usage behavior beyond design, but relevant in context. Do A; B
happens.• RARE SITUATION: Unusual issues related to idiosyncrasy of environment,
hardware, or software.
![Page 15: Software Defect Prevention Using Orthogonal Defect Classification Twin-SPIN January 6, 2005 Presented by: Megan Graham, ASQ CSQE megan@metagraham.net](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e725503460f94b70bfb/html5/thumbnails/15.jpg)
15
Function Test Triggers• SIMPLE PATH: White box – Straightforward usage of code path and
branches.
• COMPLEX PATH: White box – Exercising conditionals, and circuitous coverage.
• COVERAGE: Black box – Straightforward usage of function or single parameterized execution.
• VARIATION: Black box – Straightforward like coverage, but with a variety of parameters.
• SEQUENCING: Black box – Multiple functions, with a specific sequence (order is important).
• INTERACTION: Black box – When two or more bodies of code are involved (order is not important).
![Page 16: Software Defect Prevention Using Orthogonal Defect Classification Twin-SPIN January 6, 2005 Presented by: Megan Graham, ASQ CSQE megan@metagraham.net](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e725503460f94b70bfb/html5/thumbnails/16.jpg)
16
System Test Triggers• WORKLOAD STRESS: Pushing the limits of performance, resources, users,
queues, traffic, etc.
• RECOVERY: Invoke exception handling, recovery, termination, error percolation, etc.
• STARTUP/RESTART: Major events of turning on, off, or changing the degree of service availability.
• HARDWARE CONFIGURATION: Issues surfaced as a consequence of changes in hardware setup.
• SOFTWARE CONFIGURATION: Issues surfaced as a consequence of changes in software setup.
• BLOCKED TEST/NORMAL MODE: Test could not run during System Test, or customer found nonspecific trigger. Look for additional trigger.
![Page 17: Software Defect Prevention Using Orthogonal Defect Classification Twin-SPIN January 6, 2005 Presented by: Megan Graham, ASQ CSQE megan@metagraham.net](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e725503460f94b70bfb/html5/thumbnails/17.jpg)
17
Opener
Closer
ODC v5.11 Classification Scheme
Source: ODC v5.11, IBM Center for Software Engineering, www.research.ibm.com/softeng
DefectDefect
TypeType SourceSource
ImpactImpactTriggerTrigger
ActivityActivity
TargetTarget
QualifierQualifier
AgeAge
![Page 18: Software Defect Prevention Using Orthogonal Defect Classification Twin-SPIN January 6, 2005 Presented by: Megan Graham, ASQ CSQE megan@metagraham.net](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e725503460f94b70bfb/html5/thumbnails/18.jpg)
18
Agenda
• About Defect Prevention
• ODC Concepts
• Applying ODC to the DP Process
• Summary
![Page 19: Software Defect Prevention Using Orthogonal Defect Classification Twin-SPIN January 6, 2005 Presented by: Megan Graham, ASQ CSQE megan@metagraham.net](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e725503460f94b70bfb/html5/thumbnails/19.jpg)
19
When to Apply DP
• DP can be applied to one or more phases of the software lifecycle
• Depends on maturity, goals, etc.
RQTS DSGN CODE TEST
Defect Prevention
Field
Ana
lysi
s Action
Ana
lysi
s Action
Ana
lysi
s Action
Ana
lysi
s Action
Ana
lysi
s Action
![Page 20: Software Defect Prevention Using Orthogonal Defect Classification Twin-SPIN January 6, 2005 Presented by: Megan Graham, ASQ CSQE megan@metagraham.net](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e725503460f94b70bfb/html5/thumbnails/20.jpg)
20
Defect Analysis
Orthogonal Defect Orthogonal Defect ClassificationClassification
What are the attributes of the defects?
Defect CausalDefect CausalAnalysisAnalysis
When are defectsbeing injected?
Defect TriggerDefect TriggerAnalysisAnalysis
How are defectsbeing detected?
ODC provides a structure for the defect data
![Page 21: Software Defect Prevention Using Orthogonal Defect Classification Twin-SPIN January 6, 2005 Presented by: Megan Graham, ASQ CSQE megan@metagraham.net](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e725503460f94b70bfb/html5/thumbnails/21.jpg)
21
Defect Causal Analysis
• Purpose: Characterize process issues that lead to injection of defects.
Step 1: ID a set of defects.
Step 2: Identify Defect Types with team of experts.
Step 3: Plot the distribution of Defect Types.
Step 4: Map Defect Types back to development activity.
Step 5: Develop action plan to address process deficiencies.
Step 6: Monitor process to ensure changes were effective.
![Page 22: Software Defect Prevention Using Orthogonal Defect Classification Twin-SPIN January 6, 2005 Presented by: Megan Graham, ASQ CSQE megan@metagraham.net](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e725503460f94b70bfb/html5/thumbnails/22.jpg)
22
Defect Type Mapped to Phase
H/L Level DesignInterface
Configuration MgmtBuild/Package/Merge
ImplementationTiming/Serialization
ImplementationAssignment
LLD/ImplementationChecking
Low Level DesignLogic
Architecture/HLDFunction
![Page 23: Software Defect Prevention Using Orthogonal Defect Classification Twin-SPIN January 6, 2005 Presented by: Megan Graham, ASQ CSQE megan@metagraham.net](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e725503460f94b70bfb/html5/thumbnails/23.jpg)
23
Sample Distribution
0%
20%
40%
60%
80%
100%
Inter
face
B/P/M
Logic
Checki
ng
Assign
men
t
Timin
g/Ser
ial
Functio
n
![Page 24: Software Defect Prevention Using Orthogonal Defect Classification Twin-SPIN January 6, 2005 Presented by: Megan Graham, ASQ CSQE megan@metagraham.net](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e725503460f94b70bfb/html5/thumbnails/24.jpg)
24
Defect Trigger Analysis
• Purpose: Characterize process issues that allowed defects to escape to later phases.
Step 1: ID a set of defects.
Step 2: Identify Defect Trigger with team of experts.
Step 3: Plot distributions: Defect Trigger by Family, Review Triggers, Function Test Triggers, System Test Triggers.
Step 4: Map Defect Trigger family back to detection activity(ies).
Step 5: Develop action plan to increase missed Defect Triggers.
Step 6: Monitor process to ensure changes were effective.
![Page 25: Software Defect Prevention Using Orthogonal Defect Classification Twin-SPIN January 6, 2005 Presented by: Megan Graham, ASQ CSQE megan@metagraham.net](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e725503460f94b70bfb/html5/thumbnails/25.jpg)
25
Defect Trigger Family Mapped to Phase
Testing in real-world scenarios or extreme scenarios
System Test
Testing of specified scenariosFunction Test
Peer Reviews or InspectionsReview/Inspection
![Page 26: Software Defect Prevention Using Orthogonal Defect Classification Twin-SPIN January 6, 2005 Presented by: Megan Graham, ASQ CSQE megan@metagraham.net](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e725503460f94b70bfb/html5/thumbnails/26.jpg)
26
Sample Distributions
Defect Triggers by Family
0
10
20
30
40
50
60
Review/Inspection Function Test System Test
Function Test Triggers
02468
1012141618
Seq
uenc
ing
Var
iatio
n
Inte
ract
ion
Cov
erag
e
Sim
ple
Pat
h
Com
plex
Pat
h
![Page 27: Software Defect Prevention Using Orthogonal Defect Classification Twin-SPIN January 6, 2005 Presented by: Megan Graham, ASQ CSQE megan@metagraham.net](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e725503460f94b70bfb/html5/thumbnails/27.jpg)
27
Additional Use for DTA
• Purpose: Determine quality status.
Step 1: Plot distribution of Defect Triggers after each detection activity.
Step 2: Compare to historical defect profile to determine if profile is as expected.
Step 3: Develop action plan, if necessary, to get back on track.
• If your project thinks it’s one phase, but the distribution is of an earlier phase, then you’re really in the earlier phase!
![Page 28: Software Defect Prevention Using Orthogonal Defect Classification Twin-SPIN January 6, 2005 Presented by: Megan Graham, ASQ CSQE megan@metagraham.net](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e725503460f94b70bfb/html5/thumbnails/28.jpg)
28
Agenda
• About Defect Prevention
• ODC Concepts
• Applying ODC to the DP Process
• Summary
![Page 29: Software Defect Prevention Using Orthogonal Defect Classification Twin-SPIN January 6, 2005 Presented by: Megan Graham, ASQ CSQE megan@metagraham.net](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e725503460f94b70bfb/html5/thumbnails/29.jpg)
29
The Big Picture: DP & ODC
• Mission: Improve software quality by using readily available data to decrease defects injected and increase defects detected.
Defect Prevention
RQTS DSGN CODE TEST Field
Ana
lysi
s Action
Ana
lysi
s Action
Ana
lysi
s Action
Ana
lysi
s Action
Ana
lysi
s Action
ODC Classification of Software Defects
![Page 30: Software Defect Prevention Using Orthogonal Defect Classification Twin-SPIN January 6, 2005 Presented by: Megan Graham, ASQ CSQE megan@metagraham.net](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e725503460f94b70bfb/html5/thumbnails/30.jpg)
30
Important Points about ODC
• A defect in the software is a defect in the process.• Implementing ODC is very cost-effective
– Enhances data already collected (software defects)– Adding fields that are completed real-time make data
collection virtually free!– Tooled to quickly identify process defects (mapping)
• ODC can be implemented in stages– Start with field defects, then move to in-process analysis– Utilize defect profiling in-process to predict quality and
project status
• Fields can be tailored to your own organization
![Page 31: Software Defect Prevention Using Orthogonal Defect Classification Twin-SPIN January 6, 2005 Presented by: Megan Graham, ASQ CSQE megan@metagraham.net](https://reader035.vdocuments.us/reader035/viewer/2022062801/56649e725503460f94b70bfb/html5/thumbnails/31.jpg)
31
References
• Handbook of Software Reliability Engineering (Michael R. Lyu, Editor; IEEE Computer Society Press/McGraw-Hill)
• Ram Chillarege (www.chillarege.com)• IBM Research Center for Software Engineering (
www.research.ibm.com/softeng)• Soheil Khajenoori, SIAGroup (