© michael crosby and charles sacker, 2001 systematic software reviews software reviews are a...
DESCRIPTION
© Michael Crosby and Charles Sacker, 2001 Basic Steps: Using a static analysis technique, Perform a visual examination of the software products Detect and correct: Defects Violation of design standards Other problemsTRANSCRIPT
![Page 1: © Michael Crosby and Charles Sacker, 2001 Systematic Software Reviews Software reviews are a “quality improvement process for written material”](https://reader035.vdocuments.us/reader035/viewer/2022081514/5a4d1aec7f8b9ab05997ba7b/html5/thumbnails/1.jpg)
© Michael Crosby and Charles Sacker, 2001
Systematic Software Reviews
Software reviews are a “quality improvement process for written material”.
![Page 2: © Michael Crosby and Charles Sacker, 2001 Systematic Software Reviews Software reviews are a “quality improvement process for written material”](https://reader035.vdocuments.us/reader035/viewer/2022081514/5a4d1aec7f8b9ab05997ba7b/html5/thumbnails/2.jpg)
© Michael Crosby and Charles Sacker, 2001
Goal and Motivation:
By detecting defects early, and preventing their leakage downstream, the higher cost of later detection and rework is eliminated.
![Page 3: © Michael Crosby and Charles Sacker, 2001 Systematic Software Reviews Software reviews are a “quality improvement process for written material”](https://reader035.vdocuments.us/reader035/viewer/2022081514/5a4d1aec7f8b9ab05997ba7b/html5/thumbnails/3.jpg)
© Michael Crosby and Charles Sacker, 2001
Basic Steps:• Using a static analysis technique, • Perform a visual examination of
the software products• Detect and correct:
• Defects• Violation of design standards• Other problems
![Page 4: © Michael Crosby and Charles Sacker, 2001 Systematic Software Reviews Software reviews are a “quality improvement process for written material”](https://reader035.vdocuments.us/reader035/viewer/2022081514/5a4d1aec7f8b9ab05997ba7b/html5/thumbnails/4.jpg)
© Michael Crosby and Charles Sacker, 2001
What is a Software ProductThe term “software product” is used in a very broad sense to describe any document produced during the software lifecycle.
![Page 5: © Michael Crosby and Charles Sacker, 2001 Systematic Software Reviews Software reviews are a “quality improvement process for written material”](https://reader035.vdocuments.us/reader035/viewer/2022081514/5a4d1aec7f8b9ab05997ba7b/html5/thumbnails/5.jpg)
© Michael Crosby and Charles Sacker, 2001
Examples of Software Products
Include: Contracts Installation plans Progress reports Software design descriptions Release notes Software requirements
specifications Source code
![Page 6: © Michael Crosby and Charles Sacker, 2001 Systematic Software Reviews Software reviews are a “quality improvement process for written material”](https://reader035.vdocuments.us/reader035/viewer/2022081514/5a4d1aec7f8b9ab05997ba7b/html5/thumbnails/6.jpg)
© Michael Crosby and Charles Sacker, 2001
What Is a Defect? Any occurrence in a work product
that is determined to be incomplete, incorrect, or missing
Any instance which a requirement is not satisfied(Fagan, 1986)
Informal synonyms:bug, fault, issue, problem, anomaly
![Page 7: © Michael Crosby and Charles Sacker, 2001 Systematic Software Reviews Software reviews are a “quality improvement process for written material”](https://reader035.vdocuments.us/reader035/viewer/2022081514/5a4d1aec7f8b9ab05997ba7b/html5/thumbnails/7.jpg)
© Michael Crosby and Charles Sacker, 2001
Common Attributes:Systematic reviews have these attributes in common:• Team participation• Documented results of the review• Documented procedures for
conducting the review
![Page 8: © Michael Crosby and Charles Sacker, 2001 Systematic Software Reviews Software reviews are a “quality improvement process for written material”](https://reader035.vdocuments.us/reader035/viewer/2022081514/5a4d1aec7f8b9ab05997ba7b/html5/thumbnails/8.jpg)
© Michael Crosby and Charles Sacker, 2001
Industry Experience With Reviews
Aetna Insurance Company: FTR found 82% of errors, 25% cost reduction.
Bell-Northern Research: Inspection cost: 1 hour per defect. Testing cost: 2-4 hours per defect. Post-release cost: 33 hours per defect.
Hewlett-Packard Est. inspection savings (1993): $21,454,000
IBM C system software No errors from time of first compile.
![Page 9: © Michael Crosby and Charles Sacker, 2001 Systematic Software Reviews Software reviews are a “quality improvement process for written material”](https://reader035.vdocuments.us/reader035/viewer/2022081514/5a4d1aec7f8b9ab05997ba7b/html5/thumbnails/9.jpg)
© Michael Crosby and Charles Sacker, 2001
Inspections vs. Reviews The IEEE Standard for Software
Reviews defines 5 types of review:• Management Reviews• Technical Reviews• Inspections (Formal Peer Review)• Walk-throughs• Audits
![Page 10: © Michael Crosby and Charles Sacker, 2001 Systematic Software Reviews Software reviews are a “quality improvement process for written material”](https://reader035.vdocuments.us/reader035/viewer/2022081514/5a4d1aec7f8b9ab05997ba7b/html5/thumbnails/10.jpg)
© Michael Crosby and Charles Sacker, 2001
Details of the Five Types of Software Review
![Page 11: © Michael Crosby and Charles Sacker, 2001 Systematic Software Reviews Software reviews are a “quality improvement process for written material”](https://reader035.vdocuments.us/reader035/viewer/2022081514/5a4d1aec7f8b9ab05997ba7b/html5/thumbnails/11.jpg)
© Michael Crosby and Charles Sacker, 2001
Management Review Overview Performed by those directly
responsible for the system Monitor progress Determine status of plans and
schedules Confirm requirements and their
system allocation Or, evaluate management approaches
used to achieve fitness or purpose
![Page 12: © Michael Crosby and Charles Sacker, 2001 Systematic Software Reviews Software reviews are a “quality improvement process for written material”](https://reader035.vdocuments.us/reader035/viewer/2022081514/5a4d1aec7f8b9ab05997ba7b/html5/thumbnails/12.jpg)
© Michael Crosby and Charles Sacker, 2001
Management Review Overview Continued
Support decisions made about:• Corrective actions• Changes in the allocation of resources• Or changes to the scope of the project.
![Page 13: © Michael Crosby and Charles Sacker, 2001 Systematic Software Reviews Software reviews are a “quality improvement process for written material”](https://reader035.vdocuments.us/reader035/viewer/2022081514/5a4d1aec7f8b9ab05997ba7b/html5/thumbnails/13.jpg)
© Michael Crosby and Charles Sacker, 2001
Management Reviews Continued
Software products reviewed Audit Reports Contingency plans Installation plans Risk management plans Software Q/A
![Page 14: © Michael Crosby and Charles Sacker, 2001 Systematic Software Reviews Software reviews are a “quality improvement process for written material”](https://reader035.vdocuments.us/reader035/viewer/2022081514/5a4d1aec7f8b9ab05997ba7b/html5/thumbnails/14.jpg)
© Michael Crosby and Charles Sacker, 2001
Management Review RolesRequired:• Decision Maker• Review Leader• Recorder• Management Staff• Technical Staff
![Page 15: © Michael Crosby and Charles Sacker, 2001 Systematic Software Reviews Software reviews are a “quality improvement process for written material”](https://reader035.vdocuments.us/reader035/viewer/2022081514/5a4d1aec7f8b9ab05997ba7b/html5/thumbnails/15.jpg)
© Michael Crosby and Charles Sacker, 2001
Management Review OutputsDocumented evidence that identifies:
• Project under review• Review team members• Review objects• Software product reviewed• Inputs to the review• Action item status• List of defects identified by the review team
![Page 16: © Michael Crosby and Charles Sacker, 2001 Systematic Software Reviews Software reviews are a “quality improvement process for written material”](https://reader035.vdocuments.us/reader035/viewer/2022081514/5a4d1aec7f8b9ab05997ba7b/html5/thumbnails/16.jpg)
© Michael Crosby and Charles Sacker, 2001
Technical Review OverviewConfirms that product Conforms to specifications Adheres to regulations, standards,
guidelines, plans Changes are properly implemented Changes affect only those system areas
identified by the change specification
![Page 17: © Michael Crosby and Charles Sacker, 2001 Systematic Software Reviews Software reviews are a “quality improvement process for written material”](https://reader035.vdocuments.us/reader035/viewer/2022081514/5a4d1aec7f8b9ab05997ba7b/html5/thumbnails/17.jpg)
© Michael Crosby and Charles Sacker, 2001
Technical Reviews ContinuedSoftware products subject to technical
reviews• Software requirements specification• Software design description• Software test documentation• Software user documentation• Installation procedure• Release notes
![Page 18: © Michael Crosby and Charles Sacker, 2001 Systematic Software Reviews Software reviews are a “quality improvement process for written material”](https://reader035.vdocuments.us/reader035/viewer/2022081514/5a4d1aec7f8b9ab05997ba7b/html5/thumbnails/18.jpg)
© Michael Crosby and Charles Sacker, 2001
Technical Review RolesThe roles established for the technical
review• Decision maker• Review leader• Recorder• Technical staff
![Page 19: © Michael Crosby and Charles Sacker, 2001 Systematic Software Reviews Software reviews are a “quality improvement process for written material”](https://reader035.vdocuments.us/reader035/viewer/2022081514/5a4d1aec7f8b9ab05997ba7b/html5/thumbnails/19.jpg)
© Michael Crosby and Charles Sacker, 2001
Technical Review OutputsOutputs, documented evidence that identifies:
• Project under review• Review team members• Software product reviewed• Inputs to the review• Review objectives and status• List of resolved and unresolved software defects• List of unresolved system or hardware defects• List of management issues• Action item status• Recommendations for unresolved issues• Whether software product meets specification
![Page 20: © Michael Crosby and Charles Sacker, 2001 Systematic Software Reviews Software reviews are a “quality improvement process for written material”](https://reader035.vdocuments.us/reader035/viewer/2022081514/5a4d1aec7f8b9ab05997ba7b/html5/thumbnails/20.jpg)
© Michael Crosby and Charles Sacker, 2001
Inspections (Formal Peer Reviews)
Confirms that the software product satisfies Specifications Specified quality attributes regulations, standards, guidelines, plans Identifies deviations from standard and
specification
Failure to do so results in logging a defect
![Page 21: © Michael Crosby and Charles Sacker, 2001 Systematic Software Reviews Software reviews are a “quality improvement process for written material”](https://reader035.vdocuments.us/reader035/viewer/2022081514/5a4d1aec7f8b9ab05997ba7b/html5/thumbnails/21.jpg)
© Michael Crosby and Charles Sacker, 2001
Inspections ContinuedSoftware products subject to Inspections
• Software requirements specification• Software design description• Source code• Software test documentation• Software user documentation• Maintenance manual• Release notes
![Page 22: © Michael Crosby and Charles Sacker, 2001 Systematic Software Reviews Software reviews are a “quality improvement process for written material”](https://reader035.vdocuments.us/reader035/viewer/2022081514/5a4d1aec7f8b9ab05997ba7b/html5/thumbnails/22.jpg)
© Michael Crosby and Charles Sacker, 2001
Inspection RolesThe roles established for the Inspection
• Inspection leader• Recorder• Reader• Author• Inspector
![Page 23: © Michael Crosby and Charles Sacker, 2001 Systematic Software Reviews Software reviews are a “quality improvement process for written material”](https://reader035.vdocuments.us/reader035/viewer/2022081514/5a4d1aec7f8b9ab05997ba7b/html5/thumbnails/23.jpg)
© Michael Crosby and Charles Sacker, 2001
Inspection OutputsOutputs, documented evidence that identifies:
• Project under inspection• Inspection team members• Inspection meeting duration• Software product inspected• Size of the materials inspected• Inputs to inspection• Inspection objectives and status• Defect list (detail)• Defect summary list• Disposition of the software product• Estimate of the rework effort and completion date
![Page 24: © Michael Crosby and Charles Sacker, 2001 Systematic Software Reviews Software reviews are a “quality improvement process for written material”](https://reader035.vdocuments.us/reader035/viewer/2022081514/5a4d1aec7f8b9ab05997ba7b/html5/thumbnails/24.jpg)
© Michael Crosby and Charles Sacker, 2001
Walk-throughs• Evaluate a software product• Sometimes used for educating an
audience• Major objectives:
• Find anomalies• Improve the software product• Consider alternative implementations• Evaluate performance to standards and
specs
![Page 25: © Michael Crosby and Charles Sacker, 2001 Systematic Software Reviews Software reviews are a “quality improvement process for written material”](https://reader035.vdocuments.us/reader035/viewer/2022081514/5a4d1aec7f8b9ab05997ba7b/html5/thumbnails/25.jpg)
© Michael Crosby and Charles Sacker, 2001
Walk-throughs ContinuedSoftware products subject to walk-
throughs• Software requirements specification• Software design description• Source code• Software test documentation• Software user documentation• Maintenance manual• Release notes
![Page 26: © Michael Crosby and Charles Sacker, 2001 Systematic Software Reviews Software reviews are a “quality improvement process for written material”](https://reader035.vdocuments.us/reader035/viewer/2022081514/5a4d1aec7f8b9ab05997ba7b/html5/thumbnails/26.jpg)
© Michael Crosby and Charles Sacker, 2001
Walk-through RolesThe roles established for Walk-throughs
• Walk-through leader• Recorder• Author• Team member
![Page 27: © Michael Crosby and Charles Sacker, 2001 Systematic Software Reviews Software reviews are a “quality improvement process for written material”](https://reader035.vdocuments.us/reader035/viewer/2022081514/5a4d1aec7f8b9ab05997ba7b/html5/thumbnails/27.jpg)
© Michael Crosby and Charles Sacker, 2001
Walk-through OutputsThe outputs of the walk-through
• Walk-through team members• Software product being evaluated• Statement of objectives and their status• Recommendations made regarding each
anomaly• List of actions, due-dates, responsible parties• Recommendations how to dispose of unresolved
anomalies• Any proposal for future walk-throughs
![Page 28: © Michael Crosby and Charles Sacker, 2001 Systematic Software Reviews Software reviews are a “quality improvement process for written material”](https://reader035.vdocuments.us/reader035/viewer/2022081514/5a4d1aec7f8b9ab05997ba7b/html5/thumbnails/28.jpg)
© Michael Crosby and Charles Sacker, 2001
AuditsThe purpose of an audit is to provide an independent evaluation of conformance of software products and processes to applicable;
Regulations Standards Guidelines Plans Procedures
![Page 29: © Michael Crosby and Charles Sacker, 2001 Systematic Software Reviews Software reviews are a “quality improvement process for written material”](https://reader035.vdocuments.us/reader035/viewer/2022081514/5a4d1aec7f8b9ab05997ba7b/html5/thumbnails/29.jpg)
© Michael Crosby and Charles Sacker, 2001
Inspection Process Most popular is the Fagan method
Inspection is separated into 5/6 phases
(Planning) Overview Preparation Inspection Meeting Rework Follow-up
![Page 30: © Michael Crosby and Charles Sacker, 2001 Systematic Software Reviews Software reviews are a “quality improvement process for written material”](https://reader035.vdocuments.us/reader035/viewer/2022081514/5a4d1aec7f8b9ab05997ba7b/html5/thumbnails/30.jpg)
© Michael Crosby and Charles Sacker, 2001
Inspection Materials Source Document Checklist Supporting Documents Invitation Master Plan Issue/Defect Log Data Summary
![Page 31: © Michael Crosby and Charles Sacker, 2001 Systematic Software Reviews Software reviews are a “quality improvement process for written material”](https://reader035.vdocuments.us/reader035/viewer/2022081514/5a4d1aec7f8b9ab05997ba7b/html5/thumbnails/31.jpg)
© Michael Crosby and Charles Sacker, 2001
Planning/Overview Inspectors are
selected Roles are assigned Documents are
distributed General inspection
task is discussed
![Page 32: © Michael Crosby and Charles Sacker, 2001 Systematic Software Reviews Software reviews are a “quality improvement process for written material”](https://reader035.vdocuments.us/reader035/viewer/2022081514/5a4d1aec7f8b9ab05997ba7b/html5/thumbnails/32.jpg)
© Michael Crosby and Charles Sacker, 2001
Inspection Roles
Roles for a Review
A u th or(C re a to r o f d o cu m en t[s ])
In s p ec to rs(R e v iew e rs o f d oc u m en t[s ])
S c ribe(R e c o rd e r)
L e ad er(M o d e ra to r)
![Page 33: © Michael Crosby and Charles Sacker, 2001 Systematic Software Reviews Software reviews are a “quality improvement process for written material”](https://reader035.vdocuments.us/reader035/viewer/2022081514/5a4d1aec7f8b9ab05997ba7b/html5/thumbnails/33.jpg)
© Michael Crosby and Charles Sacker, 2001
Roles: Leader Manages inspection Acts as moderator Determines document worthiness Identifies/invites inspectors Assigns roles Distributes documents Schedules meeting times/locations
![Page 34: © Michael Crosby and Charles Sacker, 2001 Systematic Software Reviews Software reviews are a “quality improvement process for written material”](https://reader035.vdocuments.us/reader035/viewer/2022081514/5a4d1aec7f8b9ab05997ba7b/html5/thumbnails/34.jpg)
© Michael Crosby and Charles Sacker, 2001
Roles: Author Creates the document for
inspection Assists with answering questions Typically not directly involved in
inspection Makes corrections to document if
necessary
![Page 35: © Michael Crosby and Charles Sacker, 2001 Systematic Software Reviews Software reviews are a “quality improvement process for written material”](https://reader035.vdocuments.us/reader035/viewer/2022081514/5a4d1aec7f8b9ab05997ba7b/html5/thumbnails/35.jpg)
© Michael Crosby and Charles Sacker, 2001
Roles: Inspector Complete familiarization of document on
time Inspect document(s) for defects Look for assigned defects (if appropriate) Make use of checklists or other supporting
documents Contact leader early if problems arise or if
the inspection might be a waste of time
![Page 36: © Michael Crosby and Charles Sacker, 2001 Systematic Software Reviews Software reviews are a “quality improvement process for written material”](https://reader035.vdocuments.us/reader035/viewer/2022081514/5a4d1aec7f8b9ab05997ba7b/html5/thumbnails/36.jpg)
© Michael Crosby and Charles Sacker, 2001
Roles: Scribe/Recorder Records issues as they are raised Ideally not the moderator or
inspector Record information legibly
![Page 37: © Michael Crosby and Charles Sacker, 2001 Systematic Software Reviews Software reviews are a “quality improvement process for written material”](https://reader035.vdocuments.us/reader035/viewer/2022081514/5a4d1aec7f8b9ab05997ba7b/html5/thumbnails/37.jpg)
© Michael Crosby and Charles Sacker, 2001
Preparation Inspectors
acquaint themselves with the documents to be inspected
Need to be familiar with material in time for inspection meeting
![Page 38: © Michael Crosby and Charles Sacker, 2001 Systematic Software Reviews Software reviews are a “quality improvement process for written material”](https://reader035.vdocuments.us/reader035/viewer/2022081514/5a4d1aec7f8b9ab05997ba7b/html5/thumbnails/38.jpg)
© Michael Crosby and Charles Sacker, 2001
Inspection Meeting Inspection team
attempts to locate defects
Defects are not fixed at this point
Meeting < 2 hours long!
![Page 39: © Michael Crosby and Charles Sacker, 2001 Systematic Software Reviews Software reviews are a “quality improvement process for written material”](https://reader035.vdocuments.us/reader035/viewer/2022081514/5a4d1aec7f8b9ab05997ba7b/html5/thumbnails/39.jpg)
© Michael Crosby and Charles Sacker, 2001
Inspection Meeting (cont.) Round-robin approach or Reader
approach Scribe records all issues
Where defect was located Why is it a defect (cite requirement or checklist) Suggested severity level (Major, minor) Do Not record names of inspectors with defect Try to make visible to all participants (avoid
duplication)
![Page 40: © Michael Crosby and Charles Sacker, 2001 Systematic Software Reviews Software reviews are a “quality improvement process for written material”](https://reader035.vdocuments.us/reader035/viewer/2022081514/5a4d1aec7f8b9ab05997ba7b/html5/thumbnails/40.jpg)
© Michael Crosby and Charles Sacker, 2001
Rework Author receives
defect log Identifies true
defects vs. “false positives”
Fixes defects, provides justification for false positive
![Page 41: © Michael Crosby and Charles Sacker, 2001 Systematic Software Reviews Software reviews are a “quality improvement process for written material”](https://reader035.vdocuments.us/reader035/viewer/2022081514/5a4d1aec7f8b9ab05997ba7b/html5/thumbnails/41.jpg)
© Michael Crosby and Charles Sacker, 2001
Follow-Up Leader verifies all
defects have been addressed
Decides if document passes inspection or if another inspection is necessary
![Page 42: © Michael Crosby and Charles Sacker, 2001 Systematic Software Reviews Software reviews are a “quality improvement process for written material”](https://reader035.vdocuments.us/reader035/viewer/2022081514/5a4d1aec7f8b9ab05997ba7b/html5/thumbnails/42.jpg)
© Michael Crosby and Charles Sacker, 2001
Inspection Process
Product Documents
Review Process
Planning/Ov erv iew Preparation Inspection Rework
Entry
Rev iewedDocuments
Exit
Product Documents Rules/Checklist
![Page 43: © Michael Crosby and Charles Sacker, 2001 Systematic Software Reviews Software reviews are a “quality improvement process for written material”](https://reader035.vdocuments.us/reader035/viewer/2022081514/5a4d1aec7f8b9ab05997ba7b/html5/thumbnails/43.jpg)
© Michael Crosby and Charles Sacker, 2001
Review Pitfalls Insufficient Preparation Moderator Domination Incorrect Review Rate Ego-involvement and Personality
Conflict Issue Resolution and Meeting Digression Recording Difficulties and Clerical
Overhead