lecture 16 formal technical reviews (ftrs) (also know as inspections) for0383 software quality...

Lecture 16Formal Technical Reviews (FTRs)

(also know as inspections)

FOR0383 Software Quality Assurance

04/19/23 1Dr Andy Brooks

Don´t review in your spare time. Don´t let people get angry at the review meeting.

Source: Software Formal Inspections Guidebook NASA Office of Safety and Mssion Guidance. NASA-GB-A302 1993. http://satc.gsfc.nasa.gov/Documents/fi/gdb/fitext.txt http://www.gpa.etsmtl.ca/cours/gpa777/pdf/fi.pdf

19.04.23 Dr Andy Brooks 2

A definition of an inspection.

A static analysis technique that relies on visual examination of work products to detect defects, violations of development standards, and other problems.

“... there is a fault here in Use Case 15...”


A definition of an inspection.

An inspection is a formal review of a work product by the work product owner and a team of peers looking for errors, omissions, inconsistencies, and areas of confusion in the work product.

work productsSoftware Requirement Specifications, use cases, UML class/state/sequence/activity/component diagrams,code, test cases,...

Errors of omission can be difficult to detect...

úrfelling


Roles Author(s)

Person or persons primarily responsible for creating a work product. The member of the inspection team that provides information about the work product during all stages of the inspection process and corrects defects during the rework stage. (Also known as (AKA) "Owner(s)".)

Inspection Team A small group of peers who have a vested interest in the quality of the inspected work product and perform the inspection. This group usually ranges in size from 3 to 8 people and can be selected from various areas of the development life cycle (requirements, design, implementation, testing, quality assurance, user, etc.). Selected members of the inspection team fulfill the roles of moderator, author, reader, and recorder.

Inspector A person whose responsibilities include reviewing work products created by others. All members should be considered inspectors in an inspection team.


RolesModerator

Person who is primarily responsible for facilitating and coordinating an inspection. When there is no "Reader" in the inspection process, the moderator also controls the pace of review of the work product during the inspection meeting.

Reader Person who guides the team during the inspection meeting by reading or paraphrasing the work product. The role of the reader is usually fulfilled by a member of the inspection team other than the author(s). All inspection methods don't use this role.

Recorder Person who records, in writing, each defect found and its related information (location, severity, type, description) during the inspection meeting. (AKA "Scribe".)


Do not criticise team members...

• Inspections are used to judge the quality of the work product,not the author of that work product.

• Managers should not participate in inspections.• Results should be presented to managers

statistically.• If authors are criticised, they begin to find

excuses for not participating.• Inspectors can become reluctant to formally

identify defects if they too are criticised.

19/04/23 Dr Andy Brooks 7

Shut up. This is not a fault. You are just stupid.

Sit down. We are here to discuss things in a reasonable manner.

This inspection meeting is not going very well.

This is obviously a null pointer dereference!

manual inspection meeting

Such a poor meeting might not even report 20% of the defects...

poor meeting/lélegur fundur


Stages1. Planning The period of time used to determine whether the work product

to be inspected meets the entry criteria, plan the inspection, select the team, assign roles, schedule the inspection meeting, and prepare and distribute the inspection forms and materials. In addition, a determination is made on whether to hold an overview.

If necessary the moderator divides up the work product into more manageable pieces.

Inspection forms and materials include: the inspection announcement, the work product to be inspected, individual preparation logs, checklists, and inspection report form.


Stages

2. Overview The optional stage in which the inspection team is provided with

background information for the inspection. This stage may not be necessary if the team is already familiar with the work product being inspected.

Overviews should be carried out if: • the inspection team is not familiar with the product• the work product is new or being inspected for the first time• inspections are new to the project• if novel techniques have been used in the work product.


Stages3. Preparation Key stage in which members of the inspection team prepare

individually for the inspection by reviewing and finding potential defects in the work product being inspected. Potential defects found by individuals are discussed during the actual inspection meeting.

Ad-hoc preparation is where inspectors simply use their own expertise to find and report potential defects. Focussed preparation may involve the use of special checklists or scenarios as inspection aids.

Potential defects are recorded on individual preparation logs which should also record the time spent by each individual inspector.

Ideally the moderator examines the individual preparation logs to check that everyone has prepared properly for the inspection meeting.


Stages4. Inspection Meeting Meeting where team members, as a group, review the work product to

find, categorize, and record, but not resolve, defects.

Moderator performs introductions and reminds everyone of roles. Reader begins a logical and orderly interpretation of the work product.

Inspectors can interrupt at any time to report a potential defect.

If consensus cannot be reached on whether or not a raised issue is a defect, an open issue should be recorded.

Reading is resumed once the recorder has noted the potential defect (location, severity, type, description).

Answers to questions: “Do we need a third hour?” “Do you we need to reinspect?”


Inspection meeting failures

• Moderator dominates inspection– by talking too much or interrupting others

• Reader is too fast– by summarising too many paragraphs, diagrams, or lines of

code at once

• Author is under attack– time spent making unnecessary comments directly at the author

• Author begins problem solving– time spent by the author verbalising possible solutions

• Recorder is too slow– inadequate preparation might mean the recorder does not

understand the problem to be recorded


Stages5. Third Hour Optional additional time, apart from the inspection meeting, that can

be used for discussion, possible solutions, or closure of open issues raised in the inspection meeting.

6. Rework Stage when the author corrects major defects found during the

inspection. Minor defects are dealt with if the cost and schedule permit.

Author receives a copy of the inspection report form and any relevant materials from the third hour.

7. Follow-up Short meeting between the moderator and author to determine if

defects found during the inspection meeting have been corrected and to ensure that no additional defects have been introduced.


Examples of entry criteria

• Clean compilation of code:– no errors or warnings

• All automatic tool checks have been performed:– static analysers– spelling and grammar checkers– CASE (Computer Aided Software Engineering) tools


Examples of exit criteria

• All major defects have been corrected.

• All open issues have been resolved.


Possible classifications of defect severityMajor

A defect, which if not identified and removed in a work product, could result in a test or customer/field reported problem.

MinorAll other defects.

Critical A defect that might cause the system to crash or hang.

MajorA defect that could result in incorrect behaviour or results.

MinorDefects that can be overlooked with no loss of system functionality.


One classification of code defect types

A. Variable naming standardsB. Code layout standardsC. Code documentation standardsD. Missing or incorrect exception handlingE. Misuse of variable typesF. Unreachable codeG. Inverted predicatesH. Incomplete predicatesI. Method lacking cohesionJ. Improper sequencing of methodsK. Incorrect access of array componentsL. Variables with unnecessary scopeM. Infinite loopN. Initialization faultO. Inefficient codeP. Failure to implement requirement

If... then... else...


Inspection rates: 1 page/hour?

• How many lines of code should be inspected assuming 90 minutes for individual preparation and 120 minutes for the inspection meeting?

• The faster the inspection rate (pages/hour) the smaller the defect density (defects/page). It is better to go slow and not try to do too much at once. Some argue for an inspection rate of 1 page/hour.

• Your teacher has carried out documentation reviews in industry (biotechnology software) and found it difficult to go faster than 1 page/hour (in individual preparation).