SRIMCA

Software Quality Assurance - OutlineSoftware Quality Assurance - Outline

What is Software Quality assurance(SQA)?What is Software Quality assurance(SQA)? Quality Concepts.Quality Concepts. Software Quality Assurance Activities.Software Quality Assurance Activities. Software Reviews and their importanceSoftware Reviews and their importance Statistical SQA.Statistical SQA. Software ReliabilitySoftware Reliability ISO 9000 approach to SQAISO 9000 approach to SQA

SRIMCA

What is SQA?What is SQA?

Software Quality Assurance is an umbrella Software Quality Assurance is an umbrella activity that is applied throughout the activity that is applied throughout the software process...software process...

SRIMCA

It encompasses..It encompasses..

A quality management approachA quality management approach Effective software engineering technologyEffective software engineering technology Formal technical reviews that are applied Formal technical reviews that are applied

throughout the software processthroughout the software process A multitiered testing strategyA multitiered testing strategy Control of software documentation and changes Control of software documentation and changes

to itto it A procedure to assure compliance with software A procedure to assure compliance with software

development standardsdevelopment standards

Measurement and reporting techniquesMeasurement and reporting techniques

SRIMCA

Quality ???Quality ???

QualityQuality refers to any measurable refers to any measurable characteristics such as correctness, characteristics such as correctness, maintainability, portability, testability, maintainability, portability, testability, usability, reliability, efficiency, integrity, usability, reliability, efficiency, integrity, reusability and interoperability.reusability and interoperability.

Measures of program’s characteristics; as Measures of program’s characteristics; as cyclomatic complexity, cohesion, fp, loc etc.cyclomatic complexity, cohesion, fp, loc etc.

SRIMCA

Quality ConceptsQuality Concepts

Quality of DesignQuality of Design refers to the characteristics that refers to the characteristics that designer’s specify for an item.designer’s specify for an item.

Quality ControlQuality Control is the series of inspections, is the series of inspections, reviews and tests used throughout the reviews and tests used throughout the development cycle to ensure that each work development cycle to ensure that each work product meets the requirements placed upon it.product meets the requirements placed upon it.

Quality of ConformanceQuality of Conformance is the degree to which the is the degree to which the design specifications are followed during design specifications are followed during manufacturing.manufacturing.

SRIMCA

(cont'd)...(cont'd)...

Quality policyQuality policy refers to the basic aims and refers to the basic aims and objectives of an organization regarding quality as objectives of an organization regarding quality as stipulated by the management.stipulated by the management.

Quality assuranceQuality assurance consists of the auditing and consists of the auditing and reporting functions of management.reporting functions of management.

Cost of QualityCost of Quality provide a baseline for current cost provide a baseline for current cost of quality, identify opportunities for reducing the of quality, identify opportunities for reducing the cost of quality, and provide a normalized basis of cost of quality, and provide a normalized basis of comparision.comparision.

SRIMCA

(cont'd)...(cont'd)...

Quality Costs Quality Costs are divided into costs associated are divided into costs associated with prevention, appraisal, failure – internal & with prevention, appraisal, failure – internal & external costsexternal costs Prevention – Quality planning, formal technical Prevention – Quality planning, formal technical

reviews, test equipment, trainingreviews, test equipment, training Appraisal – gain insight into product condition Appraisal – gain insight into product condition

“first time through” each process; which include“first time through” each process; which include In-process and inter-process inspectionIn-process and inter-process inspection Equipment calibration and maintenanceEquipment calibration and maintenance TestingTesting

SRIMCA

(cont'd)...(cont'd)...

Failure Costs – those which disappear Failure Costs – those which disappear before shipping product to customerbefore shipping product to customerInternal – detection of defect prior to Internal – detection of defect prior to

shipmentshipmentRework, repair, failure mode analysisRework, repair, failure mode analysis

External – defect found after shipmentExternal – defect found after shipmentComplaint resolution, product return & Complaint resolution, product return &

replacement, help line support, warranty workreplacement, help line support, warranty work

SRIMCA

Relative cost of correcting an errorRelative cost of correcting an error

SRIMCA

Defn. of Software Quality AssuranceDefn. of Software Quality Assurance

Conformance to explicitly stated functional Conformance to explicitly stated functional and performance requirements, explicitly and performance requirements, explicitly documented development standards, and documented development standards, and implicit characteristics that are expected of implicit characteristics that are expected of all professionally developed software.all professionally developed software.

SRIMCA

Defn. of Software Quality AssuranceDefn. of Software Quality Assurance

1. S/W requirements are the foundation of quality; 1. S/W requirements are the foundation of quality; lack of conformance to requirements is lack of lack of conformance to requirements is lack of quality.quality.

2. Specified standards define a set of development 2. Specified standards define a set of development criteria that guide the teams in which s/w is criteria that guide the teams in which s/w is engineered; if not followed, lack of quality will engineered; if not followed, lack of quality will surely resultsurely result

3. A set of implicit requirements often goes 3. A set of implicit requirements often goes unmentioned; if not met, s/w quality is suspect.unmentioned; if not met, s/w quality is suspect.

SRIMCA

SQASQA

SQA is composed with tasks associated SQA is composed with tasks associated with:with:S/w Engineers who do technical workS/w Engineers who do technical workSQA group responsible for quality assurance SQA group responsible for quality assurance

planning, oversight, record-keeping, planning, oversight, record-keeping, analysis and reporting – to assist the s/w analysis and reporting – to assist the s/w team in achieving a high quality end team in achieving a high quality end product.product.

SRIMCA

SQA Group PlanSQA Group Plan

Evaluations to be performedEvaluations to be performed Audits and reviews to be performed Audits and reviews to be performed Standards that are applicable to the project Standards that are applicable to the project Procedures for error reporting and trackingProcedures for error reporting and tracking Documents to be produced by the SQA groupDocuments to be produced by the SQA group Amount of feedback provided to software Amount of feedback provided to software

project teamproject team

SQASQA group perform following group perform following activitiesactivities::

SRIMCA

SQA Group ActivitiesSQA Group Activities

Participates in the development of the Participates in the development of the projects software process descriptionprojects software process description

Reviews software engineering activities to Reviews software engineering activities to verify compliance with the defined software verify compliance with the defined software process.process.

Audits designated software work products Audits designated software work products to verify compliance with those defined as to verify compliance with those defined as part of the software process.part of the software process.

SRIMCA

(cont'd)...(cont'd)...

Ensures that deviations in software work Ensures that deviations in software work and work products are documented and and work products are documented and handled according to a document handled according to a document procedure.procedure.

Records any non-compliance and reports to Records any non-compliance and reports to senior management.senior management.

In addition to these, SQA group may co-In addition to these, SQA group may co-ordinate the control and management of ordinate the control and management of change, and helps to collect and analyze s/w change, and helps to collect and analyze s/w metrics.metrics.

SRIMCA

Software ReviewsSoftware Reviews

‘‘Filter’ for the software engineering processFilter’ for the software engineering process ‘‘Purify’ the software work products that Purify’ the software work products that

occur as a result of analysis, design, and occur as a result of analysis, design, and coding.coding.

Achieve technical work of more uniform, Achieve technical work of more uniform, greater and more predictable quality.greater and more predictable quality.

Detect errors and problems at the earliest Detect errors and problems at the earliest possible time.possible time.

SRIMCA

Formal Technical ReviewsFormal Technical Reviews

To uncover errors in function, logic, or To uncover errors in function, logic, or implementation for any representation of the implementation for any representation of the softwaresoftware

To verify that software meets its requirementsTo verify that software meets its requirements To ensure that software representation meets To ensure that software representation meets

predefined standardspredefined standards To achieve software development in a uniform To achieve software development in a uniform

mannermanner To make projects more manageableTo make projects more manageable

SRIMCA

Cost Impact of Software DefectsCost Impact of Software Defects

Defect = Fault (We knew it as Defect = Fault (We knew it as error error before delivery)before delivery) Industry studies reveal that almost 50-65 % of all errors Industry studies reveal that almost 50-65 % of all errors

(defects) are introduced during design activities(defects) are introduced during design activities By detecting and removing them, review process By detecting and removing them, review process

substantially reduces the cost of subsequent steps in the substantially reduces the cost of subsequent steps in the development and support phases.development and support phases.

E.g. assume that an error uncovered during design will cost E.g. assume that an error uncovered during design will cost 1 Rs., relative to this, the same error uncovered just before 1 Rs., relative to this, the same error uncovered just before testing commences will be 6.5 Rs., during testing, 15 Rs. testing commences will be 6.5 Rs., during testing, 15 Rs. And after release 60-100 Rs.And after release 60-100 Rs.

SRIMCA

Defect Amplification ModelDefect Amplification Model

Errors passed throughErrors passed through Percent Percent efficiency for efficiency for

error error detectiondetection

Amplified errors 1 : xAmplified errors 1 : x

Newly generated errorsNewly generated errors

Development StepDevelopment Step

Errors Errors from from previous previous stepstep

Errors Errors passed passed to next to next stepstep

DefectsDefects DetectionDetection

SRIMCA

Defect Amplification ModelDefect Amplification Model

SRIMCA

Defect Amplification with ReviewsDefect Amplification with Reviews

SRIMCA

Cost Comparison of Error RepairCost Comparison of Error Repair

SRIMCA

Review Guidelines..Review Guidelines..

Review the product, not Review the product, not producerproducer

Set an agenda and maintain Set an agenda and maintain itit

Limit the debate Limit the debate Enunciate problem areas, Enunciate problem areas,

not to solve every problem not to solve every problem notednoted

Take written notesTake written notes Allocate resources and Allocate resources and

time schedule for FTR’stime schedule for FTR’s

Limit the number of Limit the number of participants and insist participants and insist upon advance preparationupon advance preparation

Develop a checklist for Develop a checklist for each work product to be each work product to be reviewedreviewed

Training for all Training for all reviewer’sreviewer’s

Reviewing earlier reviewsReviewing earlier reviews

SRIMCA

Statistical Quality AssuranceStatistical Quality Assurance

Implies information about software defects is Implies information about software defects is collected and categorized collected and categorized

An attempt is made to trace each defect to its An attempt is made to trace each defect to its underlying causeunderlying cause

Isolate the vital few causes of the major Isolate the vital few causes of the major source of all errorssource of all errors

Then move to correct the problems that have Then move to correct the problems that have caused the defectscaused the defects

SRIMCA

Categories of ErrorsCategories of Errors

Incomplete or erroneous specification (IES)Incomplete or erroneous specification (IES) Misinterpretation of customer comm (MCC)Misinterpretation of customer comm (MCC) Intentional deviation from specification (IDS)Intentional deviation from specification (IDS) Violation of programming standards (VPS)Violation of programming standards (VPS) Error in data representation (EDR)Error in data representation (EDR) Inconsistent module interface (IMI)Inconsistent module interface (IMI) Error in design logic (EDL)Error in design logic (EDL)

SRIMCA

Categories of Errors (cont'd)Categories of Errors (cont'd)

Incomplete or erroneous testing (IET)Incomplete or erroneous testing (IET) Inaccurate or incomplete documentation (IID)Inaccurate or incomplete documentation (IID) Error in programming lang. Translation (PLT)Error in programming lang. Translation (PLT) Ambiguous or inconsistent human-computer Ambiguous or inconsistent human-computer

interface (HCI)interface (HCI) Miscellaneous (MIS)Miscellaneous (MIS) Most often IES, MCC and EDR are the vital few Most often IES, MCC and EDR are the vital few

causes for majority of errors.causes for majority of errors.

SRIMCA

DefinitionsDefinitions

EEi i = = the total number of errors uncovered the total number of errors uncovered

during the iduring the ith th step in the software engineering step in the software engineering processprocess

SSii = the number of serious errors = the number of serious errors

MMii = the number of moderate errors = the number of moderate errors

TTii = the number of minor errors = the number of minor errors

PS = size of the product (LOC, design PS = size of the product (LOC, design statements, pages of documentation)statements, pages of documentation)

SRIMCA

error indexerror index

Phase index for each step and then error Phase index for each step and then error index is calculatedindex is calculated

PIPIii = w = wss(S(Sii/E/Eii)+w)+wmm(M(Mii/E/Eii)+w)+wtt(T(Tii/E/Eii))

Formula:Formula:

( ) /

( ) /

i PI PS

PI PI PI iPI PS

X i

i

1 2 32 3

SRIMCA

Software ReliabilitySoftware Reliability

Defined as the probability of failure free operation Defined as the probability of failure free operation of a computer program in a specified environment of a computer program in a specified environment for a specified time.for a specified time.

It can measured, directed and estimated It can measured, directed and estimated A measure of software reliability is A measure of software reliability is mean time mean time

between failuresbetween failures where where MTBF = MTTF + MTTRMTBF = MTTF + MTTR MTTF = MTTF = mean time to failuremean time to failure MTTR = MTTR = mean time to repairmean time to repair

SRIMCA

Software AvailabilitySoftware Availability

Availability =MTTF/(MTTF + MTTR) * 100%Availability =MTTF/(MTTF + MTTR) * 100% Software availabilitySoftware availability is the probability that a is the probability that a

program is operating according to requirements at program is operating according to requirements at a given point in time a given point in time

SRIMCA

Software SafetySoftware Safety

Processes that help reduce the probability that Processes that help reduce the probability that critical failures will occur due to SWcritical failures will occur due to SW

Hazard analysesHazard analyses Identify hazards that could call failureIdentify hazards that could call failure Develop fault treeDevelop fault tree Identify all possible causes of the hazardIdentify all possible causes of the hazard Formally review the remedy for eachFormally review the remedy for each

RedundancyRedundancy Require a written software safety planRequire a written software safety plan Require independent verification & validationRequire independent verification & validation

SRIMCA

Example Fault Tree -- ThermalExample Fault Tree -- Thermal

Loss of heatLoss of heat

Power failurePower failure Computer failureComputer failure IncorrectIncorrect

inputinput

SW failed SW failed to throw to throw switchswitch

......

Computer failureComputer failure SW failed SW failed to throw to throw switchswitch

......Logic reversedLogic reversed

SRIMCA

Software SafetySoftware Safety

RedundancyRedundancy Replicated at the hardware levelReplicated at the hardware level Similar vs.. dis-similar redundancySimilar vs.. dis-similar redundancy

VerificationVerification Assuring that the software specifications are metAssuring that the software specifications are met

ValidationValidation Assuring that the product functions as desiredAssuring that the product functions as desired

IndependenceIndependence

SRIMCA

Overview of SQA PlanOverview of SQA Plan

Purpose of PlanPurpose of Plan ReferencesReferences Management Management DocumentationDocumentation Standards, Practices and Standards, Practices and

ConventionsConventions Reviews and AuditsReviews and Audits TestTest Problem Reporting and Problem Reporting and

Corrective actionCorrective action

Tools, Techniques and Tools, Techniques and MethodologiesMethodologies

Code ControlCode Control Media ControlMedia Control Supplier controlSupplier control Records Collection, Records Collection,

Maintenance and Maintenance and RetentionRetention

Training Training Risk ManagementRisk Management

SRIMCA

ISO 9000 Quality StandardsISO 9000 Quality Standards

ISO 9000 describes quality assurance elements in ISO 9000 describes quality assurance elements in generic terms that can be applied to any business.generic terms that can be applied to any business.

It treats an enterprise as a network of It treats an enterprise as a network of interconnected processes.interconnected processes.

To be ISO-complaint processes should adhere to To be ISO-complaint processes should adhere to the standards described.the standards described.

Elements include organizational structure, Elements include organizational structure,

procedures, processes and resources.procedures, processes and resources. Ensures quality planning, quality control, quality Ensures quality planning, quality control, quality

assurance and quality improvement. assurance and quality improvement.

SRIMCA

ISO 9001ISO 9001

An international standard which provides An international standard which provides broad guidance to software developers on broad guidance to software developers on how to Implement, maintain and improve a how to Implement, maintain and improve a quality software system capable of ensuring quality software system capable of ensuring high quality softwarehigh quality software

Consists of 20 requirements...Consists of 20 requirements... Differs from country to country..Differs from country to country..

SRIMCA

ISO 9001 (cont'd)..requirementsISO 9001 (cont'd)..requirements

Management Management responsibilityresponsibility

Quality systemQuality system Contract reviewContract review Design ControlDesign Control Document and data Document and data

controlcontrol PurchasingPurchasing

Control of customer Control of customer supplied productsupplied product

Product identification Product identification and traceabilityand traceability

Process controlProcess control Inspection and testingInspection and testing Control of inspection, Control of inspection,

measuring and test measuring and test equipmentequipment

SRIMCA

ISO 9001 (cont'd)..ISO 9001 (cont'd)..

Inspection and test Inspection and test statusstatus

Control of non-Control of non-confirming productconfirming product

Corrective and Corrective and preventive actionpreventive action

Handling, storage, Handling, storage, packaging, preservation packaging, preservation and deliveryand delivery

Control of quality Control of quality recordsrecords

Internal quality auditsInternal quality audits TrainingTraining ServicingServicing Statistical techniquesStatistical techniques

SRIMCA

Summary-Summary-

SQA must be applied at each stepSQA must be applied at each step SQA might be complexSQA might be complex Software reviews are important SQA activitiesSoftware reviews are important SQA activities Statistical SQA helps improve product quality Statistical SQA helps improve product quality

and software processand software process Software Safety is essential for critical systems Software Safety is essential for critical systems ISO 9001 standardizes the SQA activitiesISO 9001 standardizes the SQA activities