2007 2 software quality assurance - outline ä what is software quality assurance(sqa)? ä quality...
Post on 21-Dec-2015
237 views
TRANSCRIPT
2007 2
Software Quality Assurance - OutlineSoftware Quality Assurance - Outline
What is Software Quality assurance(SQA)?What is Software Quality assurance(SQA)? Quality Concepts.Quality Concepts. Software Quality Assurance Activities.Software Quality Assurance Activities. Software Reviews and their importanceSoftware Reviews and their importance Statistical SQA.Statistical SQA. Software ReliabilitySoftware Reliability ISO 9000 approach to SQAISO 9000 approach to SQA
2007 3
What is SQA?What is SQA?
Software Quality Assurance is an umbrella Software Quality Assurance is an umbrella activity that is applied throughout the activity that is applied throughout the software process...software process...
2007 4
It encompasses..It encompasses..
A quality management approachA quality management approach Effective software engineering technologyEffective software engineering technology Formal technical reviews that are applied Formal technical reviews that are applied
throughout the software processthroughout the software process A multitiered testing strategyA multitiered testing strategy Control of software documentation and changes Control of software documentation and changes
to itto it A procedure to assure compliance with software A procedure to assure compliance with software
development standardsdevelopment standards
Measurement and reporting techniquesMeasurement and reporting techniques
2007 5
Quality ???Quality ???
QualityQuality refers to any measurable refers to any measurable characteristics such as correctness, characteristics such as correctness, maintainability, portability, testability, maintainability, portability, testability, usability, reliability, efficiency, integrity, usability, reliability, efficiency, integrity, reusability and interoperability.reusability and interoperability.
2007 6
Quality ConceptsQuality Concepts
Quality of DesignQuality of Design refers to the characteristics that refers to the characteristics that designer’s specify for an item.designer’s specify for an item.
Quality ControlQuality Control is the series of inspections, is the series of inspections, reviews and tests used throughout the reviews and tests used throughout the development cycle to ensure that each work development cycle to ensure that each work product meets the requirements placed upon it.product meets the requirements placed upon it.
Quality of ConformanceQuality of Conformance is the degree to which the is the degree to which the design specifications are followed during design specifications are followed during manufacturing.manufacturing.
2007 7
(cont'd)...(cont'd)...
Quality policyQuality policy refers to the basic aims and refers to the basic aims and objectives of an organization regarding quality as objectives of an organization regarding quality as stipulated by the management.stipulated by the management.
Quality assuranceQuality assurance consists of the auditing and consists of the auditing and reporting functions of management.reporting functions of management.
Cost of QualityCost of Quality includes all costs incurred in the includes all costs incurred in the pursuit of quality or in performing quality related pursuit of quality or in performing quality related activities such as appraisal costs, failure costs and activities such as appraisal costs, failure costs and external failure costs.external failure costs.
20078
(cont'd)...(cont'd)...
Quality planningQuality planning is the process of assessing the is the process of assessing the requirements of the procedure and of the product requirements of the procedure and of the product and the context in which these must be observed.and the context in which these must be observed.
Quality testingQuality testing is assessment of the extent to which is assessment of the extent to which a test object meets given requirementsa test object meets given requirements
Quality assurance planQuality assurance plan is the central aid for is the central aid for planning and checking the quality assurance.planning and checking the quality assurance.
Quality assurance systemQuality assurance system is the organizational is the organizational structure, responsibilities, procedures, processes and structure, responsibilities, procedures, processes and resources for implementing quality management.resources for implementing quality management.
2007 9
Relative cost of correcting an errorRelative cost of correcting an error
2007 10
Defn. of Software Quality AssuranceDefn. of Software Quality Assurance
Conformance to explicitly stated functional Conformance to explicitly stated functional and performance requirements, explicitly and performance requirements, explicitly documented development standards, and documented development standards, and implicit characteristics that are expected of implicit characteristics that are expected of all professionally developed software.all professionally developed software.
2007
Cost of QualityCost of Quality
Prevention costsPrevention costs include include quality planningquality planning formal technical reviewsformal technical reviews test equipmenttest equipment TrainingTraining
Internal failure costsInternal failure costs include include reworkrework repairrepair failure mode analysisfailure mode analysis
External failure costsExternal failure costs are are complaint resolutioncomplaint resolution product return and replacementproduct return and replacement help line supporthelp line support warranty workwarranty work
2007 11
SQA Group PlanSQA Group Plan
Evaluations to be performedEvaluations to be performed Audits and reviews to be performed Audits and reviews to be performed Standards that are applicable to the project Standards that are applicable to the project Procedures for error reporting and trackingProcedures for error reporting and tracking Documents to be produced by the SQA groupDocuments to be produced by the SQA group Amount of feedback provided to software Amount of feedback provided to software
project teamproject team
2007 12
SQA Group ActivitiesSQA Group Activities
Participates in the development of the Participates in the development of the projects software process descriptionprojects software process description
Reviews software engineering activities to Reviews software engineering activities to verify compliance with the defined software verify compliance with the defined software process.process.
Audits designated software work products Audits designated software work products to verify compliance with those defined as to verify compliance with those defined as part of the software process.part of the software process.
2007 13
(cont'd)...(cont'd)...
Ensures that deviations in software work Ensures that deviations in software work and work products are documented and and work products are documented and handled according to a document handled according to a document procedure.procedure.
Records any non-compliance and reports to Records any non-compliance and reports to senior management.senior management.
2007 14
Software ReviewsSoftware Reviews
‘‘Filter’ for the software engineering processFilter’ for the software engineering process ‘‘Purify’ the software work products that Purify’ the software work products that
occur as a result of analysis, design, and occur as a result of analysis, design, and coding.coding.
Achieve technical work of more uniform, Achieve technical work of more uniform, greater and more predictable quality.greater and more predictable quality.
Detect errors and problems at the earliest Detect errors and problems at the earliest possible time.possible time.
2007 15
Formal Technical ReviewsFormal Technical Reviews
To uncover errors in function, logic, or To uncover errors in function, logic, or implementation for any representation of the implementation for any representation of the softwaresoftware
To verify that software meets its requirementsTo verify that software meets its requirements To ensure that software representation meets To ensure that software representation meets
predefined standardspredefined standards To achieve software development in a uniform To achieve software development in a uniform
mannermanner To make projects more manageableTo make projects more manageable
2007
The PlayersThe Players
reviewreviewleaderleader
producerproducer
recorderrecorder reviewerreviewer
standards bearer (SQA)standards bearer (SQA)
maintenance maintenance oracleoracle
user repuser rep
2007
Conducting the ReviewConducting the Review
be prepared—evaluate be prepared—evaluate product before the reviewproduct before the review
review the product, not review the product, not the producerthe producer
keep your tone mild, ask keep your tone mild, ask questions instead of questions instead of making accusationsmaking accusations
1.1.
2.2.
3.3.
stick to the review agendastick to the review agenda
raise issues, don't resolve themraise issues, don't resolve them
avoid discussions of style—stick to technical avoid discussions of style—stick to technical correctnesscorrectness
schedule reviews as project tasksschedule reviews as project tasks
record and report all review resultsrecord and report all review results
4.4.
5.5.
6.6.
7.7.
8.8.
2007
trained leadertrained leader agenda establishedagenda established reviewers prepare in advancereviewers prepare in advance producer presents productproducer presents product ““reader” presents productreader” presents product recorder takes notesrecorder takes notes checklists used to find errorschecklists used to find errors errors categorized as founderrors categorized as found issues list createdissues list created team must sign-off on resultteam must sign-off on result
IPR—informal peer review WT—WalkthroughIPR—informal peer review WT—Walkthrough IN—Inspection RRR—round robin reviewIN—Inspection RRR—round robin review
Review Options MatrixReview Options Matrix
IPRIPR WTWT ININ RRRRRR
nonomaybemaybemaybemaybemaybemaybenonomaybemaybenononononononono
yesyesyesyesyesyesyesyesnonoyesyesnonononoyesyesyesyes
yesyesyesyesyesyesnonoyesyesyesyesyesyesyesyesyesyesyesyes
yesyesyesyesyesyesnonononoyesyesnonononoyesyesmaybemaybe
**
2007
Metrics Derived from ReviewsMetrics Derived from Reviews
inspection time per page of documentationinspection time per page of documentationinspection time per KLOC or FPinspection time per KLOC or FP
errors uncovered per reviewer hourerrors uncovered per reviewer hourerrors uncovered per preparation hourerrors uncovered per preparation hourerrors uncovered per SE task (e.g., design)errors uncovered per SE task (e.g., design)number of minor errors (e.g., typos)number of minor errors (e.g., typos)
number of errors found during preparationnumber of errors found during preparation
number of major errorsnumber of major errors (e.g., nonconformance to req.) (e.g., nonconformance to req.)
inspection effort per KLOC or FPinspection effort per KLOC or FP
2007
Defect Amplification ModelDefect Amplification Model
2007
Defect Amplification with ReviewsDefect Amplification with Reviews
2007
Cost Comparison of Error RepairCost Comparison of Error Repair
2007 16
Review Guidelines..Review Guidelines..
Review the product, not Review the product, not producerproducer
Set an agenda and maintain Set an agenda and maintain itit
Limit the debate Limit the debate Enunciate problem areas, Enunciate problem areas,
not to solve every problem not to solve every problem notednoted
Take written notesTake written notes Allocate resources and Allocate resources and
time schedule for FTR’stime schedule for FTR’s
Limit the number of Limit the number of participants and insist participants and insist upon advance preparationupon advance preparation
Develop a checklist for Develop a checklist for each work product to be each work product to be reviewedreviewed
Training for all Training for all reviewer’sreviewer’s
Reviewing earlier reviewsReviewing earlier reviews
2007
Additional StructuresAdditional Structures
Requirements Control BoardRequirements Control Board All requirement changes must be formally All requirement changes must be formally
reviewed and approvedreviewed and approved
Software Control BoardSoftware Control Board All design changes must be formally reviewed All design changes must be formally reviewed
and approvedand approved
Interface Control BoardInterface Control Board
2007
Statistical SQAStatistical SQA
ProductProduct& Process& Process
measurement
... an understanding of how to improve quality ...
Collect information on all defectsFind the causes of the defectsMove to provide fixes for the process
2007 17
Statistical Quality AssuranceStatistical Quality Assurance
Implies information about software defects is Implies information about software defects is collected and categorized collected and categorized
An attempt is made to trace each defect to its An attempt is made to trace each defect to its underlying causeunderlying cause
Isolate the vital few causes of the major Isolate the vital few causes of the major source of all errorssource of all errors
Then move to correct the problems that have Then move to correct the problems that have caused the defectscaused the defects
2007
Categories of ErrorsCategories of Errors
Incomplete or erroneous specification (IES)Incomplete or erroneous specification (IES) Misinterpretation of customer comm (MCC)Misinterpretation of customer comm (MCC) Intentional deviation from specification (IDS)Intentional deviation from specification (IDS) Violation of programming standards (VPS)Violation of programming standards (VPS) Error in data representation (EDR)Error in data representation (EDR) Inconsistent module interface (IMI)Inconsistent module interface (IMI) Error in design logic (EDL)Error in design logic (EDL)
2007
Categories of Errors (cont'd)Categories of Errors (cont'd)
Incomplete or erroneous testing (IET)Incomplete or erroneous testing (IET) Inaccurate or incomplete documentation (IID)Inaccurate or incomplete documentation (IID) Error in programming lang. Translation (PLT)Error in programming lang. Translation (PLT) Ambiguous or inconsistent human-computer Ambiguous or inconsistent human-computer
interface (HCI)interface (HCI) Miscellaneous (MIS)Miscellaneous (MIS) Most often IES, MCC and EDR are the vital few Most often IES, MCC and EDR are the vital few
causes for majority of errors.causes for majority of errors.
2007 18
DefinitionsDefinitions
EEi i = = the total number of errors uncovered the total number of errors uncovered
during the iduring the ith th step in the software engineering step in the software engineering processprocess
SSii = the number of serious errors = the number of serious errors
MMii = the number of moderate errors = the number of moderate errors
TTii = the number of minor errors = the number of minor errors
PS = size of the product (LOC, design PS = size of the product (LOC, design statements, pages of documentation)statements, pages of documentation)
2007 19
error indexerror index
Phase index for each step and then error Phase index for each step and then error index is calculatedindex is calculated
PIPIii = w = wss(S(Sii/E/Eii)+w)+wmm(M(Mii/E/Eii)+w)+wtt(T(Tii/E/Eii))
Formula:Formula:
( ) /
( ) /
i PI PS
PI PI PI iPI PS
X i
i
1 2 32 3
2007 20
Software ReliabilitySoftware Reliability
Defined as the probability of failure free operation Defined as the probability of failure free operation of a computer program in a specified environment of a computer program in a specified environment for a specified time.for a specified time.
It can measured, directed and estimated It can measured, directed and estimated A measure of software reliability is A measure of software reliability is mean time mean time
between failuresbetween failures where where MTBF = MTTF + MTTRMTBF = MTTF + MTTR MTTF = MTTF = mean time to failuremean time to failure MTTR = MTTR = mean time to repairmean time to repair
2007 21
Software AvailabilitySoftware Availability
Availability =MTTF/(MTTF + MTTR) * 100%Availability =MTTF/(MTTF + MTTR) * 100% Software availabilitySoftware availability is the probability that a is the probability that a
program is operating according to requirements at program is operating according to requirements at a given point in time a given point in time
2007
Software SafetySoftware Safety
Processes that help reduce the probability that Processes that help reduce the probability that critical failures will occur due to SWcritical failures will occur due to SW
Hazard analysesHazard analyses Identify hazards that could call failureIdentify hazards that could call failure Develop fault treeDevelop fault tree Identify all possible causes of the hazardIdentify all possible causes of the hazard Formally review the remedy for eachFormally review the remedy for each
RedundancyRedundancy Require a written software safety planRequire a written software safety plan Require independent verification & validationRequire independent verification & validation
2007
Example Fault Tree -- ThermalExample Fault Tree -- Thermal
Loss of heatLoss of heat
Power failurePower failure Computer failureComputer failure IncorrectIncorrect
inputinput
SW failed SW failed to throw to throw switchswitch
......
Computer failureComputer failure SW failed SW failed to throw to throw switchswitch
......Logic reversedLogic reversed
2007
Software SafetySoftware Safety
RedundancyRedundancy Replicated at the hardware levelReplicated at the hardware level Similar vs.. dis-similar redundancySimilar vs.. dis-similar redundancy
VerificationVerification Assuring that the software specifications are metAssuring that the software specifications are met
ValidationValidation Assuring that the product functions as desiredAssuring that the product functions as desired
IndependenceIndependence
2007 22
Overview of SQA PlanOverview of SQA Plan
Purpose of PlanPurpose of Plan ReferencesReferences Management Management DocumentationDocumentation Standards, Practices and Standards, Practices and
ConventionsConventions Reviews and AuditsReviews and Audits TestTest Problem Reporting and Problem Reporting and
Corrective actionCorrective action
Tools, Techniques and Tools, Techniques and MethodologiesMethodologies
Code ControlCode Control Media ControlMedia Control Supplier controlSupplier control Records Collection, Records Collection,
Maintenance and Maintenance and RetentionRetention
Training Training Risk ManagementRisk Management
2007 23
ISO 9000 Quality StandardsISO 9000 Quality Standards
ISO 9000 describes quality assurance elements in ISO 9000 describes quality assurance elements in generic terms that can be applied to any business.generic terms that can be applied to any business.
It treats an enterprise as a network of It treats an enterprise as a network of interconnected processes.interconnected processes.
To be ISO-complaint processes should adhere to To be ISO-complaint processes should adhere to the standards described.the standards described.
Elements include organizational structure, Elements include organizational structure,
procedures, processes and resources.procedures, processes and resources. Ensures quality planning, quality control, quality Ensures quality planning, quality control, quality
assurance and quality improvement. assurance and quality improvement.
2007 24
ISO 9001ISO 9001
An international standard which provides An international standard which provides broad guidance to software developers on broad guidance to software developers on how to Implement, maintain and improve a how to Implement, maintain and improve a quality software system capable of ensuring quality software system capable of ensuring high quality softwarehigh quality software
Consists of 20 requirements...Consists of 20 requirements... Differs from country to country..Differs from country to country..
2007 25
ISO 9001 (cont'd)..requirementsISO 9001 (cont'd)..requirements
Management Management responsibilityresponsibility
Quality systemQuality system Contract reviewContract review Design ControlDesign Control Document and data Document and data
controlcontrol PurchasingPurchasing
Control of customer Control of customer supplied productsupplied product
Product identification Product identification and traceabilityand traceability
Process controlProcess control Inspection and testingInspection and testing Control of inspection, Control of inspection,
measuring and test measuring and test equipmentequipment
2007 26
ISO 9001 (cont'd)..ISO 9001 (cont'd)..
Inspection and test Inspection and test statusstatus
Control of non-Control of non-confirming productconfirming product
Corrective and Corrective and preventive actionpreventive action
Handling, storage, Handling, storage, packaging, preservation packaging, preservation and deliveryand delivery
Control of quality Control of quality recordsrecords
Internal quality auditsInternal quality audits TrainingTraining ServicingServicing Statistical techniquesStatistical techniques
2007 27
Summary-Summary-
SQA must be applied at each stepSQA must be applied at each step SQA might be complexSQA might be complex Software reviews are important SQA activitiesSoftware reviews are important SQA activities Statistical SQA helps improve product quality Statistical SQA helps improve product quality
and software processand software process Software Safety is essential for critical systems Software Safety is essential for critical systems ISO 9001 standardizes the SQA activitiesISO 9001 standardizes the SQA activities