310313 software quality assurance 1 310313 software engineering 310313 software engineering
TRANSCRIPT
310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE1
SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE
310313310313SOFTWARE ENGINEERINGSOFTWARE ENGINEERING
310313310313SOFTWARE ENGINEERINGSOFTWARE ENGINEERING
310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE2
LEARNIING OBJECTIIVES1. Understand the quality management process and the central
process activities of quality assurance, quality planning and
quality control.
2. Understand the importance of st andar ds and met r i cs i n t he
quality management process.
3. Knowwhat i s software configuration management , why i t i s
i mpor t ant and what ar e i t s maj or act i vi t i es.
4. Under st and t he pr i nci pl es of sof t war e pr ocess i mpr ovement and
why process improvement is worthwhile.
310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE3
SOFTWARE QUALITY ASSURANCE : SOFTWARE QUALITY ASSURANCE : OUTLINEOUTLINE
Software Quality Assurance– Life Cycle Role – Purpose and Impoetance– Techniques : Standards and Metrics
Achieving Software Quanlity
– Achieving Product Quality Design Quality Metrics Formal Approaches Program Quality Metrics
– Achieving Project Quality Reviews Software Configuration Management
– Achieving Process Quality ISO 9001/9000-3 SEI Process Capability Maturity Model
– Achieving People Quality The People Capability Maturity Model
10, 12
310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE4
SOFTWARE QUALITY ASSURANCE: LIFE CYCLEROLE
310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE5
THE PURPOSE SOFTWARE QUALITY ASSURANCE THE PURPOSE SOFTWARE QUALITY ASSURANCE
Quality assurance consists of those procedures, techniques, and tools applied by professionals to ensure that a product meets or exceeds pre-specified standards during it’s development cycle. E.H. Bersoff
Quality assurance consists of those procedures, techniques, and tools applied by professionals to ensure that a product meets or exceeds pre-specified standards during it’s development cycle. E.H. Bersoff
quality assurance Defines organizational standards that lead to
high quality software.
quality planing Selects appropriate standards and tailors them
to a specific software product.
quality control Ensures standards are followed.
Continuous quality improvement should be the overall goal.
310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE6
WHY SQA ACTIVITIES PAY OFFWHY SQA ACTIVITIES PAY OFF
cost to findand fix a defect
logscale
1
10
100
Reqmts
1.00
Design
1.30
Code
2.00
Test
4.00
Systemtest
13.00
Fielduse
80-130
SQA pays off here becausedefects are cheap to fix
310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE7
SQA — AN UMBRELLA ACTIVITYSQA — AN UMBRELLA ACTIVITY
Software QualitySoftware QualityAssurance TechniquesAssurance Techniques
Software QualitySoftware QualityAssurance TechniquesAssurance Techniques
Standardsand
Procedures
Standardsand
Procedures
Metricsand
Measurement
Metricsand
Measurement
SoftwareConfigurationManagement
SoftwareConfigurationManagement
TestingTesting
FormalTechnicalReviews
FormalTechnicalReviews
Methodsand
Tools
Methodsand
Tools
The foundation of quality assurance.
310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE8
SOFTWARE QUALIITY ASSURANCE: TECHNIQUES
Includes all techniques used to improve the quality of the software:
1. developing and applying standards To ensure repeatability of the development process
2. developing and using metrics and measurement procedures To meet designgoals and improve the software development process
3. using appropriate methods and tools for system development To achieve a high quality system
4. doing formal technical reviews at each step To uncover quality problems and to sign-off
5. using software configuration management and change control To ensure changes are managed and controlled
6. performing multi-tiered testing To ensure effective defect detection. (BUT testing is not a panacea)
310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE9
1. They encapsulate best (or most appropriate) practices Acquired after much trial and error helps avoid previous mistakes
2. They provide a framework around which to implement SQA process Ensures that best practices are properly followed
3. They assist in ensuring continuity of project work Reduces learning effort when starting new work
product standards: define the characteristics all product artifacts should exhibit so as to have quality
process standards: define how the software process should be conducted to ensure quality software
SQA : SQA : STANDARDSSTANDARDS
Why are software standards important for SQA ?
Standards
Standards
Handbook
Handbook
needed!
needed!
Each project needs to decide which standards should be: ignored; used as is; modified; created.
310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE10
SQA TECHNIQUES : SQA TECHNIQUES : METRICSMETRICS
A metric is A metric is any any type of measurementtype of measurement that relates to a that relates to a software system product , process or related artifactsoftware system product , process or related artifact
A metric is A metric is any any type of measurementtype of measurement that relates to a that relates to a software system product , process or related artifactsoftware system product , process or related artifact
Why are metrics important for Software Quality Assurance?
1. Metrics can be used to control (i.e., plan and manage) the development process (e.g., effort expended, elapsed time,budget spent, etc.).2. Metrics can be used to predict an associated product quality (e.g., cyclomatic complexity can predict ease of maintenance).
Metrics are the only objective way to measure quality attributes of software; otherwise it is all opinion and guess work.
310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE11
ACHIEVINGACHIEVING SOFTWARE QUALITYSOFTWARE QUALITY
ProductProjectPeople
template
participate result
software engineers set of artifacts
set of activities
Process
education&
training
management&
monitoring
testing&
measurement
How canHow canwe achievewe achieve
software quality?software quality?
measurement&
feedback
310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE12
ACHIEVINGACHIEVING SOFTWARE QUALITY(cont’d)SOFTWARE QUALITY(cont’d)
1. We have a set of standards and quality attributes that a software product must meet. There are design goal to achieve.
2. We should be able to measure the quality of a software product. There is a way to determine how well the product
conforms to the design goal.
3. We should track the values of the quality attributes. So that it is possible to assess ,over time, how well we are
doing in achieving the design goal.
4. We use information about software quality to improve the quality of future software products. There is feedback into the software development process.
keypoint
310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE13
ACHIEVING PRODUCT QUALITY : ACHIEVING PRODUCT QUALITY : DESIGN GOALSDESIGN GOALS
• Recall that a design goal is an (external) attribute that we wantthe system to have such as:– safety – understandability – portability– security – testability – usability– reliability – adaptability – reusability– resilience – modularity – efficiency– robustness – maintainability – learnability
• Usually, we can only asses whether a system has one of theseattributes (i.e., meets its design goals) after it is completed!
• However, we would like to assess whether we are achieving adesign goal during the system’s development? How to do it?
Try to use internal attributes to assess (predict)whether the system will meet its design goals.
310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE14
ACHIEVING PRODUCT QUALITY :ACHIEVING PRODUCT QUALITY : DESIGN GOALSDESIGN GOALS(cont’d)(cont’d)
Examples of possible relationships between external andinternal attributes:
ProblemsProblems: 1. hard to formulate and validate relationships between internal and external attributes
2. software metrics must be collected, calibrated, interpreted
External(design goal)
maintainability
reliability
portability
usability
Internal(measurable)
# of parameters
cyclomatic complexity
lines of code
# of error messages
length of user manual
310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE15
ACHIEVING PRODUCT QUALITY: ACHIEVING PRODUCT QUALITY: DESIGN DESIGN QUALITY METRICSQUALITY METRICS
For a For a design componentdesign component, a , a key design goalkey design goal is is maintainabilitymaintainability..For a For a design componentdesign component, a , a key design goalkey design goal is is maintainabilitymaintainability..
For design components maintainability is related to:– cohesion - how closely related is the functionality of the component?
– coupling - how independent is the component?
– understandability - how easy is it to understand what the component does?
– adaptability - how easy is it to change the component?
Problem: most of these cannot be measured directly, but it is reasonable to infer that there is a relationship between these attributes and the “complexity” of a component
measure complexity How?How?
310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE16
ACHIEVING PRODUCT QUALITY: ACHIEVING PRODUCT QUALITY: DESIGN DESIGN QUALITY METRICSQUALITY METRICS
1) Structural fan-in/fan-out
fan-in – the number of calls to a component by other components
fan-out – the number of components called by a component
high fan-in => high coupling
high fan-out => calling component has high complexity
2) Informational fan-in/fan-out– consider also the number of parameters passed plus access to
shared data structures
complexity = component-length x (fan-in x fan-out)2
It has been validated using the Unix system
It is a useful predictor of effort required for implementation
310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE17
ACHIEVING PRODUCT QUALITY: ACHIEVING PRODUCT QUALITY: DESIGN DESIGN QUALITY METRICSQUALITY METRICS
3) IEEE Standard 982.1-1988
looks at: subsystem properties (number of subsystems and degree of coupling)
database properties (number of attributes and classes)
Compute a design structure quality index—DSQI (0-1)
Used to compare with past designs; if DSQI is too low,further design work and review may be required
We can also consider changes made throughout the lifetime of the software and compute how stable the product is (i.e., how many changes have been made in subsystems in the current release)
Define a software maturity index—SMI (0-1)
As SMI approaches 1, the product begins to stabilize
310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE18
IEEE STANDARD 982.1-1988IEEE STANDARD 982.1-1988
S1 = the total number of subsystems defined in the program architecture
S2 = the number of subsystems whose correct function depends on the source of data input or that produces data to be used elsewhere
S3 = the number of subsystems whose correct function depends on prior processing
S4 = the number of database items (includes data objects and all attributes that define objects)
S5 = the total number of unique database items
S6 = the number of database segments (different records or individual objects
S7 = the number of subsystems with a single entry and exit
This slide is for information only – you are not required to know this!
310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE19
IEEE STANDARD 982.1-1988IEEE STANDARD 982.1-1988
Program structure:D1 = 1 if the architecture was developed using a distinct methodD1 = 0 otherwise
Subsystem independence: D2 = 1 - (S2/S1)
Subsystems not dependent on prior processing: D3 = 1 - (S3/S1)
Database size: D4 = 1 - (S5/S4)
Database compartmentalization: D5 = 1 - (S6/S4)
Subsystem entrance/exit characteristic: D6 = 1 - (S7/S1)
DSQI = wiDi wi = relative weighting of each Di
This slide is for information only – you are not required to know this!
310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE20
IEEE STANDARD 982.1-1988IEEE STANDARD 982.1-1988
SMI = [MT - (Fa + Fc + Fd)]MT
MT = the number of subsystems in the current release
Fc = the number of subsystems in the current release that havebeen changed
Fa = the number of subsystems in the current release that havebeen added
Fd = the number of subsystems in the preceding release that were
deleted in the current release
This slide is for information only – you are not required to know this!
310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE21
ACHIEVING PRODUCT QUALITY: ACHIEVING PRODUCT QUALITY: PROGRAM PROGRAM QUALITY METRICSQUALITY METRICS
Some approaches to measure reliability and/or difficulty: a) Halstead’s Software Science
Looks at operators and operands in an implementation component and calculates values for component volume, V, component difficulty, D, and effort, E, required to implement the component.
n1 = number of unique operators in a componentn2 = number of unique operands in a componentN1 = the total number of operatorsN2 = the total number of operandsL = N1+ N2 (component length)
V = L * log2(n1 + n2) (component volume in bits)
D = (n1/2) * (N2/n2) (difficulty of implementing the component)
E = V * D (effort required to implement the component)
For an For an implementation componentimplementation component, the key quality , the key quality attributes are attributes are reliabilityreliability and and difficulty of implementationdifficulty of implementation..
For an For an implementation componentimplementation component, the key quality , the key quality attributes are attributes are reliabilityreliability and and difficulty of implementationdifficulty of implementation..
310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE22
ACHIEVING PRODUCT QUALITY: ACHIEVING PRODUCT QUALITY: PROGRAM PROGRAM QUALITY METRICSQUALITY METRICS
2) McCabe’s Complexity Metric
Looks at control flow in a component.
Cyclomatic Complexity measures component’s logical complexity
an indication of the testing difficulty of a component
Studies have shown a distinct relationship between the Cyclomatic Complexity of a component and the number of errors found in the source code, as well as the time required to find and fix errors.
3) Other program quality metrics– line of code (LOC) – length of identifiers– depth of conditional nesting
Standards can be established to avoid complex components and/or highlight problem components.
310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE23
ACHIEVING PRODUCT QUALITY: ACHIEVING PRODUCT QUALITY: FORMAL FORMAL APPROACHESAPPROACHES
a) Proving programs/specifications correct– Logically prove that requirements have been correctly transformed
into programs
(e.g., prove assertions about programs)
b) Statistical Quality Assurance– Categorize and determine cause of software defects
– Use 80-20 rule 80% of defects can be traced to 20% of causes– Isolate and correct 20% of the causes
effort directed to things that cause the majority of defects
c) The Cleanroom Process– A combination of above two approaches
310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE24
ACHIEVING PROJECT QUALITY: ACHIEVING PROJECT QUALITY: REVIEWSREVIEWS
Requirementscapture
Analysis
Design
Implementation
Testing
requirementswalkthroughs
analysiswalkthroughs
designwalkthroughs
codewalkthroughs
test planreview
Leads to Leads to earlyearlydiscovery of discovery of
defectsdefects
Requirements, Analysis Requirements, Analysis and Designand Design
introduce 50-60%introduce 50-60%of all defects.of all defects.
Formal technical Formal technical reviews can uncover reviews can uncover
75% of these!75% of these!
The principal method of The principal method of validating the qualityvalidating the quality of a project. of a project.The principal method of The principal method of validating the qualityvalidating the quality of a project. of a project.
310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE25
ACHIEVING PROJECT QUALIITY: SOFTWARE CON FIGURATION MANAGEMENT (SCM)
310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE26
SOFTWARE CONFIGURATION MANAGEMENT (SCM)SOFTWARE CONFIGURATION MANAGEMENT (SCM)
Software Configuration Management is an activity executed Software Configuration Management is an activity executed throughout the system life cycle to throughout the system life cycle to control the changecontrol the change to the to the
productsproducts and life cycle and life cycle artifactsartifacts..
Software Configuration Management is an activity executed Software Configuration Management is an activity executed throughout the system life cycle to throughout the system life cycle to control the changecontrol the change to the to the
productsproducts and life cycle and life cycle artifactsartifacts..
auditcontrol report
identify support
To what do we want to control changes?To what do we want to control changes?
configuration item: an artifact to which we want to control changes
10.2
Software Configuration Management• plans • programs• specifications • documents/manuals• procedures • data
310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE27
SCM ITEM : SCM ITEM : ITEM IDENTIFCATION AND ITEM IDENTIFCATION AND DESCRIPTIONDESCRIPTION
Each configuration item must be identified and described– a unique name– configuration item type– project identifier– change and/or version information– resources the configuration item provides or requires– pointer to actual configuration item
The configuration items also must be organized (i.e., need to define the relationships that exist between configuration items)– object diagram <part-of> domain model– domain model <related-to> use-case model
This isThis ismetadatametadata
aboutaboutconfigurationconfiguration
itemsitems
This information is use to define and construct a software library (database) that stores, manages and tracks configuration items.
10.3.1
310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE28
SCM ITEMS : SCM ITEMS : VERSIONSVERSIONS
A configuration item usually evolves throughout the software engineering process (i.e., it will have several versions)
An evolution graph can be used to describe aconfiguration item’s change history
version– Configuration itemk is obtained by modifying configuration itemi
– usually a result of bug fixes, enhanced system functionality, etc.
– itemk supercedes original itemi; created in a linear order
branch– A concurrent development path requiring independent configuration
management
variant– Different configurations that are intended to coexist– e.g., different configurations depending on operating system type
Oracle for Windows Oracle for Linux
310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE29
SCM ITEMS : SCM ITEMS : EVOLUTION GRAPHEVOLUTION GRAPH
1.0
2.0
2.1
2.2
3.0
2.0.1
2.0.2
2.0.2.1
2.0.2.2
2.0.2.3
branch line
parallel development
mainline development
branch line
version
310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE30
SCM SUPPORT : SCM SUPPORT : SOFTWARE LIBRARYSOFTWARE LIBRARY
A software library provides facilities to store, label, identify versions and track the status of the configuration items
softwarerepository
developer’sworkspace
masterdirectory
promotions
releases
everydaydevelopment
used by asingle developer
used by other developers
for users
Stores & tracks
check-in/check-out of configuration items
after SQA
10.3.5
Stores & tracks
Used for
310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE31
SCM : SCM : BASELINESBASELINES
A A baselinebaseline is a time/phase in the software development (usually is a time/phase in the software development (usually aa project milestone project milestone) after which any ) after which any changes must be formalizedchanges must be formalized
(e.g., go through a formal change control procedure).(e.g., go through a formal change control procedure).
A A baselinebaseline is a time/phase in the software development (usually is a time/phase in the software development (usually aa project milestone project milestone) after which any ) after which any changes must be formalizedchanges must be formalized
(e.g., go through a formal change control procedure).(e.g., go through a formal change control procedure).
10.3.2
To become a baseline, the configuration item must first pass a set of formal review procedures (e.g., formal code review, documentation review, etc.).
It then becomes part of the project software library.
After this a “check-out” procedure is applied to the item(i.e., access to and change of the configuration item is controlled)
Any modified configuration item must again go through a formal review process before it can replace the original (baselined) item. Requires a version control mechanism.
310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE32
SCM BASELINESSCM BASELINES
Requirementsgathering
Analysis
Design
Implementation
Testing
Requirements Specification
Analysis Specification
Design Specification
Source code, manuals
Test Plans/Procedures/Data
Operational System
310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE33
Uncontrolledchange rapidlyleads to chaos !
Uncontrolledchange rapidlyleads to chaos !
SCM CHANGE CONTROLSCM CHANGE CONTROL
STOP
CH
AN
GE
change request submitted byusers/developers
change control authority evaluates,decides and issues an engineeringchange order if approved
configuration item is checked-out,changed, checked-in after SQA,and version controlled
configuration item made availablefor use (promotion; release)
change request submitted byusers/developers
change control authority evaluates,decides and issues an engineeringchange order if approved
configuration item is checked-out,changed, checked-in after SQA,and version controlled
configuration item made availablefor use (promotion; release)
310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE34
SCM CHANGE CONTROLSCM CHANGE CONTROL
change request from user/developer
change control authority evaluates and decides
request is queued for action change request is denied
assign individuals to configuration items user is informed
“check-out” configuration items
make changes
review (audit) the change
“check-in” the configuration items that have been changed
establish a baseline for testing
perform quality assurance and testing activities
“promote” changes for inclusion in next release
rebuild appropriate version of software
review (audit) the change to all configuration items
include changes in release
distribute the release
can be applied tointernal requests
(i.e., user = developer)and
external requests(i.e., user = end-user)
310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE35
SCM : SCM : AUDIT AND STATUS REPORTINGAUDIT AND STATUS REPORTING
Audit: ensures that changes have been properly implemented.
– Have the proper steps and procedures been followed? checklist
– Usually done by Quality Assurance (QA) group if SCM is a formal activity.
Status reporting: keeps all parties informed and up-to-date on the status of a change.
– Status reporting is a communication mechanism among project members to help keep them coordinated.
Status reporting allows management/users to determine who made what changes, when and why.
310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE36
SCM BENEFITSSCM BENEFITS
Reduces the effort required to manage and effect change
Resulting in improved productivity
Leads to better software integrity and security
Resulting in increased quality
Generates information about the process
Resulting in enhanced management control
Maintains a software development database
Resulting in better record keeping and tracking
310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE37
Does Does process quality process quality product qualityproduct quality??Does Does process quality process quality product qualityproduct quality??
ACHIEVING PROCESS QUALITYACHIEVING PROCESS QUALITY
Unlike manufactured products, software development has some unique factors that affect its quality:– software is designed, not manufactured
– software development is creative, not mechanical
– individual skills and experience have a significant influence
– external factors (application novelty, commercial pressure)
Software development processes are organization specific
Sometime people&technology may be more important than process
Insufficient resources will always adversely affect quality
310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE38
ACHIEVING PROCESS QUALIITY ACHIEVING PROCESS QUALIITY: : ISO 9001/9000-3ISO 9001/9000-3
310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE39
ACHIEVINGACHIEVING PROCESS QUALITY: PROCESS QUALITY: ISO 9001/ 9000-3ISO 9001/ 9000-3
. . .
ISO 9000/9001quality model
Organizationquality model
Project 1quality plan
Project 2quality plan
Organizationalquality process
Project qualitymanagement
instantiated as
instantiated as
documents
supports
is usedto develop
certification is easy; can be a marketing ploy
Specific a genericquality model
The ISO quality Model need to be
customized fora particularorganization
Focus is onFocus is onprocessprocess
managementmanagement
310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE40
ISO 9001/9000-3 STANDARD (1/3)ISO 9001/9000-3 STANDARD (1/3)
A. Quality system framework
1. Management responsibility quality policy defined, documented, understood, implemented and
maintained responsibilities and authorities for all personnel defined in-house verification resources defined, trained and funded a designated manager ensures that the quality program is
implemented and maintained
2. Quality system requires a documented quality system be established should be an integral process throughout the entire life cycle
3. Internal quality system audits requires audits be planned and performed results communicated to management deficiencies corrected
4. Corrective action requires causes of non-conforming product be identified potential causes of non-conformance eliminated procedures changed as a result
This slide is for information only – you are not required to know this!
310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE41
ISO 9001/9000-3 STANDARD (2/3)ISO 9001/9000-3 STANDARD (2/3)
B. Quality Life Cycle Activities
1. Contract review contracts are reviewed to
determine whether the requirements are adequately defined, agree with the bid, and can be implemented
2. Purchaser’s requirements specification
procedures to identify and collect purchaser’s requirements
3. Development planning production processes are
defined and planned
4. Quality record quality assurance process is
planned
5. Design and implementation procedures to control and
verify the design
includes planning designactivities, identifying inputs and outputs, verifying the design and controlling design changes
6. Testing and validation requires systems/products to
be tested and validated before use
records of testing are kept
7. Acceptance procedures for acceptance
should be established
8. Replication, delivery and installation
procedures for the above should be established and maintained
9. Maintenance requires that maintenance
activities be performed as specified
This slide is for information only – you are not required to know this!
310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE42
ISO 9001/9000-3 STANDARD (3/3)ISO 9001/9000-3 STANDARD (3/3)C. Quality System Supporting Activities
6. Tools and Techniques are applied appropriately to
support the development process
7. Purchasing purchased products conform
to their specified requirements
8. Included software product should be verified and
maintained
9. Training training needs should be
identified and training provided since related tasks may require qualified personnel
training records should be maintained
1. Configuration management process of development and
maintenance should be documented and controlled
2. Document control distribution and modification
of documents should be controlled
3. Quality records quality records should be
collected, maintained and dispositioned
4. Measurement where appropriate, adequate
statistical techniques should be identified and used to verify the acceptability of process capability and product characteristics
5. Rules, practices and conventions are in place and followed
This slide is for information only – you are not required to know this!
310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE43
ACHIEVINGACHIEVING PROCESS QUALITY: PROCESS QUALITY:THE SEI CAPABILITY MATURITY MODEL (CMM)THE SEI CAPABILITY MATURITY MODEL (CMM)
SEU-CMM intended to help a software organizationSEU-CMM intended to help a software organizationimproveimprove their their software development processessoftware development processes..
SEU-CMM intended to help a software organizationSEU-CMM intended to help a software organizationimproveimprove their their software development processessoftware development processes..
Level 1 Organization: Initial process (ad hoc)– no formal procedures, no cost estimates, no project plans,
no management mechanism to ensure procedures are followed
Level 2 Organization: Repeatable process (intuitive)– basic project controls; intuitive methods used
Level 3 Organization: Defined process (qualitative)– development process defined and institutionalized
Level 4 Organization: Managed process (quantitative)– measured process; process database established
Level 5 Organization: Optimizing process– improvement feedback; rigorous defect-cause analysis and prevention
Focus is onFocus is onprocessprocess
improvementimprovement
12.3
310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE44
CAPABILITY MATURITY MODEL (CMM)CAPABILITY MATURITY MODEL (CMM)
Key Processes in Place
Level 1: Initial process
Level 2: Repeatable processRequirements ManagementSoftware Project PlanningSoftware Project Tracking & OversightSoftware Subcontract ManagementSoftware Quality AssuranceSoftware Configuration Management
Level 3: Defined processOrganization Process FocusOrganization Process DefinitionTraining ProgramIntegrated Software ManagementSoftware Product EngineeringIntergroup CoordinationPeer Reviews
Level 4: Managed processQuantitative Process ManagementSoftware Quality Management
Level 5: Optimizing processFault PreventionTechnology Change ManagementProcess Change Management
This slide is for information only – you are not required to know this!
310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE45
ACHIEVINGACHIEVING PROCESS QUALITY — NOTES PROCESS QUALITY — NOTES
It is possible for a Level 1 organization to receive ISO 9000 certification!
A Level 3 organization would have little difficulty in obtaining ISO 9000 certification.
A Level 2 organization would have significant advantage in obtaining ISO 9000 certification.
excellent software developers (e.g., space shuttle) attain around level 3-4 only
310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE46
ACHIEVINGACHIEVING PEOPLE QUALITY: PEOPLE QUALITY:PEOPLE CAPABILITY MATURITY MODEL (PCMM)PEOPLE CAPABILITY MATURITY MODEL (PCMM)
Intended to Intended to improve knowledgeimprove knowledge and and skillskill of people. of people.Intended to Intended to improve knowledgeimprove knowledge and and skillskill of people. of people.
Level 1 – Initial– no technical or management training provided;
staff talent not a critical resource; no organizational loyalty
Level 2 – Repeatable– focus on developing basic work practices; staff recruiting, growth and
development important; training to fill skill “gaps”; performance evaluated
Level 3 – Defined– focus on tailoring work practices to organization’s business; strategic plan to
locate and develop required talent; skills-based compensation
Level 4 – Managed– focus on increasing competence in critical skills; mentoring; team-building;
quantitative competence goals; evaluation of effectiveness of work practices
Level 5 – Optimizing– focus on improving team and individual skills; use of best practices
Focus is onFocus is onpeoplepeople
improvementimprovement
310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE47
SQA — THE BOTTOM LINESQA — THE BOTTOM LINE
an organization should have a quality manual which documents its quality assurance procedures
each project should have a quality plan which sets out the quality attributes that are most important for that project and how they will be assessed
an organization should have established standards for documentation
mechanisms (processes) should be established that monitor compliance with all quality requirements of the organization
reviews are the primary means of carrying out quality assurance
metrics can be used to highlight anomalous parts of the software which may have quality problems
310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE48
SUMMARY — SOFTWARE QUALITYSUMMARY — SOFTWARE QUALITY
Quality software does not just happen!Quality software does not just happen!Quality software does not just happen!Quality software does not just happen!
Quality assurance mechanisms should be built into the software development process
Developing quality software requires:– Management support and involvement
– Gathering and use of software metrics
– Policies and procedures that everyone follows
– Commitment to following the policies and procedures even when things get rough!
Testing is an important part of quality assurance, but its not all there is to obtaining a quality software product.