310313 software quality assurance 1 310313 software engineering 310313 software engineering

48
310313 310313 SOFTWARE QUALITY ASSURANCE SOFTWARE QUALITY ASSURANCE 1 SOFTWARE QUALITY ASSURANCE SOFTWARE QUALITY ASSURANCE 310313 310313 SOFTWARE ENGINEERING SOFTWARE ENGINEERING

Upload: elinor-douglas

Post on 28-Dec-2015

231 views

Category:

Documents


0 download

TRANSCRIPT

310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE1

SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE

310313310313SOFTWARE ENGINEERINGSOFTWARE ENGINEERING

310313310313SOFTWARE ENGINEERINGSOFTWARE ENGINEERING

310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE2

LEARNIING OBJECTIIVES1. Understand the quality management process and the central

process activities of quality assurance, quality planning and

quality control.

2. Understand the importance of st andar ds and met r i cs i n t he

quality management process.

3. Knowwhat i s software configuration management , why i t i s

i mpor t ant and what ar e i t s maj or act i vi t i es.

4. Under st and t he pr i nci pl es of sof t war e pr ocess i mpr ovement and

why process improvement is worthwhile.

310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE3

SOFTWARE QUALITY ASSURANCE : SOFTWARE QUALITY ASSURANCE : OUTLINEOUTLINE

Software Quality Assurance– Life Cycle Role – Purpose and Impoetance– Techniques : Standards and Metrics

Achieving Software Quanlity

– Achieving Product Quality Design Quality Metrics Formal Approaches Program Quality Metrics

– Achieving Project Quality Reviews Software Configuration Management

– Achieving Process Quality ISO 9001/9000-3 SEI Process Capability Maturity Model

– Achieving People Quality The People Capability Maturity Model

10, 12

310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE4

SOFTWARE QUALITY ASSURANCE: LIFE CYCLEROLE

310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE5

THE PURPOSE SOFTWARE QUALITY ASSURANCE THE PURPOSE SOFTWARE QUALITY ASSURANCE

Quality assurance consists of those procedures, techniques, and tools applied by professionals to ensure that a product meets or exceeds pre-specified standards during it’s development cycle. E.H. Bersoff

Quality assurance consists of those procedures, techniques, and tools applied by professionals to ensure that a product meets or exceeds pre-specified standards during it’s development cycle. E.H. Bersoff

quality assurance Defines organizational standards that lead to

high quality software.

quality planing Selects appropriate standards and tailors them

to a specific software product.

quality control Ensures standards are followed.

Continuous quality improvement should be the overall goal.

310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE6

WHY SQA ACTIVITIES PAY OFFWHY SQA ACTIVITIES PAY OFF

cost to findand fix a defect

logscale

1

10

100

Reqmts

1.00

Design

1.30

Code

2.00

Test

4.00

Systemtest

13.00

Fielduse

80-130

SQA pays off here becausedefects are cheap to fix

310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE7

SQA — AN UMBRELLA ACTIVITYSQA — AN UMBRELLA ACTIVITY

Software QualitySoftware QualityAssurance TechniquesAssurance Techniques

Software QualitySoftware QualityAssurance TechniquesAssurance Techniques

Standardsand

Procedures

Standardsand

Procedures

Metricsand

Measurement

Metricsand

Measurement

SoftwareConfigurationManagement

SoftwareConfigurationManagement

TestingTesting

FormalTechnicalReviews

FormalTechnicalReviews

Methodsand

Tools

Methodsand

Tools

The foundation of quality assurance.

310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE8

SOFTWARE QUALIITY ASSURANCE: TECHNIQUES

Includes all techniques used to improve the quality of the software:

1. developing and applying standards To ensure repeatability of the development process

2. developing and using metrics and measurement procedures To meet designgoals and improve the software development process

3. using appropriate methods and tools for system development To achieve a high quality system

4. doing formal technical reviews at each step To uncover quality problems and to sign-off

5. using software configuration management and change control To ensure changes are managed and controlled

6. performing multi-tiered testing To ensure effective defect detection. (BUT testing is not a panacea)

310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE9

1. They encapsulate best (or most appropriate) practices Acquired after much trial and error helps avoid previous mistakes

2. They provide a framework around which to implement SQA process Ensures that best practices are properly followed

3. They assist in ensuring continuity of project work Reduces learning effort when starting new work

product standards: define the characteristics all product artifacts should exhibit so as to have quality

process standards: define how the software process should be conducted to ensure quality software

SQA : SQA : STANDARDSSTANDARDS

Why are software standards important for SQA ?

Standards

Standards

Handbook

Handbook

needed!

needed!

Each project needs to decide which standards should be: ignored; used as is; modified; created.

310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE10

SQA TECHNIQUES : SQA TECHNIQUES : METRICSMETRICS

A metric is A metric is any any type of measurementtype of measurement that relates to a that relates to a software system product , process or related artifactsoftware system product , process or related artifact

A metric is A metric is any any type of measurementtype of measurement that relates to a that relates to a software system product , process or related artifactsoftware system product , process or related artifact

Why are metrics important for Software Quality Assurance?

1. Metrics can be used to control (i.e., plan and manage) the development process (e.g., effort expended, elapsed time,budget spent, etc.).2. Metrics can be used to predict an associated product quality (e.g., cyclomatic complexity can predict ease of maintenance).

Metrics are the only objective way to measure quality attributes of software; otherwise it is all opinion and guess work.

310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE11

ACHIEVINGACHIEVING SOFTWARE QUALITYSOFTWARE QUALITY

ProductProjectPeople

template

participate result

software engineers set of artifacts

set of activities

Process

education&

training

management&

monitoring

testing&

measurement

How canHow canwe achievewe achieve

software quality?software quality?

measurement&

feedback

310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE12

ACHIEVINGACHIEVING SOFTWARE QUALITY(cont’d)SOFTWARE QUALITY(cont’d)

1. We have a set of standards and quality attributes that a software product must meet. There are design goal to achieve.

2. We should be able to measure the quality of a software product. There is a way to determine how well the product

conforms to the design goal.

3. We should track the values of the quality attributes. So that it is possible to assess ,over time, how well we are

doing in achieving the design goal.

4. We use information about software quality to improve the quality of future software products. There is feedback into the software development process.

keypoint

310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE13

ACHIEVING PRODUCT QUALITY : ACHIEVING PRODUCT QUALITY : DESIGN GOALSDESIGN GOALS

• Recall that a design goal is an (external) attribute that we wantthe system to have such as:– safety – understandability – portability– security – testability – usability– reliability – adaptability – reusability– resilience – modularity – efficiency– robustness – maintainability – learnability

• Usually, we can only asses whether a system has one of theseattributes (i.e., meets its design goals) after it is completed!

• However, we would like to assess whether we are achieving adesign goal during the system’s development? How to do it?

Try to use internal attributes to assess (predict)whether the system will meet its design goals.

310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE14

ACHIEVING PRODUCT QUALITY :ACHIEVING PRODUCT QUALITY : DESIGN GOALSDESIGN GOALS(cont’d)(cont’d)

Examples of possible relationships between external andinternal attributes:

ProblemsProblems: 1. hard to formulate and validate relationships between internal and external attributes

2. software metrics must be collected, calibrated, interpreted

External(design goal)

maintainability

reliability

portability

usability

Internal(measurable)

# of parameters

cyclomatic complexity

lines of code

# of error messages

length of user manual

310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE15

ACHIEVING PRODUCT QUALITY: ACHIEVING PRODUCT QUALITY: DESIGN DESIGN QUALITY METRICSQUALITY METRICS

For a For a design componentdesign component, a , a key design goalkey design goal is is maintainabilitymaintainability..For a For a design componentdesign component, a , a key design goalkey design goal is is maintainabilitymaintainability..

For design components maintainability is related to:– cohesion - how closely related is the functionality of the component?

– coupling - how independent is the component?

– understandability - how easy is it to understand what the component does?

– adaptability - how easy is it to change the component?

Problem: most of these cannot be measured directly, but it is reasonable to infer that there is a relationship between these attributes and the “complexity” of a component

measure complexity How?How?

310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE16

ACHIEVING PRODUCT QUALITY: ACHIEVING PRODUCT QUALITY: DESIGN DESIGN QUALITY METRICSQUALITY METRICS

1) Structural fan-in/fan-out

fan-in – the number of calls to a component by other components

fan-out – the number of components called by a component

high fan-in => high coupling

high fan-out => calling component has high complexity

2) Informational fan-in/fan-out– consider also the number of parameters passed plus access to

shared data structures

complexity = component-length x (fan-in x fan-out)2

It has been validated using the Unix system

It is a useful predictor of effort required for implementation

310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE17

ACHIEVING PRODUCT QUALITY: ACHIEVING PRODUCT QUALITY: DESIGN DESIGN QUALITY METRICSQUALITY METRICS

3) IEEE Standard 982.1-1988

looks at: subsystem properties (number of subsystems and degree of coupling)

database properties (number of attributes and classes)

Compute a design structure quality index—DSQI (0-1)

Used to compare with past designs; if DSQI is too low,further design work and review may be required

We can also consider changes made throughout the lifetime of the software and compute how stable the product is (i.e., how many changes have been made in subsystems in the current release)

Define a software maturity index—SMI (0-1)

As SMI approaches 1, the product begins to stabilize

310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE18

IEEE STANDARD 982.1-1988IEEE STANDARD 982.1-1988

S1 = the total number of subsystems defined in the program architecture

S2 = the number of subsystems whose correct function depends on the source of data input or that produces data to be used elsewhere

S3 = the number of subsystems whose correct function depends on prior processing

S4 = the number of database items (includes data objects and all attributes that define objects)

S5 = the total number of unique database items

S6 = the number of database segments (different records or individual objects

S7 = the number of subsystems with a single entry and exit

This slide is for information only – you are not required to know this!

310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE19

IEEE STANDARD 982.1-1988IEEE STANDARD 982.1-1988

Program structure:D1 = 1 if the architecture was developed using a distinct methodD1 = 0 otherwise

Subsystem independence: D2 = 1 - (S2/S1)

Subsystems not dependent on prior processing: D3 = 1 - (S3/S1)

Database size: D4 = 1 - (S5/S4)

Database compartmentalization: D5 = 1 - (S6/S4)

Subsystem entrance/exit characteristic: D6 = 1 - (S7/S1)

DSQI = wiDi wi = relative weighting of each Di

This slide is for information only – you are not required to know this!

310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE20

IEEE STANDARD 982.1-1988IEEE STANDARD 982.1-1988

SMI = [MT - (Fa + Fc + Fd)]MT

MT = the number of subsystems in the current release

Fc = the number of subsystems in the current release that havebeen changed

Fa = the number of subsystems in the current release that havebeen added

Fd = the number of subsystems in the preceding release that were

deleted in the current release

This slide is for information only – you are not required to know this!

310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE21

ACHIEVING PRODUCT QUALITY: ACHIEVING PRODUCT QUALITY: PROGRAM PROGRAM QUALITY METRICSQUALITY METRICS

Some approaches to measure reliability and/or difficulty: a) Halstead’s Software Science

Looks at operators and operands in an implementation component and calculates values for component volume, V, component difficulty, D, and effort, E, required to implement the component.

n1 = number of unique operators in a componentn2 = number of unique operands in a componentN1 = the total number of operatorsN2 = the total number of operandsL = N1+ N2 (component length)

V = L * log2(n1 + n2) (component volume in bits)

D = (n1/2) * (N2/n2) (difficulty of implementing the component)

E = V * D (effort required to implement the component)

For an For an implementation componentimplementation component, the key quality , the key quality attributes are attributes are reliabilityreliability and and difficulty of implementationdifficulty of implementation..

For an For an implementation componentimplementation component, the key quality , the key quality attributes are attributes are reliabilityreliability and and difficulty of implementationdifficulty of implementation..

310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE22

ACHIEVING PRODUCT QUALITY: ACHIEVING PRODUCT QUALITY: PROGRAM PROGRAM QUALITY METRICSQUALITY METRICS

2) McCabe’s Complexity Metric

Looks at control flow in a component.

Cyclomatic Complexity measures component’s logical complexity

an indication of the testing difficulty of a component

Studies have shown a distinct relationship between the Cyclomatic Complexity of a component and the number of errors found in the source code, as well as the time required to find and fix errors.

3) Other program quality metrics– line of code (LOC) – length of identifiers– depth of conditional nesting

Standards can be established to avoid complex components and/or highlight problem components.

310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE23

ACHIEVING PRODUCT QUALITY: ACHIEVING PRODUCT QUALITY: FORMAL FORMAL APPROACHESAPPROACHES

a) Proving programs/specifications correct– Logically prove that requirements have been correctly transformed

into programs

(e.g., prove assertions about programs)

b) Statistical Quality Assurance– Categorize and determine cause of software defects

– Use 80-20 rule 80% of defects can be traced to 20% of causes– Isolate and correct 20% of the causes

effort directed to things that cause the majority of defects

c) The Cleanroom Process– A combination of above two approaches

310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE24

ACHIEVING PROJECT QUALITY: ACHIEVING PROJECT QUALITY: REVIEWSREVIEWS

Requirementscapture

Analysis

Design

Implementation

Testing

requirementswalkthroughs

analysiswalkthroughs

designwalkthroughs

codewalkthroughs

test planreview

Leads to Leads to earlyearlydiscovery of discovery of

defectsdefects

Requirements, Analysis Requirements, Analysis and Designand Design

introduce 50-60%introduce 50-60%of all defects.of all defects.

Formal technical Formal technical reviews can uncover reviews can uncover

75% of these!75% of these!

The principal method of The principal method of validating the qualityvalidating the quality of a project. of a project.The principal method of The principal method of validating the qualityvalidating the quality of a project. of a project.

310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE25

ACHIEVING PROJECT QUALIITY: SOFTWARE CON FIGURATION MANAGEMENT (SCM)

310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE26

SOFTWARE CONFIGURATION MANAGEMENT (SCM)SOFTWARE CONFIGURATION MANAGEMENT (SCM)

Software Configuration Management is an activity executed Software Configuration Management is an activity executed throughout the system life cycle to throughout the system life cycle to control the changecontrol the change to the to the

productsproducts and life cycle and life cycle artifactsartifacts..

Software Configuration Management is an activity executed Software Configuration Management is an activity executed throughout the system life cycle to throughout the system life cycle to control the changecontrol the change to the to the

productsproducts and life cycle and life cycle artifactsartifacts..

auditcontrol report

identify support

To what do we want to control changes?To what do we want to control changes?

configuration item: an artifact to which we want to control changes

10.2

Software Configuration Management• plans • programs• specifications • documents/manuals• procedures • data

310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE27

SCM ITEM : SCM ITEM : ITEM IDENTIFCATION AND ITEM IDENTIFCATION AND DESCRIPTIONDESCRIPTION

Each configuration item must be identified and described– a unique name– configuration item type– project identifier– change and/or version information– resources the configuration item provides or requires– pointer to actual configuration item

The configuration items also must be organized (i.e., need to define the relationships that exist between configuration items)– object diagram <part-of> domain model– domain model <related-to> use-case model

This isThis ismetadatametadata

aboutaboutconfigurationconfiguration

itemsitems

This information is use to define and construct a software library (database) that stores, manages and tracks configuration items.

10.3.1

310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE28

SCM ITEMS : SCM ITEMS : VERSIONSVERSIONS

A configuration item usually evolves throughout the software engineering process (i.e., it will have several versions)

An evolution graph can be used to describe aconfiguration item’s change history

version– Configuration itemk is obtained by modifying configuration itemi

– usually a result of bug fixes, enhanced system functionality, etc.

– itemk supercedes original itemi; created in a linear order

branch– A concurrent development path requiring independent configuration

management

variant– Different configurations that are intended to coexist– e.g., different configurations depending on operating system type

Oracle for Windows Oracle for Linux

310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE29

SCM ITEMS : SCM ITEMS : EVOLUTION GRAPHEVOLUTION GRAPH

1.0

2.0

2.1

2.2

3.0

2.0.1

2.0.2

2.0.2.1

2.0.2.2

2.0.2.3

branch line

parallel development

mainline development

branch line

version

310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE30

SCM SUPPORT : SCM SUPPORT : SOFTWARE LIBRARYSOFTWARE LIBRARY

A software library provides facilities to store, label, identify versions and track the status of the configuration items

softwarerepository

developer’sworkspace

masterdirectory

promotions

releases

everydaydevelopment

used by asingle developer

used by other developers

for users

Stores & tracks

check-in/check-out of configuration items

after SQA

10.3.5

Stores & tracks

Used for

310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE31

SCM : SCM : BASELINESBASELINES

A A baselinebaseline is a time/phase in the software development (usually is a time/phase in the software development (usually aa project milestone project milestone) after which any ) after which any changes must be formalizedchanges must be formalized

(e.g., go through a formal change control procedure).(e.g., go through a formal change control procedure).

A A baselinebaseline is a time/phase in the software development (usually is a time/phase in the software development (usually aa project milestone project milestone) after which any ) after which any changes must be formalizedchanges must be formalized

(e.g., go through a formal change control procedure).(e.g., go through a formal change control procedure).

10.3.2

To become a baseline, the configuration item must first pass a set of formal review procedures (e.g., formal code review, documentation review, etc.).

It then becomes part of the project software library.

After this a “check-out” procedure is applied to the item(i.e., access to and change of the configuration item is controlled)

Any modified configuration item must again go through a formal review process before it can replace the original (baselined) item. Requires a version control mechanism.

310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE32

SCM BASELINESSCM BASELINES

Requirementsgathering

Analysis

Design

Implementation

Testing

Requirements Specification

Analysis Specification

Design Specification

Source code, manuals

Test Plans/Procedures/Data

Operational System

310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE33

Uncontrolledchange rapidlyleads to chaos !

Uncontrolledchange rapidlyleads to chaos !

SCM CHANGE CONTROLSCM CHANGE CONTROL

STOP

CH

AN

GE

change request submitted byusers/developers

change control authority evaluates,decides and issues an engineeringchange order if approved

configuration item is checked-out,changed, checked-in after SQA,and version controlled

configuration item made availablefor use (promotion; release)

change request submitted byusers/developers

change control authority evaluates,decides and issues an engineeringchange order if approved

configuration item is checked-out,changed, checked-in after SQA,and version controlled

configuration item made availablefor use (promotion; release)

310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE34

SCM CHANGE CONTROLSCM CHANGE CONTROL

change request from user/developer

change control authority evaluates and decides

request is queued for action change request is denied

assign individuals to configuration items user is informed

“check-out” configuration items

make changes

review (audit) the change

“check-in” the configuration items that have been changed

establish a baseline for testing

perform quality assurance and testing activities

“promote” changes for inclusion in next release

rebuild appropriate version of software

review (audit) the change to all configuration items

include changes in release

distribute the release

can be applied tointernal requests

(i.e., user = developer)and

external requests(i.e., user = end-user)

310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE35

SCM : SCM : AUDIT AND STATUS REPORTINGAUDIT AND STATUS REPORTING

Audit: ensures that changes have been properly implemented.

– Have the proper steps and procedures been followed? checklist

– Usually done by Quality Assurance (QA) group if SCM is a formal activity.

Status reporting: keeps all parties informed and up-to-date on the status of a change.

– Status reporting is a communication mechanism among project members to help keep them coordinated.

Status reporting allows management/users to determine who made what changes, when and why.

310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE36

SCM BENEFITSSCM BENEFITS

Reduces the effort required to manage and effect change

Resulting in improved productivity

Leads to better software integrity and security

Resulting in increased quality

Generates information about the process

Resulting in enhanced management control

Maintains a software development database

Resulting in better record keeping and tracking

310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE37

Does Does process quality process quality product qualityproduct quality??Does Does process quality process quality product qualityproduct quality??

ACHIEVING PROCESS QUALITYACHIEVING PROCESS QUALITY

Unlike manufactured products, software development has some unique factors that affect its quality:– software is designed, not manufactured

– software development is creative, not mechanical

– individual skills and experience have a significant influence

– external factors (application novelty, commercial pressure)

Software development processes are organization specific

Sometime people&technology may be more important than process

Insufficient resources will always adversely affect quality

310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE38

ACHIEVING PROCESS QUALIITY ACHIEVING PROCESS QUALIITY: : ISO 9001/9000-3ISO 9001/9000-3

310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE39

ACHIEVINGACHIEVING PROCESS QUALITY: PROCESS QUALITY: ISO 9001/ 9000-3ISO 9001/ 9000-3

. . .

ISO 9000/9001quality model

Organizationquality model

Project 1quality plan

Project 2quality plan

Organizationalquality process

Project qualitymanagement

instantiated as

instantiated as

documents

supports

is usedto develop

certification is easy; can be a marketing ploy

Specific a genericquality model

The ISO quality Model need to be

customized fora particularorganization

Focus is onFocus is onprocessprocess

managementmanagement

310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE40

ISO 9001/9000-3 STANDARD (1/3)ISO 9001/9000-3 STANDARD (1/3)

A. Quality system framework

1. Management responsibility quality policy defined, documented, understood, implemented and

maintained responsibilities and authorities for all personnel defined in-house verification resources defined, trained and funded a designated manager ensures that the quality program is

implemented and maintained

2. Quality system requires a documented quality system be established should be an integral process throughout the entire life cycle

3. Internal quality system audits requires audits be planned and performed results communicated to management deficiencies corrected

4. Corrective action requires causes of non-conforming product be identified potential causes of non-conformance eliminated procedures changed as a result

This slide is for information only – you are not required to know this!

310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE41

ISO 9001/9000-3 STANDARD (2/3)ISO 9001/9000-3 STANDARD (2/3)

B. Quality Life Cycle Activities

1. Contract review contracts are reviewed to

determine whether the requirements are adequately defined, agree with the bid, and can be implemented

2. Purchaser’s requirements specification

procedures to identify and collect purchaser’s requirements

3. Development planning production processes are

defined and planned

4. Quality record quality assurance process is

planned

5. Design and implementation procedures to control and

verify the design

includes planning designactivities, identifying inputs and outputs, verifying the design and controlling design changes

6. Testing and validation requires systems/products to

be tested and validated before use

records of testing are kept

7. Acceptance procedures for acceptance

should be established

8. Replication, delivery and installation

procedures for the above should be established and maintained

9. Maintenance requires that maintenance

activities be performed as specified

This slide is for information only – you are not required to know this!

310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE42

ISO 9001/9000-3 STANDARD (3/3)ISO 9001/9000-3 STANDARD (3/3)C. Quality System Supporting Activities

6. Tools and Techniques are applied appropriately to

support the development process

7. Purchasing purchased products conform

to their specified requirements

8. Included software product should be verified and

maintained

9. Training training needs should be

identified and training provided since related tasks may require qualified personnel

training records should be maintained

1. Configuration management process of development and

maintenance should be documented and controlled

2. Document control distribution and modification

of documents should be controlled

3. Quality records quality records should be

collected, maintained and dispositioned

4. Measurement where appropriate, adequate

statistical techniques should be identified and used to verify the acceptability of process capability and product characteristics

5. Rules, practices and conventions are in place and followed

This slide is for information only – you are not required to know this!

310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE43

ACHIEVINGACHIEVING PROCESS QUALITY: PROCESS QUALITY:THE SEI CAPABILITY MATURITY MODEL (CMM)THE SEI CAPABILITY MATURITY MODEL (CMM)

SEU-CMM intended to help a software organizationSEU-CMM intended to help a software organizationimproveimprove their their software development processessoftware development processes..

SEU-CMM intended to help a software organizationSEU-CMM intended to help a software organizationimproveimprove their their software development processessoftware development processes..

Level 1 Organization: Initial process (ad hoc)– no formal procedures, no cost estimates, no project plans,

no management mechanism to ensure procedures are followed

Level 2 Organization: Repeatable process (intuitive)– basic project controls; intuitive methods used

Level 3 Organization: Defined process (qualitative)– development process defined and institutionalized

Level 4 Organization: Managed process (quantitative)– measured process; process database established

Level 5 Organization: Optimizing process– improvement feedback; rigorous defect-cause analysis and prevention

Focus is onFocus is onprocessprocess

improvementimprovement

12.3

310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE44

CAPABILITY MATURITY MODEL (CMM)CAPABILITY MATURITY MODEL (CMM)

Key Processes in Place

Level 1: Initial process

Level 2: Repeatable processRequirements ManagementSoftware Project PlanningSoftware Project Tracking & OversightSoftware Subcontract ManagementSoftware Quality AssuranceSoftware Configuration Management

Level 3: Defined processOrganization Process FocusOrganization Process DefinitionTraining ProgramIntegrated Software ManagementSoftware Product EngineeringIntergroup CoordinationPeer Reviews

Level 4: Managed processQuantitative Process ManagementSoftware Quality Management

Level 5: Optimizing processFault PreventionTechnology Change ManagementProcess Change Management

This slide is for information only – you are not required to know this!

310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE45

ACHIEVINGACHIEVING PROCESS QUALITY — NOTES PROCESS QUALITY — NOTES

It is possible for a Level 1 organization to receive ISO 9000 certification!

A Level 3 organization would have little difficulty in obtaining ISO 9000 certification.

A Level 2 organization would have significant advantage in obtaining ISO 9000 certification.

excellent software developers (e.g., space shuttle) attain around level 3-4 only

310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE46

ACHIEVINGACHIEVING PEOPLE QUALITY: PEOPLE QUALITY:PEOPLE CAPABILITY MATURITY MODEL (PCMM)PEOPLE CAPABILITY MATURITY MODEL (PCMM)

Intended to Intended to improve knowledgeimprove knowledge and and skillskill of people. of people.Intended to Intended to improve knowledgeimprove knowledge and and skillskill of people. of people.

Level 1 – Initial– no technical or management training provided;

staff talent not a critical resource; no organizational loyalty

Level 2 – Repeatable– focus on developing basic work practices; staff recruiting, growth and

development important; training to fill skill “gaps”; performance evaluated

Level 3 – Defined– focus on tailoring work practices to organization’s business; strategic plan to

locate and develop required talent; skills-based compensation

Level 4 – Managed– focus on increasing competence in critical skills; mentoring; team-building;

quantitative competence goals; evaluation of effectiveness of work practices

Level 5 – Optimizing– focus on improving team and individual skills; use of best practices

Focus is onFocus is onpeoplepeople

improvementimprovement

310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE47

SQA — THE BOTTOM LINESQA — THE BOTTOM LINE

an organization should have a quality manual which documents its quality assurance procedures

each project should have a quality plan which sets out the quality attributes that are most important for that project and how they will be assessed

an organization should have established standards for documentation

mechanisms (processes) should be established that monitor compliance with all quality requirements of the organization

reviews are the primary means of carrying out quality assurance

metrics can be used to highlight anomalous parts of the software which may have quality problems

310313310313 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE48

SUMMARY — SOFTWARE QUALITYSUMMARY — SOFTWARE QUALITY

Quality software does not just happen!Quality software does not just happen!Quality software does not just happen!Quality software does not just happen!

Quality assurance mechanisms should be built into the software development process

Developing quality software requires:– Management support and involvement

– Gathering and use of software metrics

– Policies and procedures that everyone follows

– Commitment to following the policies and procedures even when things get rough!

Testing is an important part of quality assurance, but its not all there is to obtaining a quality software product.