1University of Sunderland

CIFM03 Lecture 2

Quality Management of IT

CIFM03

Lecture 2

2University of Sunderland

CIFM03 Lecture 2

Software Process Assessment - SEI Capability Maturity Model

Quality & Information Systems Strategies

CIFM03

3University of Sunderland

CIFM03 Lecture 2

Objectives for this Lecture

To identify the concept of software process

• To introduce the Capability Maturity Model (CMM) and the Software Engineering Institute (SEI)

• To overview Software Process Improvement

• To introduce the Capability Maturity Model Integration (CMMI)

4University of Sunderland

CIFM03 Lecture 2

The Software Process

• The software process – A structured set of activities required to develop a

software system– Specification– Design– Implementation– Validation– Evolution

• A software process model– an abstract representation of a process. It represents

a useful abstracted description of the process, e.g. useful to a developer.

5University of Sunderland

CIFM03 Lecture 2

What is the CMM?...

‘The CMM is a 5-level model where each maturity level is “a well-defined evolutionary plateau on the path towards becoming a mature software organisation.’

~ SEI

6University of Sunderland

CIFM03 Lecture 2

Again, What is the CMM?...

‘The CMM provides a conceptual structure

for improving the management and

development of software products in a

disciplined and consistent way.’

~ SEI

7University of Sunderland

CIFM03 Lecture 2

SEI CMM Levels

REPEATABLE

INITIAL

DEFINED

MANAGED

OPTIMISING

discipline

standard,consistent

predictable

continuousimprovement

project management

engineering management

quantitative management

changemanagement

8University of Sunderland

CIFM03 Lecture 2

CMM levels 1 - 3

1) Initial. The process is ad hoc/chaotic. Few processes are defined; success depends on highly competent, committed individuals.

2) Repeatable. Basic project management processes are in place to track cost, schedule, and functionality. The discipline is in place to repeat earlier successes on projects with similar applications.

3) Defined. Project management and software engineering activities are integrated into a standard software process for the organization. All projects use an approved, tailored version of it for developing and maintaining software.

9University of Sunderland

CIFM03 Lecture 2

CMM levels 4 and 5

4) Managed. Detailed measures of the software process and product quality are collected. Both the software process and products are quantitatively understood and controlled.

5) Optimising. Continuous process improvement is supported with quantitative feedback from the process and from controlled testing of new ideas and technologies

Each maturity level is decomposed into several key process areas that indicate the areas an organization should focus on to improve its software process.

10University of Sunderland

CIFM03 Lecture 2

The CMM Structure, (see notes)

Maturity Level

Key Process Areas

Implementation/institutionalisation

Infrastructure/activities

Common Features

Key Practices

Process capability

Goals

contain

organised by

contain

indicate

achieve

address

describe

11University of Sunderland

CIFM03 Lecture 2

Applying the CMM• Self assessment

possible because the CMM document is detailed, cheaper to do in-house; but suffers from lack of experience/independence

• Formal assessmentindependent and experienced, identifies company improvement priorities

• Software Capability Evaluationto identify qualified contractors

12University of Sunderland

CIFM03 Lecture 2

Assessment Steps• Team selection

small, qualified team chosen, which is as independent as possible• Maturity questionnaire

using a questionnaire, obtain a sample response from departments/projects; the results are analysed for further investigation/clarification

• Site visitactual departments/projects are visited; interviews conducted, documents reviewed, priority process areas scrutinised

• Presentation of findings to managementTWO parts - assessment of current level of organisation: identify strengths & weaknesses and key improvement areas

13University of Sunderland

CIFM03 Lecture 2

Critical Evaluation (I)...• Best practice – how do we know its best practice, as defined by

who, with respect to what type of development, what application area, with what tools?

• Management and process discipline- do we know what made the software good and how was it produced?

• Is the CMM maturity scale really ordinal ?Is there an identifiable graded increase in capability at each level?

• Are the levels stable evolutionary plateaus ? Yes if each lower level is nested in the subsequent level and all processes are continually applied? (L1 in L2?)

14University of Sunderland

CIFM03 Lecture 2

Critical Evaluation (II)...

• Scoring approach-CMM employs a multi-hurdle system: questions and ‘key questions’

• Sparse data sets -approx. 100 questions to cover everything in a software development process

• Number vs. Profile - the number of answers is potentially very large and therefore difficult to interpret, however, the profile is enlightening

15University of Sunderland

CIFM03 Lecture 2

ISO 9001 and CMM

comparedCMM

• Specific to software development

• Used in USA, less widely elsewhere

• Provides detailed and specific definition of what is required for given levels

• Assesses on 5 levels

• CMM Level 3 ISO 9001

• Relevant to software development process

• No specific time limit

• Ongoing audit

ISO 9001

• Intended as generic

• Recognised and accepted in most countries

• Specifies concepts, principles and safeguards that should be in place

• Establishes one acceptable level

• ISO 9001 Stabilises the customer - supplier relationship

• Certification valid for three years

• Auditors regularly return for checks during the lifetime of the certificate

16University of Sunderland

CIFM03 Lecture 2

• It focuses on project management rather than product development.

• It ignores the use of technologies such as rapid prototyping.

• It does not incorporate risk analysis as a key process area

• It does not define its domain of applicability

The Software Engineering Institute model’s

problems

17University of Sunderland

CIFM03 Lecture 2

SEI Capability Maturity Model Integration (CMMI)

• A New generation of maturity model: the CMMI, replacing the existing CMM.

• CMMI covers a broader domain since it addresses not only software but also system aspects.

• It is also published in 2 representations: staged (cf. CMM) or continuous (cf. ISO15504, a result of the Software Process Improvement and

Capability Determination or SPICE project). SPICE is a major international initiative to support the development of an International

Standard for Software Process Assessment.

• Expertise on CMMI is still rare because the transition to CMMI is ongoing.

• The emphasis is on each process area in the model and on goals and practices associated with each area.

18University of Sunderland

CIFM03 Lecture 2

SEI CMMI Levels

MANAGED

INITIAL

DEFINED

QUANTITATIVELYMANAGED

OPTIMISING

discipline

standard,consistent

predictable

continuousimprovement

project management

engineering management

quantitative management

changemanagement

19University of Sunderland

CIFM03 Lecture 2

A Final Word

“There are serious measurement questions... We must

ensure that the models are appropriate (see Ch. 2 of

Software Metrics) .... We must understand how reliable and

valid measurements and models are. We must know what

entities are being measured, and we must test the

relationships between the maturity scores and the

behaviours that ‘maturity’ is supposed to produce or

enhance.”

Fenton & Pfleeger, Software Metrics, Edn. 2 , 1997