software productivity consortium

Copyright © 2001, Software Productivity Consortium NFP, Inc. All rights reserved.

1-1S O F T W A R E

P R O D U C T I V I T YC O N S O R T I U M

SOFTWAREPRODUCTIVITYCONSORTIUM

Concepts of Return on Investment forProcess Improvement

Presentation for PSM Conference

July, 2001

Bob MacIver


1-2S O F T W A R E


Typical ConcernsTypical Concerns

• How can I maximize my process improvement

efforts?

• Which process improvement

technologies are the most cost

effective?

• How can I measure the return

on investment (ROI) for process

improvement initiatives?


1-3S O F T W A R E


A SurveyA Survey

• What kind of organizations and projects are represented?

• How is process improvement currently measured and

justified?

— What indicators?

— Which initiatives?

• How are benefits measured?

• How are costs measured?


1-4S O F T W A R E


Maturity Levels of RespondentsMaturity Levels of Respondents

0 2 4 6 8 10 12 14 16

NoLevel

1

2

3

4

5 SW CMM

SE CMM

EIA 731

Number of Organizations

Matu

rity Level


1-5S O F T W A R E


SurveySurveyGeneral FindingsGeneral Findings

• Survey size was small, but relevant

• Respondents were evenly distributed across CMM Levels

• Little consistency in the definition of measures used among

organizations

• Most of the organizations track SPI using a growth or

improvement factor, rather than financially

• Very few organizations are tracking the true cost or benefit of

their SPI initiatives

• No definitive patterns associated with CMM Maturity, or

Government vs. Commercial Marketplace

(Exception was Earned Value)


1-6S O F T W A R E


SurveySurveyKey Findings (cont.)Key Findings (cont.)

• 38% of respondents tracked the cost of SPI initiatives

— Formal inspections tracked financially by just over half of

those who perform them

• 30% track financial benefits of indicators

— Financial benefit of quality, productivity, or cycle time

tracked by less than 20% of responding organizations

• 38% of the respondents track rework above the project level

— One organization tracks the cost of rework for all or most

projects

• 12% track the Cost of Quality at organizational and enterprise

level


1-7S O F T W A R E


Survey ConclusionsSurvey Conclusions

• Responses reflect a strong engineering focus with a relatively

low level focus on cost/benefit of SPI

• Respondents generally are not well positioned to calculate

financial ROI of their SPI program

• Lack of standard measurement definitions and ROI process

models inhibit progress in justifying SPI from a financial

perspective


1-8S O F T W A R E


The Problem With ROIThe Problem With ROI

Rework Cost ?

Productivity ?

Quality ?Customer

Satisfaction

???

Multiple Relationships Make it Difficult toMultiple Relationships Make it Difficult to

Assign the BenefitsAssign the Benefits

Cycle Time ?

IV&V ?

Inspections,Peer Reviews

Direct Relationship

Potential Relationship

How do we showrelationships to

profitability and value?


1-9S O F T W A R E


ROI ConundrumROI Conundrum

Training

Productivity

Management Control

Defect Prevent

Customer Satisfaction

Cost

Inspections,Peer Reviews

ABC, ABM

Cost of Quality

CM

Measurement

QA

Estimation

Quality

Defect Analysis

Cycle Time

Tools

Planning

Rework Infrastructure

Supply Mgmt.Reuse

PAL

Reqmts. Mgmt.

COTS

SEPG IV&V

Defined Process

Example OnlyDoes not incorporate allelements or relationships


1-10S O F T W A R E


Key IndicatorsKey Indicators

Primary Indicators

ü Quality

ü Productivity

ü Cycle Time

ü Cost

ü Customer Satisfaction

Other Important Indicators

ü Cost of Quality

ü Cost of Rework


1-11S O F T W A R E


- 1 = ROI

5Process

ImprovementInitiatives

ProcessImprovement

Initiatives

ProcessImprovement

Initiatives

2

� (Benefits)f (x)

3

� (Costs)f (labor, equipment, training,…)

4

Conceptual ROI ModelConceptual ROI Model

Why?

1

Processes IndicatorsBase Measures


1-12S O F T W A R E


Secondary IndicatorsPrimary Indicators Business Goals

QualityCustomer

Satisfaction

Cycle Time Cost

Reducedrework

RequiredFunctionality,

Fewerfailures

Lower price

Lower support andmaintenance costs

Productivity

Increasedcapacity

Reducedeffort

Market ShareProfit

Shorterschedules

Time-to-market

Higher marginLow-costprovider

RetentionReferrals

Award feesBonuses

Image

ProducesDrives

Indicator RelationshipsIndicator Relationships


1-13S O F T W A R E


Project ManagementEngineering Management Executive Management

Business Goals

QualityCustomer

Satisfaction

Cycle Time Cost

Reducedrework

RequiredFunctionality,

Fewerfailures

Lower Price

Lower support andmaintenance costs

Productivity

Increasedcapacity

Reducedeffort

Market Share

&Profit

Shorterschedules

Time-to-market

Higher marginLow-costprovider

RetentionReferrals

Award feesBonuses

Image

SPI Activities

Focusing ROI on the BusinessFocusing ROI on the Business

CustomerProcess Financial


1-14S O F T W A R E


Mapping Indicators to InitiativesMapping Indicators to Initiatives(Cycle Time Example)(Cycle Time Example)

DefinedProcess

HistoricalData

EstimatingModel

ChangeControl

RequirementsManagement

DefinedProcess

MeasurementProgram

Quality

Cycle Time

ReducedRework

Productivity

IncreasedCapacity

ShorterSchedules(Cust. Sat.)

ReducedLabor

(Cost)

Early toMarket

(Market Share)EfficientProcess

PredictableSchedules

DefinedProcess

ProcessEngineering

ProcessMapping

Process Initiatives Desired Results ofInitiatives

BetterEstimates

StableRequirements

Fewer Steps,Delays


1-15S O F T W A R E


Cost of Quality (COQ)Cost of Quality (COQ)

• What is the cost of poor quality?

• What are the key drivers?

• What is the cost of achieving higher quality?

• Which should be the highest priorities?

• How successful are the efforts designed to drive the COQ

downward?


1-16S O F T W A R E


Elements of COQElements of COQ

Cost of Poor Quality

(Non-conformance)

Cost of AchievingHigher Quality

Cost of Internal Failure

Defects Discovered Prior to Shipment

Cost of External Failure

Defects Discovered After Shipment

Appraisal Costs

Testing, Inspections, Quality Audits, Assessments

Prevention CostsSEPG, SQA, CM,

Reqmts. Management, Defect Prevention,

Training, Risk Mgmt.


1-17S O F T W A R E


0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

% o

f To

tal P

roje

ct C

ost

s

1988 1989 1990 1991 1992 1993Years

Cost of performanceCost of appraisalCost of preventionCost of nonconformance

Reducing the Cost of QualityReducing the Cost of Quality

Adapted from: Dion, R., Process Improvement and the Corporate Balance Sheet , IEEE Software, July 1993


1-18S O F T W A R E


Cost of ReworkCost of Rework

• Typically 30% to 50% (or more) of project cost for lower

maturity organizations

• Only one respondent (of 16) tracks Cost of Rework on all or

most projects

• True Cost of Rework is not

well known in most organizations


1-19S O F T W A R E


INJECTIONPHASE

CAPTURE, ANALYZE, PRIORITIZE, APPROVE

RE

WO

RK

AC

TIV

ITIE

S P

ER

PH

AS

E

Req. HLD LDD Code IT ST ReleaseAT

DISCOVERYPHASE

Cost of ReworkCost of ReworkCoding Defects Found in System TestCoding Defects Found in System Test


1-20S O F T W A R E


INJECTIONPHASE

CAPTURE, ANALYZE, PRIORITIZE, APPROVE

RE

WO

RK

AC

TIV

ITIE

S P

ER

PH

AS

E

Req. HLD LDD Code IT ST ReleaseAT

DISCOVERYPHASE

Cost of ReworkCost of ReworkRequirements Defects Found in Acceptance TestRequirements Defects Found in Acceptance Test


1-21S O F T W A R E


SummarySummary

• Measurement programs are typically focused on engineering

effectiveness rather than business case

• Organizations generally are not well positioned to calculate

financial ROI of their SPI program

• ROI Conundrum can be resolved by focusing on costs and

benefits separately

• A focus on Cost of Quality and Cost of Rework can provide

significant results

• Data indicate ROI is a ‘growth area’

— Provides a means for focusing SPI investments on business goals andpriorities

— Helps in establishing effective measurement programs/habits

software productivity consortium

Documents