an analysis of changes in productivity and cocomo cost drivers over the years

University of Southern California

Center for Systems and Software Engineering

© 2009, USC-CSSE 1

An Analysis of Changes in Productivity and COCOMO Cost

Drivers over the YearsVu Nguyen, Barry Boehm

Center for Systems and Software Engineering (CSSE)

24th Int’l COCOMO Forum

Nov 3rd, 2009



© 2009, USC-CSSE 2

Outline

Objectives and Background

Productivity Changes

Discussions and Conclusions

Cost Driver Changes



© 2009, USC-CSSE 3

Objectives and Background



© 2009, USC-CSSE 4

Objectives

• Analysis of Productivity

– How the productivity of the COCOMO data projects has changed over the years?

– What caused the changes in productivity?

• Analysis of COCOMO cost drivers

– How cost driver ratings have changed over the periods?

– Are there any implications from these changes?

• Inviting further discussions



© 2009, USC-CSSE 5

Estimation models need upgrading

• It has been 10 years since the release of COCOMO II.2000

– Data collected during 1970 – 1999

• Software engineering practices and technologies are changing

– Process: CMM CMMI, ICM, agile methods

– Tools are more sophisticated

– Advanced communication facility

• Improved storage and processing capability



© 2009, USC-CSSE 6

COCOMO Data Projects Over the Five-year Periods

• Dataset has 255 projects completed between 1970 and 2009

– 161 used in COCOMO II 2000

– 87 collected since 2000

16

32

0

17 20

101

55

14

0

20

40

60

80

100

1970-1974

1975-1979

1980-1984

1985-1989

1990-1994

1995-1999

2000-2004

2005-2009

Periods

# o

f d

ata

pro

ject

s

16

32

0

17 20

101

55

14

0

20

40

60

80

100

1970-1974

1975-1979

1980-1984

1985-1989

1990-1994

1995-1999

2000-2004

2005-2009

Periods

# o

f d

ata

pro

ject

s

7 projects were collected before 2000 but not included in COCOMO II 2000



© 2009, USC-CSSE 7

COCOMO II Formula

• Effort estimate (PM)

– COCOMO II 2000: A and B constants were calibrated using 161 data points with A = 2.94 and B = 0.91

• Productivity =

• Constant A is considered as the inverse of adjusted productivity

EMSizeAPM

SFB**

01.0

EMSize

PMA

SFB*

01.0

PM

Size



© 2009, USC-CSSE 8

Changes in Productivity and Cost Drivers



© 2009, USC-CSSE 9

Average productivity is increasing over the periods• Two productivity increasing trends exist: 1970 – 1994 and 1995 –

2009

1970-1974 1975-1979 1980-1984 1985-1989 1990-1994 1995-1999 2000-2004 2005-2009

Five-year Periods

SL

OC

per

PM

• 1970-1999 productivity trends largely explained by cost drivers and scale factors

• Post-2000 productivity trends not explained by cost drivers and scale factors



© 2009, USC-CSSE 10

Effort Multipliers and Scale Factors

• EM’s and SF’s don’t change sharply as does the productivity over the periods

EA

F

1970- 1975- 1980- 1985- 1990- 1995- 2000- 2005-1974 1979 1984 1989 1994 1999 2004 2009

Su

m o

f S

cale

Fac

tors

1970- 1975- 1980- 1985- 1990- 1995- 2000- 2005-1974 1979 1984 1989 1994 1999 2004 2009

Effort Adjustment Factor (EAF) or ∏EM Sum of Scale Factors (SF)



© 2009, USC-CSSE 11

Changes in Product Factors and Platform Factors

• Testing data (DATA) increases over the years, suggesting an increase in size of the testing data

Product Factors

0.8

0.9

0.9

1.0

1.0

1.1

1.1

1.2

1.2

1.3

0 1 2 3 4 5 6 7 8 9

EM

CPLX

DATA

RELY

• Storage constraint (STOR) is almost constant in the last three periods

• Storage constraint was important during 1970 – 1979, but not during the other periods

1970- 1975- 1980- 1985- 1990- 1995- 2000- 2005-1974 1979 1984 1989 1994 1999 2004 2009

Platform Factors

0.8

0.9

0.9

1.0

1.0

1.1

1.1

1.2

0 1 2 3 4 5 6 7 8 9

EM

STOR

TIME

PVOL

1970- 1975- 1980- 1985- 1990- 1995- 2000- 2005-1974 1979 1984 1989 1994 1999 2004 2009



© 2009, USC-CSSE 12

Personnel Factors

0.6

0.7

0.8

0.9

1.0

1.1

0 1 2 3 4 5 6 7 8 9

EM

APEX

PLEX

LTEX

PCAP

ACAP

Changes in Product Factors and Platform Factors

• Programmer capability is low in the 70-74 period

• Language and platform experience seems improving

• Application experience seems declining

– Programmers change projects more often?

• Tool support gradually increases over the years

– 20% improvement in effort over 40 years

• Multisite development drops sharply in the last period

– Supported by technologies?

1970- 1975- 1980- 1985- 1990- 1995- 2000- 2005-1974 1979 1984 1989 1994 1999 2004 2009

Project Factors

0.6

0.7

0.8

0.9

1.0

1.1

1.2

0 1 2 3 4 5 6 7 8 9

EM

TOOL

SITE

SCED

1970- 1975- 1980- 1985- 1990- 1995- 2000- 2005-1974 1979 1984 1989 1994 1999 2004 2009

Personnel Factors



© 2009, USC-CSSE 13

Constant A decreases over the periods

• Calibrate the constant A while stationing B = 0.91

• Constant A is the inverse of adjusted productivity

– adjusts the productivity with SF’s and EM’s

• Constant A decreases over the periods

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

0 1 2 3 4 5 6 7 8 9

Con

stan

t A

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

0 1 2 3 4 5 6 7 8 9

Con

stan

t A

1970- 1975- 1980- 1985- 1990- 1995- 2000- 2005- 1974 1979 1984 1989 1994 1999 2004 2009

EMSize

PMA

SFB*

01.0

Productivity is not fully characterized by SF’s and EM’s

50% decrease over the post-2000 period

What factors can explain the phenomenon?



© 2009, USC-CSSE 14

Analysis of Productivity Ranges

• Productivity range is the ratio between the highest and lowest rating value for a cost driver

– Represents the degree of impact of the cost driver on effort

• Calibrated SF’s and EM’s by dividing the dataset into five sub-datasets

– 161 projects used in COCOMO II 2000

• completed and collected by 1999– 190 projects completed between 1990 – 2009

– 51 maintenance projects

• completed between 1989 – 2009– 149 new development projects

• completed between 1989 – 2009



© 2009, USC-CSSE 15

PR of Cost Drivers of COCOMO II and all projects in 1990 - 2009

Major changes:

- Increase CPLX

- Decrease Θ DATA, ACAP, PMAT

1.28

1.21

1.21

1.26

1.31

1.74

1.35

2.78

1.73

1.44

1.61

1.37

1.43

1.10

1.29

1.67

1.35

1.48

1.62

1.34

1.52

1.42

1.00 1.50 2.00 2.50 3.00

PMAT

PREC

TEAM

FLEX

RESL

PCAP

RELY

CP LX

TIME

STOR

ACAP

PEXP

LTEX

DATA

RUSE

DOCU

PVOL

AEXP

PCON

TOOL

SITE

SCED

1.41

1.32

1.25

1.26

1.40

1.78

1.49

2.51

1.71

1.44

2.00

1.35

1.41

1.39

1.22

1.61

1.47

1.48

1.51

1.47

1.52

1.43

1.00 1.20 1.40 1.60 1.80 2.00 2.20 2.40 2.60

PMAT

PREC

TEAM

FLEX

RESL

PCAP

RELY

CP LX

TIME

STOR

ACAP

PEXP

LTEX

DATA

RUSE

DOCU

PVOL

AEXP

PCON

TOOL

SITE

SCED

1990 – 2009: including 97 projects used in COCOMO II.2000

COCOMO II.2000

1. Complexity is more influential in the more recent data

2. Analyst capability and process are less influential in the more recent data

3. Testing data is least influential

1. Complexity is more influential in the more recent data

2. Analyst capability and process are less influential in the more recent data

3. Testing data is least influential



© 2009, USC-CSSE 16

1.36

1.30

1.24

1.26

1.28

1.77

1.37

2.71

1.72

1.50

1.67

1.37

1.42

1.16

1.29

1.54

1.39

1.48

1.67

1.34

1.52

1.42

1.00 1.20 1.40 1.60 1.80 2.00 2.20 2.40 2.60 2.80

PMAT

PREC

TEAM

FLEX

RESL

PCAP

RELY

CP LX

TIME

STOR

ACAP

PEXP

LTEX

DATA

RUSE

DOCU

PVOL

AEXP

PCON

TOOL

SITE

SCED

1.34

1.18

1.21

1.26

1.37

1.62

1.47

2.38

1.63

1.40

1.22

1.40

1.43

1.29

1.32

1.65

1.44

1.50

1.53

1.41

1.51

1.44

1.00 1.20 1.40 1.60 1.80 2.00 2.20 2.40 2.60 2.80

PMAT

PREC

TEAM

FLEX

RESL

PCAP

RELY

CPLX

TIME

STOR

ACAP

PEXP

LTEX

DATA

RUSE

DOCU

PVOL

AEXP

PCON

TOOL

SITE

SCED

PR of Cost Drivers of New Development and Maintenance

Major changes:

- Increase DATA, DOCU

- Decrease Θ PCON, ACAP, PCAP, CPLX, PREC

New Development

Maintenance

1. Testing data and documentation are more important in maintenance

2. Personnel capacity, complexity in maintenance is not as critical as in development

1. Testing data and documentation are more important in maintenance

2. Personnel capacity, complexity in maintenance is not as critical as in development



© 2009, USC-CSSE 17

Discussions

• Productivity has doubled over the last 40 years

– But scale factors and effort multipliers did not fully characterize this increase

• Hypotheses/questions for explanation

– Is standard for rating personnel factors different among the organizations?

– Were automatically translated code reported as new code?

– Were reused code reported as new code?

– Are the ranges of some cost drivers not large enough?

• Improvement in tools (TOOL) only contributes to 20% reduction in effort

– Are more lightweight projects being reported?

• Documentation relative to life-cycle needs



© 2009, USC-CSSE 18

Conclusions

• Productivity is generally increasing over the 40-year period

– SF’s and EM’s only partially explain this improvement

• The impact of some cost drivers on effort has changed when calibrating more recent data

– Increased impact of complexity (CPLX)

– Decreased impact of testing data (DATA), Analyst capability (ACAP), and Process capability (PMAT)

• Need for further analysis and calibration using recent data points

an analysis of changes in productivity and cocomo cost drivers over the years

Documents