cmmi for development scampi sm class a appraisal ... appraisal program the software engineering...

Software Engineering InstituteCarnegie Mellon

CMMI®- DEV v1.1/v1.2 – SCAMPI v1.1/V1.2 Class A Appraisal ResultsSoftware Engineering InstituteCarnegie Mellon

Process Maturity Profile

CMMI® For DevelopmentSCAMPISM Class A Appraisal Results 2008 End-Year Update

March 2009

March 2009 Process Maturity Profile - © 2009 by Carnegie Mellon University

March 2009

We could not produce this report

without the support of the organizations

and lead appraisers who reported their

appraisal results to the SEISM.

Our many thanks for their continuing

cooperation with our data collection

and analysis efforts.

CMMI Appraisal Program

The Software Engineering Institute is a federally funded research and development center sponsored by the U.S. Department of Defense and operated by Carnegie Mellon® University

® CMMI, Capability Maturity Model and Carnegie Mellon are registered in the U.S. Patent and Trademark Office by Carnegie Mellon University

SM SCAMPI and SEI are service marks of Carnegie Mellon University

© 2009 by Carnegie Mellon University


CMMI®- DEV v1.1/v1.2 – SCAMPI v1.1/V1.2 Class A Appraisal Results

Outline

Introduction

Current Status

Community Trends

Organizational Trends


Organizational Trends

Summary And Terms used in this Briefing

How to Report your Appraisal Results to the SEI

This briefing is based solely on the appraisals reported to the SEI. It cannot be used as a source for an exact indicator of all organizations in the world using CMMI models or appraisal methods nor can it be used for certification or verification purposes.

The SEI does not certify appraisal results or organizations. The SEI licenses SEI Partners, authorizes/certifies lead appraisers, team leaders and high maturity lead appraisers to conduct appraisals. Neither the SEI nor any other organization is a “certifying authority” of the results from a SCAMPI appraisal. Therefore requests to the SEI to provide information to be used for this purpose can not be fulfilled. Information provided by the SEI, such as this briefing, is to demonstrate reported results of the use of CMMI products and services only.

Organizations performing source selection or verification should consider performing an evaluation appraisal. For more information on appraisal methods and for a directory of authorized lead appraisers who can perform them, visit: http://www.sei.cmu.edu/appraisal-program/



Introduction: Purpose

Characterize the adoption of CMMI® - DEV®

Describe results from Standard CMMI Appraisal Method for Process Improvement (SCAMPISM) v1.1/v1.2 Class A appraisals using Capability Maturity Model® Integration(CMMI-SE/SW/IPPD/SS v1.1 or CMMI - DEV v1.2) *

Encourage continued reporting of results


* Organizations previously appraised against CMMI v1.0 and which have not reappraised against v1.1 or v1.2 are not included in this report

Please visit: http://www.sei.cmu.edu/appraisal-program/profile/index.html for additional information or questions you may have about this briefing before contacting the SEI directly



This briefing includes three primary sections:

1. Current Status- Snapshot of the software community based on the most

recent appraisal, since 2002, of reporting organizations

2. Community Trends- Global distribution of appraisals- Growth in the number of appraisals performed

Introduction - 2: Briefing Contents


- Growth in the number of appraisals performed - Shifts in the maturity profile over time

3. Organizational Trends- Analysis of Process Area (PA) satisfaction - Time to move up in maturity based on change in maturity

level and time between appraisals



Current Status

SCAMPI v1.1/v1.2 Class A appraisals conducted from the SCAMPI V1.1 April 2002 release through and reported to the SEI by

• appraisals• organizations

January 2009

4,1343,446

December 2008


• organizations• participating companies• reappraised organizations• projects• Non-USA organizations

Please refer to: Terms Used in this Report on page 29

Additional charts providing different views and break down of this information will be added to this briefing

as more appraisal data is reported to the SEI over time.

2,544

564

21,141

71.4%

3,446


CMMI®- DEV v1.1/v1.2 – SCAMPI v1.1/V1.2 Class A Appraisal ResultsN

um

ber

Of

Org

an

izati

on

s

1200

1400

1600

1800

2000

Process Maturity Profile by All Reporting Organizations

30.2%

48.3%


Based on most recent appraisal of organizations

Nu

mb

er

Of

Org

an

izati

on

s

0

200

400

600

800

1000

Not Given Initial Managed Defined Quantitatively Managed

Optimizing

3446

7.7%

1.1%

30.2%

2.9%

9.8%



800

1000

1200

Nu

mb

er

Of

Org

an

izati

on

s

Process Maturity Profile (V1.2 Only)by All Reporting Organizations

58.3%


Based on most recent appraisal of organizations1500

0

200

400

600


Optimizing

Nu

mb

er

Of

Org

an

izati

on

s

5.8%

0.8%

28.5%

1.0%

5.6%



Reporting Organization Categories

Contractor for

Commercial/In-house

21.1%

74.1%


Based on organizations reporting an organization category

0 500 1000 1500 2000 2500 3000 3500

Military/Government Agency

Contractor for Military/Government

Number of Organizations

21.1%

4.8%

3446


CMMI®- DEV v1.1/v1.2 – SCAMPI v1.1/V1.2 Class A Appraisal Results%

Of

Org

an

izati

on

s W

ith

in C

ate

go

ry

Process Maturity Profile by Reporting Organization Categories

60%

70%

80%

90%

100%

Commercial/In-house = 2525

Contractor for Military/Government = 718

Military/Government Agency = 163


Based on most recent appraisal of organizations reporting an organization category

% O

f O

rgan

izati

on

s W

ith

in C

ate

go

ry

6.3%

0.9%

29.1%

50.4%

3.4%

9.9%8.6%

1.7%

31.9%

45.1%

1.5%

11.1%

23.3%

1.8%

44.8%

25.8%

0.6%

3.7%

0%

10%

20%

30%

40%

50%


Optimizing

3406



Retail Trade0.5%

Wholesale Trade0.2%

Transportation, Communication, Electric, Gas

and Sanitary Services4.7%

Finance, Insurance and Real Estate9.1%

Public Administration (Including Defense)

5.7%

Fabricated Metal Products0.2%

Primary Metal Industries

Engineering & Management Services36.1%

Organization TypeBased on Primary Standard Industrial Classification (SIC) Code


Primary Metal Industries0.6%

Industrial Machinery And Equipment

1.5%

Electronic & Other Electric Equipment

15.9%

Instruments And Related Products

1.9%

Transportation Equipment4.8%

Other Manufacturing Industries

1.8%

Health Services2.4%

Other Services14.5%

Based on organizations reporting SIC code. For more information visit: http://www.osha.gov/oshstats/sicser.html2652

Manufacturing

17.5%

Services

69.2%



25 or fewer14.1%

201 to 3008.5%

301 to 5006.9%

501 to 10006.0%

1001 to 20003.3%

2000+2.1%

Organization SizeBased on the total number of employees within the area of the organization that was appraised

201 to 2000+

26.9%


26 to 5018.3%

51 to 7513.0%

76 to 1008.2%

101 to 20019.6%

8.5%

Based on organizations reporting size data3407

1 to 100

53.5%

26.9%



Maturity Profile by Organization Size Based on the total number of employees within the area of the organization that was appraised

% o

f O

rgan

izati

on

s b

y S

ize

59

.5%

51

.8% 55

.7%

57

.5%

53

.1%

51

.7%

50

.7%

60%

70%

80%

90%

100%


Based on organizations reporting size data

% o

f O

rgan

izati

on

s b

y S

ize

8.4

%

6.1

%

6.1

% 8.2

%

8.7

%

8.3

%

4.7

% 8.3

%

12

.5%

8.5

%

1.5

%

0.6

%

1.1

%

1.3

%

2.1

%

0.4

%

1.0

%

2.7

%

39

.0%

29

.6%

23

.2%

22

.9%

25

.9%

19

.5%

10

.7%

4.5

%

1.4

%

28

.8%

51

.8%

53

.1%

46

.2% 5

1.7

%

44

.2%

42

.9%

50

.7%

0.4

%

1.0

%

2.5

%

3.6

%

3.7

%

5.5

%

4.7

%

5.8

%

1.8

%

2.8

%

1.5

%

1.4

% 5.0

% 7.5

% 10

.2%

12

.1%

19

.1%

30

.1%

35

.7%

36

.6%

0%

10%

20%

30%

40%

50%

25 or fewer 26 to 50 51 to 75 76 to 100 101 to 200 201 to 300 301 to 500 501 to 1000 1001 to 2000 2000+

Not Given Initial Managed Defined Quantitatively Managed Optimizing

3407



Contractor for

Commercial/In-house

USA and Non-USAReporting Organization Categories

318

2207

555


Based on USA organizations and Non-USA organizations reporting an organization category

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Military/Government Agency

Contractor for Military/Government

% of Organizations

USA: 100% = 979

Non-USA: 100% = 2427

2427985

555

163

106

57



60%

70%

80%

90%

100%

USA: 100 % = 985

Non-USA: 100 % = 2461

% o

f O

rgan

izati

on

s

Maturity Profile by All Reporting USA and Non-USA Organizations

1284


Based on USA organizations and Non-USA organizations

0%

10%

20%

30%

40%

50%


Optimizing

% o

f O

rgan

izati

on

s

985 2461

134

129

14 24

362

679

382

1284

990

84255



Appraisals conducted through and reported to the SEI

• 67 Countries reported appraisals- most of the appraisals are from Asia and North America

December 2008

Section 2: Community Trends


• Appraisals were available on PARS- the expired ones were removed and the reappraised were

replaced by the newest appraisals


3081



Countries where Appraisals have been Performed and Reported to the SEI


Red country name: New additions with this reporting

Argentina Australia Austria Bahrain Bangladesh Belarus Belgium Brazil

Bulgaria Canada Chile China Colombia Costa Rica Czech Republic Denmark

Dominican Republic Egypt Finland France Germany Greece Hong Kong Hungary

India Indonesia Ireland Israel Italy Japan Korea, Republic Of Latvia

Lithuania Luxembourg Malaysia Mauritius Mexico Morocco Nepal Netherlands

New Zealand Norway Pakistan Panama Peru Philippines Poland Portugal

Romania Russia Saudi Arabia Singapore Slovakia South Africa Spain Sri Lanka

Sweden Switzerland Taiwan Thailand Turkey Ukraine United Arab Emirates United Kingdom

United States Uruguay Viet Nam



Number of Appraisals Reported to the SEI by Continent

Nu

mb

er

of

Ap

pra

isals

1901

13281400

1600

1800

2000

2200

2400


Based on appraisals

Nu

mb

er

of

Ap

pra

isals

4134

47

536

1328

33

289

0

200

400

600

800

1000

1200

1400

Africa Asia Europe North America Oceania South America



Number of Appraisals and Maturity LevelsReported to the SEI by Country

Country

Number of

Appraisals

Maturity

Level 1

Reported

Maturity

Level 2

Reported

Maturity

Level 3

Reported

Maturity

Level 4

Reported

Maturity

Level 5

Reported Country

Number of

Appraisals

Maturity

Level 1

Reported

Maturity

Level 2

Reported

Maturity

Level 3

Reported

Maturity

Level 4

Reported

Maturity

Level 5

Reported

Argentina 64 45 12 2 3 Malaysia 56 20 31 5

Australia 32 1 7 5 2 4 Mauritius 10 or fewer

Austria 10 or fewer Mexico 57 24 25 3 4

Bahrain 10 or fewer Morocco 10 or fewer

Bangladesh 10 or fewer Nepal 10 or fewer

Belarus 10 or fewer Netherlands 10 or fewer

Belgium 10 or fewer New Zealand 10 or fewer

Brazil 106 1 50 42 1 9 Norway 10 or fewer

Bulgaria 10 or fewer Pakistan 25 1 18 4 1

Canada 51 1 12 22 5 3 Panama 10 or fewer

Chile 30 17 10 2 Peru 10 or fewer

China 745 1 117 540 27 41 Philippines 21 2 11 7

Colombia 22 7 11 1 2 Poland 10 or fewer


Colombia 22 7 11 1 2 Poland 10 or fewer

Costa Rica 10 or fewer Portugal 10 or fewer

Czech Republic 10 or fewer Romania 10 or fewer

Denmark 10 or fewer Russia 10 or fewer

Dominican Republic 10 or fewer Saudi Arabia 10 or fewer

Egypt 34 1 17 11 2 2 Singapore 19 3 10 1 4

Finland 10 or fewer Slovakia 10 or fewer

France 141 4 81 45 1 2 South Africa 10 or fewer

Germany 64 9 32 11 1 1 Spain 105 1 60 35 2 4

Greece 10 or fewer Sri Lanka 10 or fewer

Hong Kong 18 2 11 5 Sweden 10 or fewer

Hungary 10 or fewer Switzerland 10 or fewer

India 409 14 191 24 166 Taiwan 117 1 74 38 2

Indonesia 10 or fewer Thailand 27 12 13 1

Ireland 10 or fewer Turkey 14 12 2

Israel 17 3 10 2 Ukraine 10 or fewer

Italy 31 14 14 United Arab Emirates 10 or fewer

Japan 267 17 75 121 13 16 United Kingdom 93 3 42 30 1 3

Korea, Republic Of 138 1 47 61 13 7 United States 1272 27 448 462 21 124

Latvia 10 or fewer Uruguay 10 or fewer

Lithuania 10 or fewer Viet Nam 12 9 1 2

Luxembourg 10 or fewer



896917

800

1000

1200

TOTAL

POSTED ON PARs

Number of SCAMPI A Appraisals Reported to SEI by Year

Nu

mb

er

of

Ap

pra

isals

842

1029

988


76

214

363

609

0

200

400

600

2002 2003 2004 2005 2006 2007 2008

Based on appraisals

Nu

mb

er

of

Ap

pra

isals

52

186

437

600

4134



Number of SCAMPI A Appraisals Reported to SEI by Year

Nu

mb

er

of

Ap

pra

isals 834

1006

733

849

752

109

180

236

800

1000

1200

First Appraisal

Reappraisal


Based on appraisals

Nu

mb

er

of

Ap

pra

isals

52

186

437

598

51

163

394

504

1

23

43

96

0

200

400

600

2002 2003 2004 2005 2006 2007 2008

4134



1112

1845

2694

3446

Trends in the Community Maturity Profile

% o

f O

rgan

izati

on

s

Year Orgs

48.3

%

60%

70%

80%

90%

100%

2002-2005

2006

2007

2008


Based on a cumulative view of the most recent appraisals of organizations up through the year indicated

% o

f O

rgan

izati

on

s

9.2

%

2.2

%

32.9

%

33.1

%

4.4

%

18.2

%

9.0

%

1.5

%

32.0

%

37.3

%

4.5

%

15.7

%

8.6

%

1.3

%

32.5

%

41.8

%

3.4

%

12.4

%

7.7

%

1.1

%

30.2

%

48.3

%

2.9

%

9.8

%

0%

10%

20%

30%

40%

50%

Not Given Initial Managed Defined Quantitatively Managed Optimizing



Section 3: Organizational Trends

Appraisals conducted through and reported to the SEI

• Process Area (PA) profiles- satisfaction of PAs by maturity level for organizations

appraised at levels 1 and 2

• reappraised organizations564

1289

December 2008


• reappraised organizations- accounting for appraisals- although some organizations conducted multiple

reappraisals, only the first and latest appraisals were used in creating the charts on pages 25


5641,252



OPD

OT

IPM

RSKM

ISM

DAR

OPP

QPM

OID

CAR

Fully Satisfied

Rated

Process Area Profiles - 1Organizations Appraised at Maturity Level 1

Defined

QuantitativelyManaged

Optimizing


0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

REQM

PP

PMC

SAM

MA

PPQA

CM

RD

TS

PI

VER

VAL

OPF

Based on appraisals reporting a maturity level 1 rating

% of Appraisals

Managed

72



RSKM

IT

ISM

DAR

OEI

QPM

OPP

Fully Satisfied

Rated

Quantitatively Managed

Process Area Profiles - 2Organizations Appraised at Maturity Level 2


Based on appraisals reporting a maturity level 2 rating

0% 1% 2% 3% 4% 5% 6% 7% 8% 9% 10%

RD

TS

PI

Ver

Val

OPF

OPD

OT

IPM

Defined

% of Appraisals

1289



52.1

%

60%

70%

80%

90%

100%

First

Latest

Maturity Level ofFirst and Latest Appraisal

Org

an

izati

on

s


11.0

%

5.9

%

40.4

%

27.7

%

3.2

%

11.9

%

7.4

%

0.4

%

8.2

%

52.1

%

4.6

%

27.3

%

0%

10%

20%

30%

40%

50%


Optimizing

Based on reappraised organizations using their first and latest appraisal

% o

f O

rgan

izati

on

s

564



Time to Move Up

Number of months to move to nextmaturity level

Largest observed value that is not an outlier

50

30

40


5

75th Percentile

25th Percentile

Median

Smallest observed value that is not an outlier

20

10

0

All (2002 to Present)

1 to 2 2 to 3 3 to 4 4 to 517 204 34 25

2006 to Present

1 to 2 2 to 3 3 to 4 4 to 55 90 11 8

19

Time Period of Initial Appraisal

LevelOrgs

1819

13

2002 to 2005

1 to 2 2 to 3 3 to 4 4 to 512 144 23 17

19

4

15

9

19

4

14



Summary - 1

4134 appraisals have been reported to the SEI in 82 months

The Organization Category of “Commercial/In-house” reports most of the appraisals

The highest percentage of “Commercial/In-house” reporting appraisals is from outside the USA


The highest percentage of “Military/Government Agency” reporting appraisals is from the USA

Outside the USA: China, India, Spain, Brazil, Thailand and France reported appraisals are increasing at a rapid rate



Summary - 2

For organizations that began their CMMI-based SCAMPI effort in

2002 or later, the median time to move from:

• maturity level 1 to 2 is 4 months






Note:

Since the September 2008 report, the median time to move from maturity level 2to 3 increased from 17 months to 18 months, the median time to move from maturity level 3 to 4 increased from 15.5 months to 19 months, and the median time to move from maturity level 4 to 5 increased from 12.5 months to 13 months



Terms Used in this Report

Company - Parent of the appraised entity

A company can be a commercial or non-commercial firm, for-profit

or not for-profit business, a research and development unit, a higher

education unit, a government agency, or branch of service, etc.

Organization - a.k.a. Appraised entity

The organization unit to which the appraisal results apply. An

appraised entity can be the entire company, a selected business


appraised entity can be the entire company, a selected business

unit, units supporting a particular product line or service, etc.

Non-USA - Appraised entity whose geographic location is not within the United

States. The parent of the appraised entity may or may not be based

within the United States.

PARS - Published Appraisal Results webpage contains CMMI® appraisals

published at the request of the appraisal sponsor.

The link is: http://sas.sei.cmu.edu/pars/



Report your Appraisal Results to the SEI

The briefing is only possible due to the cooperation of organizations and individuals sending in their appraisal results to the SEI

In order to provide this information and service in the future, it will depend on this continued cooperation


Please visit:

http://www.sei.cmu.edu/appraisal-program/profile/index.html

for forms, information, and instructions on how to report appraisals to the SEI



Contact Information

Please visit:

http://www.sei.cmu.edu/appraisal-program/profile/index.html

and review the information provided before contacting:

SEI Customer Relations (412) 268-5800SEI FAX number (412) 268-5758


SEI FAX number (412) 268-5758

Internet [email protected]

Mailing AddressCustomer RelationsSoftware Engineering InstituteCarnegie Mellon UniversityPittsburgh, PA 15213-3890

cmmi for development scampi sm class a appraisal ... appraisal program the software engineering...

Documents