COSYSMO 2.0 Workshop Summary(held Monday, March 17th 2008)

USC CSSE Annual Research ReviewMarch 18, 2008

Jared Fortune

2

Agenda• Topics covered

– Overview of COSYSMO– Summary of COSYSMO 2.0 improvements– Reuse (overview)– Integration between SWE and SysE– Risk modeling (overview)– Assumption of linearity in cost drivers– Modeling organizational factors in space systems– Best practice guidance– Cost drivers vs. scale factors– Cost driver impact survey results– Recursive levels of design

• Joint meeting with SoS cost estimation group

3

Attendees (18)

• Jeff Allen (Lockheed Martin)*

• Barry Boehm (USC)

• Jim Cain (BAE)

• Jared Fortune (USC)

• Gary Hafen (Lockheed Martin)

• JoAnn Lane (USC)

• Dan Ligett (Softstar)

• Miles Nesman (Boeing)

• Ali Nikolai (SAIC)

• Cynthia Nikolai (Notre Dame)

• Elizabeth O’Donnell (Boeing)

• Garry Roedler (Lockheed Martin)

• Rick Selby (Northrop Grumman)

• Howard Schimmdler (Lockheed Martin)*

• Ricardo Valerdi (MIT)*

• Gan Wang (BAE)

• Darryl Webb (Aerospace)

• Marilee Wheaton (Aerospace)

* Via Phone

4

Recommended COSYSMO 2.0 Improvements

1. Reuse

2. Integration of SwE & SysE estimation

3. Assumption of linearity in COSYSMO cost drivers

4. Effect of cost drivers and scale factors

5. Number of recursive levels of design

6. Risk modeling

7. Establishing best practice guidance

8. Consideration of SoS scope in COSYSMO

9. Estimation in Operation & Maintenance Phase

10. Requirements volatility

5

Reuse• What is the effect of reuse in estimating systems

engineering size/effort?• On-going efforts by Valerdi, BAE, and Lockheed

– Definitions: New, Modified, Adopted, Deleted, Managed

• Incorporate reuse into COSYSMO– Collect data to obtain statistical significance– Opportunistic vs. systematic reuse

• Select exemplar program to support reuse definitions• Assist in development of best practice guide

– Ensure consistency across organizations

6

SWE and SysE Integration• What is the overlap between COCOMO II and

COSYSMO?– Not model integration, scope overlap

• WBS element ownership exercise– Attempt at capturing SysE effort– System, Software, Supportability, PEM; in COSYSMO or not

• COSYSMO data covers a broader scope that textbook definition of “systems engineering”

• Sometimes to distinguish who is responsible (funds) specific WBS elements– Areas of overlap

7

Risk Modeling• How can risk associated with the COSYSMO estimate

be quantified?• On-going effort at Lockheed with COSYSMOR• Provide Low, Likely, High estimates• Presented one methodology, will be undergoing peer

review• Illustrated need for best practice guide to reduce

unnecessary (incorrect) risk assessments• To be incorporated into COSYSMO 2.0 baseline (Valerdi

book)

8

Linearity Across the Life Cycle• Do all cost drivers have the same impact across the

lifecycle?• Values of some drivers may not exist in all phases• Single “step” between driver values can be very large

– Raised possibility of weighting drivers

• Subgroup will discuss further, report results at next meeting

9

Modeling Organizational Factors• What are the quantifiable organizational issues that drive

cost?• Help explain why a predicted cost is reasonable• Highlight potential biases in estimates• Presented data from STS and ISS• Results provided strong indicators that program

complexity and organizational structure are major cost drivers

10

Best Practice Guidance• How can misuse of the COSYSMO cost drivers be

avoided?• Correct assessment of values for cost drivers

• Limit underestimation• Reuse

• Ensure consistency across programs• Created Committee

• Will present results at next meeting

11

Cost vs. Size Driver Correlation• Can some of the cost drivers become scale factors in the

cost estimating relationship calibrated by the new data set?

• Presented results of October 2007 survey• Distributed new survey• Result: “it depends”

12

Recursive Levels• How can the integration complexity of system elements

one layer below the system-of-interest be operationalized?

• Generated a lot of discussion, but a long way to go on guidance

• Attempt to incorporate in best practice guide

13

Prioritization

Improvements

Reuse

SWE &

SysE

Linea

rity

Cost D

river

s & S

cale

Fac

tors

Rec, L

ev, o

f Des

ign

Risk

Model

ing

Best P

ract

ice

Guide

SoS in C

OSYSMO

O&MReq

. Vola

tility

Cri

teri

a

Availability of Data M L L L H LImpact on TOC H H H H M MFrequency of Use H H H H H LCompatible with Models H L L M L MAddress Trends L L L M M MFactor Interactions L H H L H LPriority H M+ M M M H M+ L+ L L

Cri

teri

a