J e t P r o p u l s i o n L a b o r a t o r y

Jairus HihnJet Propulsion Laboratory, California Institute of Technology

Domain-Oriented Modeling, Estimation and Improvement for Aerospace

November 2, 201126th International Forum on COCOMO

and Systems/Software Cost Modeling

J e t P r o p u l s i o n L a b o r a t o r y

2

Background

JPL is a FFRDC for NASA operated by Caltech JPL is a highly decentralized organization that allows great

autonomy to its project managers at all levels JPL’s Engineering and Science Directorate (3000 people)

has twice been certified at CMMI ML 3 (2007, 2010) Approximately 1000 people ($253M/year budget) in the software

community

This raises the question How do we do it How do we measure it?

We turn process sins into virtues

J e t P r o p u l s i o n L a b o r a t o r y

Data Sources

Data Sources• Metrics Collection at Key Milestones• Software Inventory (2006, 2007, 2009 and 2011)• Process performance measures (Tailoring Record, Work

Product Checklist)• Defect tracking systems (PRS, ISA, AAMS, local databases)• SQI Surveys and Contact Log• Product and Process Quality Assurance Activities• Customer Feedback

Communicate results as widely as possible • State of Software Report• Noontime seminars• Section manager meetings

3

J e t P r o p u l s i o n L a b o r a t o r y

4

Measuring Process Performance

Overall performance is used to establish baselines

Identifying the least performed activities provides a short list from which stakeholders can be engaged to discuss process improvement opportunities.

Metrics use is one of the least performed activities

J e t P r o p u l s i o n L a b o r a t o r y

5

JPL Cost and Planning Models

Collect data at major milestones for mission critical and mission support software Labor intensive requiring numerous interviews

MSL flight software required 30-35 interviews

Enables us to Calibrate cost models Develop effort by role

use to do schedule models but they no longer work

Effort growth from PDR Cost to repair defects Size growth

Enables cost research Data mining models developed with Tim Menzies (WVU)

J e t P r o p u l s i o n L a b o r a t o r y

6

Baselines and Trends

How are we doing? • Trends Suggest Improvements in process discipline• Projects following a disciplined process exhibit below average

budget growth and higher productivity rates

6

Project Flight Software

J e t P r o p u l s i o n L a b o r a t o r y

7

NASA Software Cost improvement Task

Year 1: Establish Process and Product Baseline Product: Report that documents current state of

practice at NASA with set of improvement recommendations and recommended metrics.

 Year 2: Core team works to facilitate the implementation of high

priority recommendations across the Agency through a tailorable Agency-wide process. Institute a cost metrics repository to gather information on cost estimate accuracy and cost estimation processes.

Year 3: Initiate a series of annual reports on NASA software cost

estimation accuracy and cost estimation practices.• Continue to monitor and make recommendations