© The Aerospace Corporation 2009

Overview of Affordability Assessment Approach

November 3, 2009

Dr. David BeardenThe Aerospace Corporation

2

Overview

• The Aerospace Corporation frequently asked to assess the cost and schedule realism of projects at different points in the project lifecycle

• Tools used to evaluate numerous proposals and conduct Independent Cost Estimates (ICEs) for a variety of NASA missions and programs

• Different cost and schedule estimating methods apply depending on stage of the project

• Suite of tools exists to assess cost and schedule based upon available information

• These analyses form the basis for strategic analysis of affordability (so-called “Sand Charts”) providing assessment of projects within a portfolio of missions

3

Agenda

• Overview

• Magnitude of Cost and Schedule Growth– Evolution of Concepts Over Time

• Assessing Project Cost and Schedule– Multiple Methods at Various Levels

• Assessing Program Affordability– Sand Chart Tool Overview

• Application, Observations & Challenges– Decadal Support and Review of HSF Plans Committee

4

0.0

0.5

1.0

1.5

2.0

2.5

3.0

0.0 0.5 1.0 1.5 2.0 2.5 3.0

Cost Ratio

Sche

dule

Rat

io

within cost and schedule

within 100% overrun

Forty NASA Robotic Science Missions Experienced 27% Cost and 22% Schedule Growth During Development

5

Reasons for Growth - Study of 40 NASA Missions: Internal versus External Factors Driven-Growth

• Internal Growth (within Project’s control)

– Technical• Spacecraft development difficulties • Instrument development difficulties• Test failures• Optimistic heritage assumptions

– Programmatic• Contractor management issues• Inability to properly staff an activity

• External Growth (outside Project’s control)

– Launch vehicle delay– Project redesign– Requirements growth– Budget constraint– Labor strike– Natural disaster

External Only

20.0%

Both10.0%

No Growth12.5%

Internal Only

57.5%

Distribution of Growth

S/C Only22.2%

Other14.8%

Inst. Only33.3%

Both Inst & S/C29.6%

Launch83.3%

Other16.7%

Distribution of Internal Growth

Distribution of External Growth

6

77 Historical NASA Projects Demonstrated ~51% Cost Growth from Formulation Start to Launch

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

‐100% ‐50% 0% 50% 100% 150% 200% 250% 300%

Cost Growth 

Cumulative Prob

ability

Historical data‐77 missions

Lognormal "Historical data ‐77 missions"

Mean historical cost growth

A

A B

B

A 51% average cost growth corresponds to mean of lognormal approximation to data. Mean = 51% historical cost growth.

B0% average cost growth corresponds to 17% confidence on lognormal approximation to data. 17%-tile = 0 % historical cost growth (no reserves).

Cumulative Probability Function of Cost Growth for 77 Historical NASA Projects (Human Space Flight (HSF), Non-HSF, Ground Ops)

(Includes cost and schedule uncertainty effects )

7

Evolution of Mission Concepts Over Time Can Lead to Estimation of a “Moving Target” at Each Milestone

• Potential causative factor for cost growth is evolution of mission over time

• Results in underestimation of technical specifications such as mass, power, data rate, and complexity of a system

• Underestimation of system resources leads to underestimation of the cost of the mission

• Success oriented schedules often shorter than historical comparisons would indicate

• Cost estimators, in effect, trying to estimate “moving target” as requirements evolve and system resources grow

8

Summary of Ten NASA Science Missions Show that Mass Growth Exceeds Typical Reserve Guidance

• Average mass growth (43%) for ten missions exceeds typical guidelines of mass reserves (30% over CBE) at start of Phase B

9

Realization of Cost Growth Occurs Primarily After CDR as Programmatic Baseline Catches Up to Technical Baseline

• Average cost growth for ten missions studied is 76% over baseline with reserves (and 113% over baseline without reserves)

10

Example: Substantial Differences Exist between Science Definition Team (SDT) and Final Configuration

STEREO STEREOProgrammatics SDT Final

Schedule (months) 40 70Launch Vehicle Taurus Delta II

TechnicalMass (kg)

Satellite (wet) 211 612Spacecraft (dry) 134 414Payload 69 133

Power (W)Satellite (Orbit Average) 152 515Payload (Orbit Average) 58 108

OtherTransponder Power (W) 20 60Downlink Data Rate (kbps) 150 720Data Storage (Gb) 1 8

SDT Configuration

Final Configuration

Illustrations reprinted courtesy of NASA

11

Typical Cost-risk Analyses Won’t Capture Large Changes During Concept Evolution

0%10%20%30%40%50%60%70%80%90%

100%

$200 $250 $300 $350 $400 $450 $500 $550 $600 $650 $700 $750 $800Total Mission Cost (FY07$M)

Perc

ent L

ikel

ihoo

d of

Com

plet

ing

at C

ost

SDT DesignFinal Design

$300M Estimate with 20% Reserve

Final Cost$550M

12

Schedule as Function of Complexity y = 24.22e1.6479x

R2 = 0.6889

01224364860728496

108120132

0% 20% 40% 60% 80% 100%

Complexity Index

Dev

elop

men

t Tim

e (m

onth

s)

Successful

Failed

Impaired

To-be-determinedSTEREO @ SDT

STEREO @Launch

Effect of Increased Complexity on Development Time:STEREO Complexity Increased from 40% to 60%

13

Agenda

• Overview

• Magnitude of Cost and Schedule Growth– Evolution of Concepts Over Time

• Assessing Project Cost and Schedule– Multiple Methods at Various Levels

• Assessing Program Affordability– Sand Chart Tool Overview

• Application, Observations & Challenges– Decadal Support and Review of HSF Plans Committee

14

Primary Tenants of Cost Estimating to Establish Robust Estimates

• Use Multiple Methods– Ensures that no one model/database biases the estimate

• Industry Standard Methods• Aerospace Developed Models

• Use Analogy Based Estimating– Ties cost to systems that have been built with known cost– Allows contractor specific performance to be addressed– Forces estimator and project to look at cost and complexity of new

concepts with respect to previously built hardware

• Use Both System Level and Lower Level Approaches– Ensures that lower level approaches do not omit elements or

underestimate overall cost relative to system level complexity

15

Analogies Anchor to Actual Cost Data Adjusted Based on Functional Cost Estimating Relationships

• Use analogies as the basis for an estimate• Every historical program has “unique” aspects that affect cost• Use multiple analogies to average out impacts of unique

aspects

X+Cost of analogy system

CER estimate of new system

Analogy-based estimate

Input Variable

Cost CERO*

CER estimate of analogy system

16

$0

$20

$40

$60

$80

$100

$120

$140

Inst

rum

ent C

ost F

Y05$

M

EstimateAdj. AnalogiesModel

Cost Risk Process Uses Multiple Methods to Provide a Distribution of Possible Outcomes

.

+

+..

WBS Element 1

WBS Element 2

Total Cost

Total Distribution is a Combination of CostDistributions for WBS Elements

Example of Multiple Cost Estimates

Example Cost “S-Curve” Distribution

Optimistic(Minimum of

Viable Estimates)(Average of Estimates)

Pessimistic(Highest Viable Estimate+ Design Maturity Factor)

L M H$

Triangular Distribution

Most Likely

DEN

SITY

L M H

0%10%20%30%40%50%60%70%80%90%

100%

$180 $190 $200 $210 $220 $230 $240 $250 $260 $270 $280Total Mission Cost (RY$M)

Perc

ent L

ikel

ihoo

d of

C

ompl

etin

g at

Cos

t

17

Agenda

• Overview

• Magnitude of Cost and Schedule Growth– Evolution of Concepts Over Time

• Assessing Project Cost and Schedule– Multiple Methods at Various Levels

• Assessing Program Affordability– Sand Chart Tool Overview

• Application, Observations & Challenges– Decadal Support and Review of HSF Plans Committee

18

Inadequate Budget for One Project Results in a Domino Effect for Other Projects in Program Portfolio

$0

$50

$100

$150

$200

1999 2000 2001 2002 2003 2004 2005 2006

MMSSTEREOSolar-BTIMED

$0

$50

$100

$150

$200

1999 2000 2001 2002 2003 2004 2005 2006

MMSSTEREOSolar-BTIMED

STP 2000 Planned Funding STP Actual Funding History

Total Program Funding 1999-2006• Planned = $690M• Actual = $715M

Although the total program funding remained consistent over this time period, implementation of successive missions substantially affected

Although the total program funding remained consistent over this time period, implementation of successive missions substantially affected

MMS

STEREO

MMS

STEREO

19

Affordability Analysis (“Sand Chart”) Needed to Support Long Term Decision Making Process

• Simulate portfolio of related projects, based on the individual projects’cost-risk and program funding constraints

– Monte-Carlo analysis, using S-Curves, simulates cost growth– Algorithms derived from historical behavior of past projects– Schedule interdependencies combined with individual project cost growth

• Portfolio forced to fit within available funding – Shift start of projects, modify duration between milestones, or eliminate

projects to create new plan within available funding – Projects with significant fixed costs (e.g. workforce/facility requirements),

may see increased cost if forced to stretch over longer duration– Cost reductions and milestone delays can lead to increased costs due to

inefficiencies (e.g. contract modifications, workforce costs, etc.)

• Cost and schedule outputs probabilistic– Typically reported at 65-70% confidence level

20

Designed to Evaluate Scenarios for Given Program/ Portfolio or Develop Plan Given Individual Elements

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2019 2021 2023 2025 2027 2029

Date

Perc

ent L

ikel

ihoo

d

BaselineNo HSF Gap

3-4 year delayin HLR

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2019 2021 2023 2025 2027 2029

Date

Perc

ent L

ikel

ihoo

d

BaselineNo HSF Gap

3-4 year delayin HLR

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

4000 6000 8000 10000 12000 14000 16000Project Cost, $M

Con

fiden

ce L

evel

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

4000 6000 8000 10000 12000 14000 16000Project Cost, $M

Con

fiden

ce L

evel

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

4000 6000 8000 10000 12000 14000 16000Project Cost, $M

Con

fiden

ce L

evel

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

4000 6000 8000 10000 12000 14000 16000Project Cost, $M

Con

fiden

ce L

evel

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

4000 6000 8000 10000 12000 14000 16000Project Cost, $M

Con

fiden

ce L

evel

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

4000 6000 8000 10000 12000 14000 16000Project Cost, $M

Con

fiden

ce L

evel

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

4000 6000 8000 10000 12000 14000 16000Project Cost, $M

Con

fiden

ce L

evel

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

4000 6000 8000 10000 12000 14000 16000Project Cost, $M

Con

fiden

ce L

evel

Input:baseline plan, cost likelihood curves, program linkages

Perform Monte Carlo probabilistic analysis

Output:schedule likelihood curves, # of missions complete, etc.

Apply Stretch Algorithms

Input:baseline plan, program linkages

Output:New strawmanplan that fits budget available

Evaluation Planning

0

500

1000

1500

2000

2500

2010 2012 2014 2016 2018 2020 2022 2024

RY$

M

0

500

1000

1500

2000

2500

2010 2012 2014 2016 2018 2020 2022 2024

RY$M

21

Agenda

• Overview

• Magnitude of Cost and Schedule Growth– Evolution of Concepts Over Time

• Assessing Project Cost and Schedule– Multiple Methods at Various Levels

• Assessing Program Affordability– Sand Chart Tool Overview

• Application, Observations & Challenges– Decadal Support and Review of HSF Plans Committee

22

Observations & ChallengesReview of HSF Committee and Decadal Support • Decadal Survey Support

– Objective is to apply a uniform and historical data informed affordability analysis to each of the activities examined

– Historical mass, power & schedule growth used to assess potential cost “threats”

– Limited data for ground based projects– Limited project/activity interactions – due to timeline and scope

• Review of HSF Plans Committee Work – Objective to apply uniform and historical data informed affordability

analysis methodology to each option at a high level– Traditional cost estimates not performed– Need to compare in-development systems with “paper concepts”; some

project elements had little design analysis performed– Historical cost growth from project formulation (SDR or Phase B) while

many project elements considered had not reached SDR

23

Decadal Support Flow Diagram

Request for Information (RFI)

To Normalize Data

Across Concepts

Use extensive parametric estimating tools and empirical databases of technical and programmatic information. Cross-checking with Analogies. Seek Committee buy-in on Analogies

Develop

Funding profile

from consensus

cost and schedule

estimate

Support of Survey Committee’s Steering Group Meetings

Develop Quad Summary charts with overall evaluation Roll-up and Risk Rating

Develop funding profiles & Budget Analysis

Gather Information Estimate the Cost and

Schedule for Missions

Summary & Comparison of Data

Support of Survey Committee

EvaluateSpacecraft/FlightSystems,Mission/ProjectDesign, Instruments,Ground systemstechnology readiness

Assess Technical Readiness

Reporting

Final report of Quad charts and applicable document and analysis

24

Decadal Technical, Cost and Schedule Assessment

Concepts

Evaluate Responses

Concept MaturityImplementationDevelopment

HeritageContingency

AnalogiesModelsReserve

Cost EstimateCost Threats

Project PhasesCritical PathAnalogies

Technical

Cost

Schedule

Concept Risks Leveled Technical RatingMultiple Iterationsand Interaction

Double S-Curve Bounds Cost ThreatsMass maturity impact on LV CostSchedule slippage impact on CostMass & Power reserve impact on Cost

25

1. What should be the future of the Space Shuttle?

2. What should be the future of the International Space Station (ISS)?

3. On what should the next heavy-lift launch vehicle be based?

4. How should crews be carried to low-Earth orbit?

5. What is the most practicable strategy for exploration beyond low-Earth orbit?

Future of U.S. Human Spaceflight in the UpcomingDecades Formulated in Terms of Five Key Questions

Reprinted courtesy of NASA

26

Summary of Integrated Options Evaluated by Review of HSF Committee

27 Used with Permission of Review of US Human Space Flight Plans Committee

Integrated Option Decision Analysis Tree

Constrained to the budget?

Content? Strategy?

Launch system?

Option 4A

Constrained Less Constrained

Option 2Option 1

PoR ISS+ to2020

ISS+

Moon

Moon First Flexible Path first

Launch system?

Option 5AOption 4B Option 5B Option 5C

Moon

ISS+

Moon

ISS+

Moon

ISS+

Flexible Path

ISS+

Flexible Path

ISS+

Flexible Path

Ares V Lite SDV Ares V Lite SDVEELVderived

Option 3

Moon

Ares V

28

Methodology Overview: Review of Human Space Flight (HSF)

1. Point Estimate Without Reserve

2. Cost-Risk Curve Informed byHistorical Program Experience 3. Starting Project Budget Profiles and

Schedule Dependencies

4. Assembled Program Architecture Budgets

5. Program Budget Under Funding Constraints

29

Source of Point Estimates

Original point

estimate, no reserves

Adjustment to accommodate

use in alternative architectures.

1. Program of Record (POR) ElementsBasis of Estimate

• PMR '08 Rev 1B, without reserves, for Phase B or later projects

• PMR’09 without reserves, for pre-phase B projects Point estimate, without reserves, adjusted to account for additional work required to accommodate interface changes for new architectures

2. Non-POR Elements

Review of Basis of Estimate• Iterative discussions on basis of estimate and

assumptions with NASA• Pedigree review of source data• Comparison to analogy or reference data

Basis of Estimate • Related program data (e.g. EELV)• Historical analogy data• Prior NASA studies on similar or related

program elements• HSF Committee discussion

Point Estimate to Cost Risk%

Lik

elih

ood

$

Point estimate, without reserves.

S-curve is derived from historical cost-growth factors.

17%-tile confidence is aligned with point estimate

Mean is aligned to average cost growth (51%). < 51% historical cost growth from PMR09 cost-to-go for POR elements past Phase B.

1

17th %-tile = 0 % historical cost growth = no reserves3

2

Mean4

3

Point cost estimate without reserves. Confidence level is indeterminate.

2

4

1 S-curve is derived from historical cost-growth factors, based on effects of cost and schedule risk realized with historical projects.

• Point estimates for POR elements past Phase B start from PMR08 Rev 1B • Point estimates for POR elements before Phase B start from PMR09• Point estimates for non-POR elements from NASA, Aerospace, and

Committee sources

31

Establish starting budget profiles for each project in program, and assemble program

• Assemble project funding wedges into architecture / program • Schedule dependencies establish precedence and constraints• Schedule unaffected by funding availability (unconstrained budget)

$ / Y

rYear

1

2

34

$ / Y

r

Year

Area under curve = point estimate without reserves

Year

1

23

4

$ / Y

r

Year

Area under curve = point estimate without reserves$

/ Yr

Year

Area under curve = point estimate without reserves$

/ Yr

Year

Area under curve = point estimate without reserves

12 3 4

Project Schedule Dependencies

Project Budget Profiles

+

32

Constrain to Budget Scenario

Year

$ / Y

r• Monte-Carlo based on cost risk curve for each project element• Program dependencies and funding constraints determine project phasing• Total cost increases to accommodate inefficiencies with adjusted schedule• Outputs: realized cost/schedule, cost and schedule confidence S-curves

Funding Constraint

12 3

4

% L

ikel

ihoo

d

Cost & Months From Start

1

% L

ikel

ihoo

d

Cost and Months From Start

2

% L

ikel

ihoo

d

Cost and Months From Start

3

% L

ikel

ihoo

d

Cost and Months From Start

4

% L

ikel

ihoo

d

Cost & Months From Start

Program

33

Crew Module

LEO Launch VehicleHeavy Launch Vehicle

Lander

Ground Ops

Other Program elementsLunar Surface

Systems

Other Non-Program elements

ISS

STS

0

2,000

4,000

6,000

8,000

10,000

12,000

14,000

16,000

18,000

20,000

FY09 FY10 FY11 FY12 FY13 FY14 FY15 FY16 FY17 FY18 FY19 FY20 FY21 FY22 FY23 FY24 FY25 FY26 FY27 FY28

FY09 FY10 FY11 FY12 FY13 FY14 FY15 FY16 FY17 FY18 FY19 FY20 FY21 FY22 FY23 FY24 FY25 FY26 FY27 FY28 FY29 FY30

Notional Sand Chart Used for Program Assessment

Shuttle Retirement ISS Retirement

Program Initial Capability (IC) dates shown as range from 50th to 80th percent likelihood LEO Capability HLR

Outpost

Total Program Cost shown at 65th percent likelihood. Calculated project cost in excess of initial point estimate, is aggregated and redistributed to each project proportionally.

$ in

M

ScheduleOutputs

Constraints

Two Budget Scenarios: FY10 and “Less Constrained”IC = development, test flight, production, and

fixed costs through first operational flight

34

Summary – Re-cap

• Initial project estimates may be unreliable due to design and technology immaturity and inherent optimism

• While estimates become more accurate as project matures, the greatest growth manifests itself late in project development

• Methods exist to estimate cost and schedule at the conceptual phase albeit with some level of uncertainty

• Cost estimating process includes use of multiple methods, adjusted analogies, and system level and lower level approaches

• These analyses feed strategic analysis (“Sand Chart”) capabilities providing a long term, robust assessment of projects and their effect on a portfolio of programs and missions

• Decadal Study and Review of HSF Plans Committee support demonstrate application of and challenges with program-level affordability analysis

Backup

36

1) Bearden, David A., “A Complexity-based Risk Assessment of Low-Cost Planetary Missions: When is a Mission Too Fast and Too Cheap?”, Fourth IAA International Conference on Low-Cost Planetary Missions, JHU/APL, Laurel, MD, 2-5 May, 2000.

2) Bearden, David A., "Small Satellite Costs", Crosslink Magazine, The Aerospace Corporation, Winter 2000-2001.

3) Bitten R.E., Bearden D.A., Lao N.Y. and Park, T.H., “The Effect of Schedule Constraints on the Success of Planetary Missions”, Fifth IAA International Conference on Low-Cost Planetary Missions, 24 September 2003.

4) Bearden, D.A., “Perspectives on NASA Robotic Mission Success with a Cost and Schedule-constrained Environment”, Aerospace Risk Symposium, Manhattan Beach, CA, August 2005

5) Bitten R.E., Bearden D.A., Emmons D.L., “A Quantitative Assessment of Complexity, Cost, And Schedule: Achieving A Balanced Approach For Program Success”, 6th IAA International Low Cost Planetary Conference, Japan, 11-13 October 2005.

6) Bitten R.E., “Determining When A Mission Is "Outside The Box": Guidelines For A Cost- Constrained Environment”, 6th IAA International Low Cost Planetary Conference, October 11-13, 2005.

7) Bitten R., Emmons D., Freaner C., “Using Historical NASA Cost and Schedule Growth to Set Future Program and Project Reserve Guidelines”, IEEE Aerospace Conference, Big Sky, Montana, March 3-10, 2007.

8) Emmons D., “A Quantitative Approach to Independent Schedule Estimates of Planetary & Earth-orbiting Missions”, 2008 ISPA-SCEA Joint International Conference, Netherlands, 12-14 May 2008.

9) Freaner C., Bitten R., Bearden D., and Emmons D., “An Assessment of the Inherent Optimism in Early Conceptual Designs and its Effect on Cost and Schedule Growth”, 2008 SSCAG/SCAF/EACE Joint International Conference, Noordwijk, The Netherlands, 15-16 May 2008.

References and Further Reading

37

How Reliable are the Projects’ Estimates at Conceptual Design Stage and How Does Confidence Progress?

39.3%

29.6%

33.7%

0%

10%

20%

30%

40%

50%

60%

70%

Perc

ent C

ost G

row

th (%

)

Avg.

Avg.

Avg.

@ ATP @ PDR @ CDR

Accuracy of Project Estimates Increases Over Time however Cost Growth, Over and Above Reserves, Still Occurs Deep into the Project Life Cycle

Accuracy of Project Estimates Increases Over Time however Cost Growth, Over and Above Reserves, Still Occurs Deep into the Project Life Cycle

38

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5

5.0

5.5

6.0

0.3 m 0.3 m 0.5 m 0.85 m 0.95 m 2.4 m

Gro

wth

per

Dev

elop

men

t Yea

r

Comparison of Schedule Growth Data with Agency Guidelines: NASA Telescope Missions

7 7 13

21 16

36

0

10

20

30

40

50

60

70

80

90

100

110

120

0.3 m 0.3 m 0.5 m 0.85 m 0.95 m 2.4 m

Syst

em D

evel

opm

ent T

ime

(Mon

ths)

Schedule GrowthInitial Schedule

General Rule of Thumb1 Month per Year

NASA/JPL Guidance1.8 Month per Year

Four of Six Telescope Missions Exceeded Common Schedule

Reserve Guidelines

Four of Six Telescope Missions Exceeded Common Schedule

Reserve Guidelines

39

Comparison of Schedule Growth and Success for Planetary Missions vs. Earth-orbiting Missions

0

12

24

36

48

60

72

84

96

0 12 24 36 48 60 72 84 96

Planned development time (mos)

Act

ual d

evel

opm

ent t

ime

(mos

)

Earth Orbiting MissionsPlanetary Missions

• Development times for Planetary missions less than Earth-orbiting missions due to constrained launch windows

• Planetary missions experienced less schedule slip on average than earth-orbiting missions

• However, planetary missions failed or impaired more often

0 10 20 30 40 50

Failures

Successes

Average Development Time

Outcome Planetary Earth-orbiting

% Successful 30% 84% % Partial 40% 7% % Catastrophic 30% 9%

Planetary Earth-Orbiting

Sample Size 10 56

Schedule Growth 3.9% 38.3%

40

Effect of Increased Complexity on Flight System Cost:STEREO Complexity Increased from 40% to 60%

System Cost as Function of Complexity y = 11.523e5.7802x

R2 = 0.8832

10

100

1000

10000

0% 20% 40% 60% 80% 100%

Complexity Index

Cos

t (FY

08$M

)

Successful

Failed

Impaired

To-be-determinedSTEREO @ SDT

STEREO @Launch

41

Partnering Arrangements - Implications for Cost and Schedule growth

Data sources: • 77 NASA missions

(primarily robotic, non-human missions)

• 40 NASA robotic missions

Manifesting

42

0

20

40

60

80

100

120

140

160

0 2 4 6 8 10 12 14

BeforeAfter

0

20

40

60

80

100

120

140

160

0 2 4 6 8 10 12 14

BeforeAfter

Shift project that just started right one year to fit

under wedge

Year

Cos

t

0

20

40

60

80

100

120

140

160

0 2 4 6 8 10 12 14

BeforeAfter

Shift project that just started right one year to fit

under wedge

Year

Cos

t

0

20

40

60

80

100

120

140

160

0 2 4 6 8 10 12 14

BeforeAfter

Hold project at Phase A/B

funding to fit under wedge

Year

Cos

t

Budget reduced for several projects, and added back later (with an inefficiency factor)

Year

Cos

t

With Cost Overruns Included Adjusted to Fit Budget

ApplyPenalties

Penalty 1:Delay Start

Penalty 2:Phase A/B Hold

Penalty 3:Funding Reduction

0

0

0

0

0

0

0

0

0

2007 2009 2011 2013 2015 2017 2019 2021 2023 2025 2027 2029 2031 2033 2035

Year

Cos

t

0

0

0

0

0

0

0

0

0

2007 2009 2011 2013 2015 2017 2019 2021 2023 2025 2027 2029 2031 2033 2035

Year

Cos

t

Cos

t

02007 2009 2011 2013 2015 2017 2019 2021 2023 2025 2027 2029 2031 2033

Year

Cos

t

02007 2009 2011 2013 2015 2017 2019 2021 2023 2025 2027 2029 2031 2033

Year

Sandchart Applies Real World Penalties to Projects Based on Performance of Other Elements in Portfolio

Allows realistic assessment of interaction of multiple program elements or multiple missions within a given portfolio

43

• Develop Request for Information– Technical implementation and required technology development– Concept maturity and basis of cost and schedule estimates

• Evaluate concepts in a Leveled process– Technical, cost and schedule assessment performed in parallel– Discussion and interaction between disciplines important– Concepts have different levels of maturity which impacts technical and

cost risk– Quantified cost threats identified based on:

• Schedule maturity and optimism• Technical maturity based on mass and power contingencies

• Present Initial findings to each panel– Discuss concepts and receive immediate and written feedback

• Re-evaluate, where appropriate, and provide final results – Level all concepts– Summary charts for committee and detailed charts for each panel

Key Phases of Astro2010 CATE Process

44

Project Funding Profiles and Schedule Milestone Constraints• Sand Chart requires that cost point estimates are spread over time

into budget profiles– For POR elements, development cost phasing, milestones, and project

duration defined by the PMR ‘09 funding profile– For non-POR elements, development cost phasing, milestones, and

project duration defined by assuming 40/50 beta curves (40% cost at midpoint) spread over several years (depending on element)

– Operations cost profiles calculated by assuming both marginal/unit costs (tied to a flight manifest) and fixed/year costs (when appropriate).

– Costs are modeled in FY09$, and converted to RY$ using the NASA New Start inflation index. Future costs - beyond 2009 - were inflated using 2.4% inflation factor consistent with the budget inflation factor.

• Schedule linkages are used to link projects when necessary– Example: Used to ensure that launch vehicle (e.g. Ares I) and crew

capsule (e.g. Orion) finish at the same date– Example: Used to push out costs for starting operations (fixed/yr,

marginal/unit) when development experiences schedule delays

$

Surface Systems

Lander

CapsuleBooster

TimeMoon Flexible Path

In space elements

DDT&E

$ CapsuleBooster

LanderSurface Systems

In space elements

Moon Flexible Path Time

DDT&E

Development Cost Phasing: Lunar vs. Flexible

Used with Permission of Review of US Human Space Flight Plans Committee