war-winning capabilities…on time, on cost acquisition cost estimating update 28 feb 08 ken...
TRANSCRIPT
War-winning Capabilities…On Time, On Cost
Acquisition Cost Acquisition Cost
Estimating UpdateEstimating Update
28 Feb 0828 Feb 08
Ken Kennedy, AAC/FMC Kristy Golden, AFCAA/FMAE
Air Armament CenterAir Armament Center
DISTRIBUTION STATEMENT A: Approved for public release; distribution is unlimited.1
2
War-winning Capabilities…On Time, On Cost
Outline
• Estimating Trends• Leadership Concern• Congressional Reaction• Root Cause Analysis• 2007/2008 Initiatives • AAC Cost Demand• AFCAA Standup• Industry Help• Summary
3
War-winning Capabilities…On Time, On Cost
3
Estimating Trends
• GAO-07-406SP Defense Acquisitions Assessments of Selected Weapon Programs– GAO assessed 62 weapon systems with a total
investment of over $950 billion, some two-thirds of the $1.5 trillion DOD plans for weapons acquisition
– “Fully mature technologies were present in 16 % of the systems at development start” - the point at which best practices indicate mature levels should be present.
– Programs that began development with immature technologies (84%) experienced a 32.3 percent cost increase, whereas
– Those that began with mature technologies (16%) increased just 2.6 percent.”
4
War-winning Capabilities…On Time, On Cost
4
Leadership Concern
• Sue Payton: Jane's Defence [sic] Weekly, 07/17/2007– Reduce pressure on budget by improving methods for
calculating the overall cost of acquisition programme [sic]
– Stop taking contractors' one-off cost estimates at face value
– Develop own methods for calculating the true, long-term cost of a weapon system
– Tendency to highlight the basic price tag of a weapon system and play down the much bigger 'lifecycle cost
– Wait until a preliminary design review has been completed, rather than trying to guess a weapon's price before the final requirements are defined
– Push contractors to build more prototypes … important insight into future development costs
5
War-winning Capabilities…On Time, On Cost
5
Congressional Reaction
Nunn-McCurdy Breach Thresholds• 2006 Defense Appropriations Act (Public Law No:
109-148)
• Requires Increased emphasis on – Accurate TRL assessments– Quantification of risk and uncertainty – Optimal decisions within trade space constraints– Better cost and schedule forecasts– Budgeting to Independent Cost Estimates
Type Breach Notification CertificationInitial APB 30% 50%Current APB 15% 25%
Growth Over Baseline
6
War-winning Capabilities…On Time, On Cost
Root Cause
LowMed-LowMedMed-HighHigh
Cost risk levels
Extremely High
Range of Missile Program Cost Outcomes
TRL Level
AOA Risk Reduction SDD LRIP FRP
1 2 3 4 5 6 7 8 9
Risk is retired as a program progresses and technology matures
LowMed-LowMedMed-HighHighExtremely
High
We are too optimistic about risk/uncertainty
Ris
k/U
ncer
tain
ty
High
High
Med
Med
Low
7
War-winning Capabilities…On Time, On Cost
2007/2008 Initiatives
• Better Risk ID and Modeling
• Continued Sufficiency Reviews
• New Source Selection Focus
• More Contract Risk Sharing
8
War-winning Capabilities…On Time, On Cost
• Integrate Probability of Consequence Screening (P/CS) “risk cube” and Probability of Program Success (PoPS) assessment into cost estimating process – Existing infrastructure tools– Realistic and consistent approach– Results in a “time now” WBS risk rating
• Use Monte Carlo simulation with WBS correlation to define total program cost – Present decision maker with better picture of
uncertainty based on knowledge today
Better Risk ID and Modeling
9
War-winning Capabilities…On Time, On Cost
• Cost “gray beard” review – Reinforce estimating best practices– Verify data sources and methodology– Suggest alternate benchmarks– Request appropriate SME inputs
• Independent technical assessment• Independent schedule assessment
Continued Sufficiency Reviews
10
War-winning Capabilities…On Time, On Cost
• Cost risk # 1 evaluation priority– High, medium, or low rating based on
percent difference between proposed contractor cost and government estimate of offeror’s approach
• Requires clear communication– Full data source ID and extract for element
basis of estimates (BOEs)– Full data source adjustment justification in
element BOEs– Full discussion of risk and uncertainty in
element BOEs
New Source Selection Focus
11
War-winning Capabilities…On Time, On Cost
More Contract Risk Sharing
• Solicitation and proposal– Request WBS element uncertainty/risk and $
magnitude as part of each narrative basis of estimate worksheet
– Request sum of element risk $ to be included as a separate proposal line
• Evaluation– Description of element uncertainty used to
model risk for independent estimate• Execution
– Proposal risk line becomes earned value MR– SOO requirement for discussion before use
Air Force Cost Analysis Agency (AFCAA) Standup
Organization Headquarters
• Deputy Assistant Secretary (Cost & Economics), AFCAA– Policy and functional oversight
– Major program milestone decisions -- Independent Cost Estimates (ICE) for delegated programs (required by law); Component Cost Analysis (CCA); AF CAIG service cost position (SCP)
– Headquarters/functional support (analysis, review, etc.)
THE ASSISTANT SECRETARYOF THE AIR FORCE
(Financial Management & Comptroller)
THE ASSISTANT SECRETARYOF THE AIR FORCE
(Financial Management & Comptroller)
DEPUTY ASSISTANTSECRETARY
(Plans, Systems & Analysis)
DEPUTY ASSISTANTSECRETARY
(Plans, Systems & Analysis)
DEPUTY ASSISTANTSECRETARY
(Cost & Economics)
DEPUTY ASSISTANTSECRETARY
(Cost & Economics)
DEPUTY ASSISTANTSECRETARY
(Budget)
DEPUTY ASSISTANTSECRETARY
(Budget)
Deputy Assistant Secretary(Cost & Economics) SAF/FMC
*The Deputy Assistant Secretary holds two titles as SAF/FMC and AFCAA/FM
Deputy Assistant Secretary for Cost & Economics (SAF/FMC)Executive Director (AFCAA/FM)
Mr Hartley, SES*Associate Deputy Assistant Secretary for Cost & Economics
Ms Woods, SES
Deputy Assistant Secretary for Cost & Economics (SAF/FMC)Executive Director (AFCAA/FM)
Mr Hartley, SES*Associate Deputy Assistant Secretary for Cost & Economics
Ms Woods, SES
AFCAA Technical Director
Mr. Jordan
AFCAA Technical Director
Mr. Jordan
Space ProgramsDivision
(AFCAA/FMS)Mr. Seeman
Space ProgramsDivision
(AFCAA/FMS)Mr. Seeman
Info TechDivision
(AFCAA/FMI)Mr. Moul
Info TechDivision
(AFCAA/FMI)Mr. Moul
ResourcesDivision
(AFCAA/FMR)Ms. Cann
ResourcesDivision
(AFCAA/FMR)Ms. Cann
Force Analysis Division
(AFCAA/FMF)Lt Col Hurley
Force Analysis Division
(AFCAA/FMF)Lt Col Hurley
A/C & Weapons Division
(AFCAA/FMA)Mr. McVicker
A/C & Weapons Division
(AFCAA/FMA)Mr. McVicker
IntegrationDivision
(AFCAA/FMC)Col DuPre
IntegrationDivision
(AFCAA/FMC)Col DuPre
Economics & Business Mgmt
(SAF/FMCE)Mr. Connair
Economics & Business Mgmt
(SAF/FMCE)Mr. Connair
Principal Economic & Financial Advisor
Mr. Hosey
Principal Economic & Financial Advisor
Mr. Hosey
ASC/AACOL
ASC/AACOL
AFSCP/SMCOL
AFSCP/SMCOL
ESCOL
ESCOL
AFCAA Eglin OL
Kristy GoldenKristy GoldenChief, AFCAA Eglin OL
YC-501-03
Paul PinkowskiPaul PinkowskiCost AnalystYA-501-02
VacantVacantCost AnalystYD-1515-02
John CargillJohn CargillTechnical Director
YD-1515-03
Justin NeeceJustin NeeceCost AnalystYD-1515-01
Beth FloresBeth FloresCost Analyst (PAQ)
YD-1515-01
Operating Locations
Los Angeles AFB CA
Peterson AFB CO
Eglin AFB FL
AFCAA - DC
Wright-Patterson AFB OH
Hanscom AFB MA
Air Force Concern
• Current AF-wide policy does not consistently require adequate, objective cost analysis or review– Limited to infrequent milestone reviews
– Inconsistent leadership value placed on cost analysis (contrast DAB to AFIPT)
• Cost resources down 60% since 1992 -- approximately 400 positions!– Staffs hit hardest, no career path/pipeline, compare poorly with
Army, Navy, others
• Little “functional” governance – Inexperienced analysts “isolated” in programs with little functional
support -- fractured estimating methods, databases, models -- inconsistent data/estimate quality, analyst qualifications
There is a Cost Growth Problem.
Cost Estimating is a Significant, but not Lone, Contributing Factor.
However, it is a factor we can manage!
Acquisition Cost Capability Initiative
• Better cost estimating processes– Policy and standards
– Re-writing AFPD 65-5 and accompanying AFIs and AFMANs– Building standards and templates– Performance metrics
– Collaboration– Encourage team efforts– Establish operating locations at Product Centers and MAJCOMs
– Transparency – Share data / research / methods
• Repopulate personnel pipeline • Upgrade education, training, and certification programs
– Provide Cost Estimators with the tools to succeed
Improve Air Force Cost Estimating
Current Status
• Policy AFD 65-5– The acquiring organization shall build program cost estimates*
• Annually
• Milestone decisions
• Nunn-McCurdy Cost Breaches
• Source Selections
• POM initiatives, disconnects, and offsets – Headquarters shall provide non-advocate cost / risk assessments to decision makers– Collaboratively develop and execute training, data collection/methods and model
development programs – Headquarters shall review cost, economic, or business case analyses to be presented
to SECAF, USECAF, CSAF, VCSAF• Personnel
– Hired 20+ positions• Operating Locations
– Completed beddown agreements at Peterson AFB, Wright-Patterson AFB, Eglin AFB, Hanscom AFB, and LA AFB
– Working Host Tenet Support Agreement at each location• Training and Certification
– Partnered with SCEA and DAU to improve certification program– Developed Training OI for new analysts
* Led by government employees
* SAF/FMC defined standards and procedures
AFCAA OL Mission
Shorten “Kill Chain” (Cost Budget)
• AFCAA-OL will function as the liaison with the Center
• Work collaboratively with the program office estimators
• Non Advocate Cost Assessments (NACAs)
• Independent Cost Estimates (ICEs)
How the AFCAA-OLs will Help
• Collaborative program cost estimates– Build cost models– Cost estimating relationships– Risk analysis– Data / research / methods
• Potential roles– Team lead– Team member– Sufficiency reviewer– Consultant– Researcher– Facilitator between Program Office and DCARC (OSD CAIG)
22
War-winning Capabilities…On Time, On Cost
Industry Help
• Don’t over promise capability with unrealistic resources
• Define and quantify uncertainty for your proposed concept
• Provide true risk now versus what it could be in a future year
• Identify the right level of MR to neutralize problems before they overwhelm the program
23
War-winning Capabilities…On Time, On Cost
Summary
• Cost estimating community needs to provide more realistic resource projections for Congress and DoD
• Additional AFCAA resources are in place now with more FMC positions coming to help programs
• New initiatives are in work to improve estimate quality, quantify uncertainty, and fund appropriate risk efforts