a case study of earned schedule to do predictions
DESCRIPTION
A Case Study of Earned ScheduleTRANSCRIPT
-
1A Case Study of Earned Schedule to do Predictions
By Lewis HechtConsultant in EVMS
Vestal, NY 13850607-765-2731
-
2Agenda of Presentation Introduction to earned schedule Summary of technical references Our case study Defense Contract Management Agency concerns Mathematical modeling details Review of contractor performance Prediction accuracy Summary and conclusions Addendum:
Bio of the author
-
3Introduction to earned schedule-1
Earned schedule methodology was developed by Walt Lipke in 2003. Kym Henderson from Australia has published additional work in 2004 and 2005.
Traditional independent estimate at completion formulas do not predict the final duration of a contract.
-
4Introduction to earned schedule-2
In traditional EVMS analysis, cumulative schedule variance is zero at the end of the period of performance by definition.
Lipke proposed new measurements to track a contractors schedule performance. These are SV(t) schedule variance with respect to time, and SPI(t) schedule performance index with respect to time.
-
5Introduction to earned schedule-3
Henderson in 2003 suggested using the concepts PD (planned duration) and PDWR (planned duration of work remaining) as predictors for an independent estimate of completion date for the contract.
Jacob and Kane have published additional earned schedule concepts in 2004.
The IPM conference in the last 2 years has had additional papers supporting ES concepts.
-
6Summary of technical references-1
Earned Schedule: A Breakthrough Extension to Earned Value Management,Proceedings of PMI Global Congress Asia Pacific 2007, January 2007 by Kym Henderson.
Why EVM Is Not Good for Schedule Performance Analyses (and how it could be), The Measurable News, Winter 2006-2007, by Radenko Corovic.
Applying Earned Schedule to the Critical Path and More, The Measurable News, Fall 2006, by Walt Lipke.
A Simulation and Evaluation of Earned Value Metrics to Forecast Project Duration, Journal of Operational Research Society, September 2006, by Mario Vanhoucke & Stephan Vandevoorde.
Can You Tell the Time?, Journal of the Association of Project Management, August/September 2006, by Mick Higgins.
-
7Summary of technical references-2
A Comparison of Different Project Forecasting Methods Using Earned Value Metrics, International Journal of Project Management, May 2006, by Stephan Vandevoorde & Dr. Mario Vanhoucke
Earned Schedule in Action, The Measurable News, Spring 2005, by Kym Henderson.
"Further Developments in Earned Schedule,The Measurable News, Winter 2004, by Walt Lipke.
"Further Developments in Earned Schedule, The Measurable News, Spring 2004, by Kym Henderson.
"Earned Schedule: A Breakthrough Extension to Earned Value Theory?, A Retrospective Analysis of Real Project Data, The Measurable News, Summer 2003, by Kym Henderson.
"Schedule is Different,The Measurable News, March & Summer 2003, by Walt Lipke.
-
8Our case study-1 Purpose of this contract was to build a second avionics
maintenance trainer for Navy helicopters Navy decided to implement schedule incentive to expedite
deployment constraints with moving previous maintenance trainer from East Coast to West Coast
Problems encountered by the contractor in first trainer build effort were ignored deliberately
Navy customer wanted DCMA to generate predictions as to when the contractor could make the enhanced schedule delivery date
This contract was picked to test out the successful prediction capability of earned schedule methodology
-
9Our case study-2
Contract start date was 12/29/2003 Contract end date was 11/16/2005 Major milestone was shipment by 7/1/2005 Schedule incentive was substantial for
shipping earlier than 7/1/2005
-
10
Defense Contract Management Agency Concerns
No lessons learned by contractor on this contractprevious experience in similar contracts was ignored by contractor program management
CSSR data not accuratetoo many serious mistakes made for the first three months in cost reports
Traditional EV analysis not able to predict long term schedule issues to customers satisfaction
Contractor decision on work package structure not acceptable
Contractor used work packages whose duration was for entire period of performance for this contract
Too many changes of key contractor personnel
-
11
Mathematical Modeling Details
Earned schedule calculator was used from ES website to predict completion date
Running comparison was kept between contractors LRE and estimated delivery date vs. DCMAs independent estimates
All formulas were from ES website
-
12
Review of contractor performance
Plot #1 shows % CV and %SV as parameters of time
Plot #2 shows a typical box plot of % CV and %SV
Plot #3 shows contractors own estimate of schedule slippage vs. DCMAs estimate
-
13
Traditional EVMS Parameters
-50.00%
-40.00%
-30.00%
-20.00%
-10.00%
0.00%
10.00%
20.00%
Jan 04CSSR
Feb 04CSSR
Mar 04CSSR
Apr 04CSSR
May 04CSSR
Jun 04CSSR
Jul 04CSSR
Aug 04CSSR
Sep 04CSSR
Oct 04CSSR
Nov 04CSSR
Dec 04CSSR
Jan 05CSSR
Feb 05CSSR
Mar 05CSSR
Apr 05CSSR
May 05CSSR
Jun 05CSSR
Time (CSSR report date)
P
e
r
c
e
n
t
a
g
e
c
u
m
%CV
%SV
-
14
Box Plot of %CV vs %SV
-8.00%
-6.00%
-4.00%
-2.00%
0.00%
2.00%
4.00%
6.00%
-10.00% -5.00% 0.00% 5.00% 10.00% 15.00%
Under cost, ahead of schedule
Under cost, behind schedule
Over cost, behind schedule
Over cost, ahead of schedule
-
15
Comparison of estimated days behind schedule
-120
-100
-80
-60
-40
-20
0
Months Jan 04 thur June 05
D
a
y
s
b
e
h
i
n
d
s
c
h
e
d
u
l
e
Contractor's estimate behind schedule(days)
DCMA estimated behind schedule(days)
Linear (DCMA estimated behindschedule (days))
Confidence factor for linear regression line on DCMA data is R squared = 0.598
-
16
Prediction Accuracy-1
Actual history of contract schedule performance was that 1) contractor worked multiple shifts and with
extra help to keep delivery date within 10 day slip in last 2 months of final contract work
2) shipment was actually 7 days late 3) contractor did not win any schedule
incentives
-
17
Prediction Accuracy-2
Traditional EV analysis would state that contractor was under staffed and running behind schedule
Due to changes in key contractor personnel, a learning curve had to be factored into schedule performance considerations
-
18
Prediction Accuracy-3
DCMA estimates of completion date were much more realistic than contractor.
DCMA, acting for benefit of Navy customer, forced contractor to implement recovery plan and to speed up work effort to make delivery
-
19
Summary and conclusions
Earned schedule proved to be very useful in predicting how much work remained to be completed.
US Navy customer was very satisfied with ES predictions
-
20
Addendum: Bio of the author
BS in physics from Ohio State; MS in physics from Rensselaer Poly Tech
20+ years in electronic packaging technology, mainly at IBM Microelectronics
6 years at Defense Contract Management Agency as GS 12 electronic engineerretired in February 2007
Certified level 2 in Defense Acquisition Workforce Improvement Act education requirements
EVMS expert on Navy helo programs for DCMA at that office
CD-ROM Opening ScreenIntroductionTable of ContentsSearchHelpPractice SymposiaPS 02 Introduction to the NDIA EVM Standard and GuidesPS 03 Recognizing EVM GamingPS 04 A Success- Oriented and Collaborative Approach to EVM System ValidationPS 05 Interfacing Earned Value Management and Risk Management ProcessPS 06 Efforts to Improve Air Force Cost AnalysisPS 07 A Case Study of Earned Schedule to do PredictionsPS 08 NDIA EARNED VALUE MANAGEMENT SYSTEMS APPLICATION GUIDEPS 09 Statistical Methods Applied to Project ManagementPS 10 Department of Defense Earned Value Management Policy and InitiativesPS 11 Lessons Learned in Implementing and Certifying and EVMS in Europe On the Presidential Helicopters ProgramPS 12 What is New with EVM at NASA?PS 13 Relationship Diagramming MethodPS 14 NDIA EVMS Intent GuidePS 15 Adoption the Real Challenge in Implementing EVMSPS 16 Lessons Learned in Implementing EVM within the Federal EnterprisePS 17 Exploration of Cultural ChangePS18 Electronic Facilitation of NASA Integrated Cost Risk for Improved Cost ManagementPS 19 Implementing EVMS in a Dynamic Project EnvironmentPS 21 FAA Portfolio/ Program Performance MetricsPS 22 The Risk Based Estimate at Completion (EAC)PS 23 Earned Schedule in ActionPS 24 GAOs Cost Assessment Guide & How It Defines EVM Best PracticesPS 25 Applying Earned Value Management to Project Formulation?PS 26 NDIA EVMS System Acceptance GuidePS 27 Achieving ANSI/ ESI Standard 748 Compliance at VA using PrimaveraPS 28 Integrating EVM into Project AssessmentsPS 29 Building a Professional Project Planning and Control CommunityPS 30 Linked Control Account Analysis and the Use of Risk IndexesPS 31 Estimating EVMS Implementation & Execution RequirementsPS 32 EVMS SurveillancePS 33 The Project Cost Variance Model: a Project Management ToolPS 34 Program Office Execution of an Integrated Baseline ReviewPS 35 Earned Value in an Agile EnvironmentPS 36 Using EVM Reporting to Develop Cost Estimating Models: The Value and Pit Falls
Professional Education Program (PEP) (Training Seminars)CPM- 100 Principles of Project Management Special Topics100- A Integration, Scope, Time, and Cost Management100- B Overview of PMI EVM Practice Standard100- C Capital Asset Plan and Business Case (OMB Exhibit 300)100- D Earned Value Management System Surveillance100- E Human Resources and Communications Management100- F Developing and Implementing an Integrated PMIS
CPM- 200 Principles of Schedule Management200- A Critical Path Method Scheduling200- B Introduction to Resource Loading / Analysis Techniques200- C Schedule Status Updates and Analysis Techniques200- D EV Cost and Schedule Performance Management200- E Schedule Analysis Techniques200- F Basic Approach to Cost and Schedule Risk Analysis
CPM- 300 Principles of Earned Value Management300- A EVM Principles and Terminology (Part 1)300- B EVM Principles and Terminology (Part 2)300- C Control System Design (Part 1)300- D Control System Design (Part 2)300- E Establishing an Earned Value Management System (Part 1)300- F Establishing an Earned Value Management System (Part 2)
CPM- 400 Principles of Earned Value Metrics & Analysis400- A Developing and Maintaining the PMB (Part 1)400- B Developing and Maintaining the PMB (Part 2)400- C Understanding Resource Management through EVM400- D Integrated Baseline Review400- E Earned Value Data Analysis Techniques (Part 1)400- F Earned Value Data Analysis Techniques (Part 2)
CPM- 500 Principles of Technical Management500- A Development of the WBS500- B Technical Risk Management500- C Integrating Systems Engineering with EVM (Part 1)500- D Integrating Systems Engineering with EVM (Part 2)500- E Material and Subcontract Issues in EVMS500- F Implementing Technical Performance Measurement
CPM- 600 Principles of Project Integration600- A Integrating the Projects Technical Components600- B Integrating Scheduling and EVM Metrics600- C IMP and Its Impact on IMS600- D Integrating Risk into PMB Development600- E Project Integration: Buyers Perspective (Case Study)600- F Project Integration: Sellers Perspective (Case Study)
Tools TrackTT 04 Effective Assessment Tools EVM and PROJECTRxTT 05 AT& L Knowledge Sharing System (AKSS), Acquisition Community Connection (ACC) and EVM Community of Practice (EV CoP)
WorkshopsWS 01 Validation Reviews: Whats the big deal?WS 02 Earned Value Management for ServicesWS 05 Well now that we have an NDIA Validation Guide, how is it going?WS 06 DCMA Trip Wire Metrics: Preparing and Analyzing