humphreys.gary
TRANSCRIPT
EVMSThe Principles, Best Practices and Benefits
Gary C. HumphreysHumphreys & Associates, Inc.
NASA PM Challenge 2010
Used with Permission
PART 1
Overview, Earned Value Management
1NASA PM Challenge 2010
What is EVM?
• Earned Value Management (EVM) is an integrated project management process for assessing, understanding and quantifying what a contractor or in-house project is achieving with program/project dollars
– Integrates technical, cost, and schedule with risk management
– Allows objective assessment and quantification of current project performance
– Helps predict future performance based on trends
EVM is simply good project management!
“Management 101” “EVM 101”
Plan Organize
Organize Plan & Budget
Staff Account for Costs
Direct Analyze
Control Incorporate Changes
2NASA PM Challenge 2010
EVM enhances the chances of program/project success
• To Succeed, you must Manage
• To Manage, you must Measure
• To Measure, you must Define
• To Define, you must Understand
So why use EVM?
3NASA PM Challenge 2010
NASA EVM Policy & Requirements
• NASA EVM requirements are found in NPR 7120.5, NASA Program and Project Management Processes and Requirements and NPR 7120.7, NASA Information Technology and Institutional Infrastructure Program and Project Management Requirements. Policy is contained in NASA FAR Supplement (NFS) 1834.201. NASA EVM requirements are predicated on the principles and guidelines in ANSI/EIA-748, Industry Guidelines for Earned Value Management Systems.
• NPR 7120.5 requires the project EVM approach to be in place by Key Decision Point (KDP) C (the conclusion of Preliminary Design Review (PDR)) and implemented in Phase C (Final Design and Fabrication) through KDP E [Flight Readiness Review (FRR) or Launch Readiness Review (LRR)].
• Additionally, project performance reporting must begin within 60 days after the start of Phase C. To meet this timeline, it is highly recommended that projects begin implementation planning during Phase B (Preliminary Design and Technology Completion).
4NASA PM Challenge 2010
NPR 7120.5 Guidance
• NPR 7120.5 also requires that NASA programs and projects use EVM on projects and contracts that meet the following criteria:
1. For projects with a total anticipated final value of $20 million or more, EVM principles should be applied, as defined by ANSI/EIA-748, Earned Value Management Systems.
a. If the project's primary NASA Center has a validated EVMS, the project uses that system rather than the EVM principles application approach.
2. For development contracts and subcontracts valued at or between $20M and $50M, the contractor is expected to have an EVMS that complies with the 32 guidelines in the ANSI/EIA-748 Standard.
3. For development contracts and subcontracts valued at $50M or more, the contractor is required to have an EVMS that has been formally validated by the Government.
4. EVM is not required on contracts for non-developmental engineering support services, steady state operations, basic and applied research, and routine services such as janitorial services or grounds maintenance services. In these cases, application of EVM is at the discretion of the Program/Project Manager (PM).
• NASA projects that do not have a validated EVMS will use the seven management principles that are contained in ANSI/EIA-748 standard.
5NASA PM Challenge 2010
1. Plan all work scope for the project to completion.
2. Break down the project work scope into finite pieces that can be assigned to a responsible person or organization for control of technical schedule and cost objectives.
3. Integrate project work scope, schedule, and cost objectives into a performance measurement baseline plan against which accomplishments may be measured. Control changes to the baseline.
4. Use actual costs incurred and recorded in accomplishing the work performed.
5. Objectively assess accomplishments at the work performance level.
6. Analyze significant variances from the plan, forecast impacts, and prepare an estimate at completion based on performance to date and all remaining work that must be performed.
7. Incorporate Earned Value Management in the project decision-making and review processes.
Seven EVM Principles (ANSI/EIA-748-B-2007)
6NASA PM Challenge 2010
• Organization (5 guidelines)– Define contractual effort and assign responsibilities for the work. Integrate planning,
scheduling, work authorization, and accounting systems.
• Planning, Scheduling and Budgeting (10 guidelines) – Plan, schedule, budget and authorize the work. Establish and maintain a time-phased
budget baseline.
• Accounting Considerations (6 guidelines) – Accumulate costs of work and material.– Report on progress/accomplishments to date.
• Analysis & Management Reports (6 guidelines)– Compare planned, earned and actual costs, analyze variances. Develop revised
estimates at complete. Identify managerial actions.
• Revisions and Data Maintenance (5 guidelines)– Execute timely incorporation of contractual changes. Maintain traceability from original
budgets.
Guideline Categories
32 EVM Guidelines (ANSI/EIA-748-B-2007)
7NASA PM Challenge 2010
Earned Value Management Process
Scheduling & Resource Allocation Systems
Informed ManagementDecisions
Risk Monitoring
AccountingSystem
Technical Performance Measurement
Earned Value Mgmt System(Progress)
Progress
8NASA PM Challenge 2010
Features of a “Good” EVM system
• Thorough up-front planning based on a detailed product-oriented Work
Breakdown Structure (WBS), incorporating Risk Management
– Timely Performance Measurement Baseline (PMB) establishment and control
– Information broken down by product, as well as by organization or function
• Objective measurement of technical accomplishment against the plan at
levels where the work is being performed
• Reporting consistency and discipline – a structured process and methodology− Summarized data reporting to higher management (performance metrics) for use
in effective decision making
− Provides for analysis of significant variances and their impact on the project
− Facilitates proper management actions to mitigate risk and manage technical, cost and schedule performance
9NASA PM Challenge 2010
PART 2
EVMS Terms and the Performance Measurement Baseline
10NASA PM Challenge 2010
Basic EVM Terms
• Budgeted Cost for Work Scheduled (BCWS)– The sum of the time-phased budgets established for all effort (including in-
process work) scheduled to be accomplished within a given time period
• Budgeted Cost for Work Performed (BCWP)– The sum of the time-phased budgets for work actually completed during a
specified time period
– BCWP is the budgeted value for the work actually accomplished
– Earned value can only be determined after the performance (or scheduled performance) of work
• Actual Cost of Work Performed (ACWP)– The costs (taken from the Accounting System) actually incurred (or in some
cases estimated) and recorded, in accomplishing the work performed within a given time period
11NASA PM Challenge 2010
• Budget At Completion (BAC)– The budgetary goal (excluding MR) for accomplishment of all authorized work
• Estimate At Completion (EAC)– Actual direct and indirect costs allocable to the contract, plus the estimate of all
costs for authorized work remaining
– Estimated cost to complete the entire project scope of work
– Supported with a rationale
– Best case, worst case, most likely
Basic EVM Terms (continued)
12NASA PM Challenge 2010
• Total Allocated Budget (TAB)– Total Allocated Budget is the sum of all budgets allocated by the contractor
against which contract performance is to be measured and reported
• Contract Budget Base (CBB)– Contract Budget Base is the negotiated contract cost plus the estimated cost of
authorized unpriced work
• Authorized Unpriced Work (AUW)– Contract work authorized by the contracting officer, not yet negotiated, aka not
yet definitized work
Basic EVM Terms (continued)
13NASA PM Challenge 2010
Contract Price (Project Price)
Total Allocated Budget (TAB) Contract Fee/Profit
Contract Budget Base (CBB) or Project
Budget Base (PBB)
Authorized Unpriced Work (AUW)
Performance Measurement Baseline (PMB) Management Reserve (MR)
Over Target Baseline (OTB)
Distributed Budget Undistributed Budget (UB)
Control Account (CA) Summary Level Planning Package (SLPP)
Work Package (WP) Planning Package (PP)
Contract/In-house Project –PMB Relationship
MR – Held for known unknowns but in-scope future effort. Not used to cover cost overruns.
UB – Temporary holding place for allocation to control accounts not to exceed two accounting periods following negotiations.
Total of all authorized work, both negotiated and un-negotiated
The total is also called Budget at Completion (BAC)
14NASA PM Challenge 2010
The Performance Measurement Baseline (PMB) –Is all the work planned?
• A time-phased budget plan against which contract performance is measured
• Developed by the contractor and jointly reviewed with the customer within six months of contract award
• Based on the anticipated flow of work and outcomes• Incorporates dynamic, authorized project changes
What is the PMB?
15NASA PM Challenge 2010
Establishing the PMB is a 3-step process:
PMB Development
1. DEFINE THE WORK
2. SCHEDULE THE WORK
3. ALLOCATE BUDGETS100
40
60
15
25
3030
$
CONTRACT BUDGET BASE
MR
TIME
PMB – Time-phased, budgeted plan where contract performance can be measured.
• Statement of Work (SOW)• Work Breakdown Structure (WBS)• Organizational Breakdown
Structure (OBS)• Responsibility Assignment Matrix
(RAM)
• Work Authorization System • Control Account Plan (CAP)• Work Packages (WP) &
Planning Packages (PP)
Integrated Master Schedule (IMS)• Master Schedule• Intermediate Schedule• Detailed Schedules
16NASA PM Challenge 2010
Work Breakdown Structure (WBS)
• A product-oriented hierarchical division of the hardware, software, services, and data required to produce the project’s end product(s)
• Should not be functional or organizational oriented• Should reflect the total project scope of work• Structured according to the way the work will be performed• Reflects the way in which project costs, schedule, technical and risk data
are to be accumulated, summarized, and reported• WBS is the most important data item in defining and controlling the
contracted technical scope• Summary level WBS must be the sum of all lower level WBS hierarchical
in nature• Supports historical cost collection necessary for future costing activities of
similar scoped products
“It’s the WHAT that is Going to Be Done”
17NASA PM Challenge 2010
Properly Structured WBS
Product Oriented
Standard Level 2’s
Clear WBS Titles
Center Breakout at Appropriate Level
18NASA PM Challenge 2010
Responsibility Assignment Matrix (RAM)
The Responsibility Assignment Matrix (RAM) Integrates WBS with OBS. The intersecting points are the Control Accounts.
CA
1.0
1.1 1.2 1.3
1.1.1 1.1.2 1.1.3 1.1.4
1.1.2.1 1.1.2.2 1.1.2.3 1.1.2.4 1.1.2.5
Elec
Mec
hSW
ENG
Contract Work Breakdown Structure (WBS)
Org
aniz
atio
n B
reak
dow
n St
ruct
ure
(OB
S)
Work Packages
RAM
Control Account• Task Description• Perform. Measure• Work Authorization• Cost Accumulation• Work Package, Schedules,
& Budget
WBS Reporting Level
CWBS Extension
Func
tiona
l Org
19NASA PM Challenge 2010
Control Account (CA)
• The CA forms the hub of the technical, cost and, schedule management focus
• Virtually all elements of planning and control come together at the CA level including work authorization, schedules, budgets, earned value, cost collection (actuals), problem identification/variance analysis and estimate to complete
• Consists of:– Work Packages (WPs) – near term work that is well defined.– Planning Packages (PPs) - more far term effort used for budgeting and
scheduling purposes. A vehicle for planning at a higher level when details may not yet be well defined. Planning packages must also reflect horizontal integration with assignment of predecessors and successors.
• SCOPE• SCHEDULE• BUDGET
Control Account BUDGET – 150
1010
1520
20WORK
PACKAGES
PLANNING PACKAGES
4530
20NASA PM Challenge 2010
• Using the WBS:– Develop a logic network schedule for all contractual activities, including
technical performance parameters/outcomes
– Scheduling is a process of changing work scope into defined tasks:• WBS oriented• Defined time duration (phased over the total life of the program)• Logically connected to all other related tasks
– Determine resources required to complete scheduled activities• Labor
– Direct Labor– Civil Service Labor (when applicable)– Support Contractors
• Material• Contracts/Subcontracts• Other Direct Costs• Indirect Costs
Schedule Development
21NASA PM Challenge 2010
Horizontal & Vertical Schedule Integration (VSI)
The Scheduling System: Consists of an integrated, multi-tiered hierarchy of schedule levels. With a complete Logic Network Schedule, the time-phased PMB will be accurate, allowing for the identification of the true critical path(s).
Master Schedule(Contract Summary)
Intermediate Schedule(Logic Network
Schedule)
Detailed Schedules(Control Accounts)
VER
TIC
ALIN
TEG
RAT
ION
HORIZONTAL
INTEGRATION
INTEGRATED MASTER SCHEDULE (IMS)
22NASA PM Challenge 2010
Schedule Considerations
• Is the schedule logic (activity sequence) established and valid?• Are the schedule milestones compatible with the schedule logic?• Is the critical path analysis valid?• Are durations reasonable compared with other project’s history?• Are resources used to their maximum?• Are parallel activities reasonable or risky, i.e., flight hardware being
built before qualification is complete?
23NASA PM Challenge 2010
• Is the plan fully defined?
• Is the plan realistic?
• What are the risky areas of the plan?
Understanding the Baseline
How can we understand and assess inherent risk in the plan?
24NASA PM Challenge 2010
• How is Earned Value determined?
– When planning work three basic classifications are used:• Discrete Work – Definable end product• Level of Effort (LOE) – End product not defined• Apportioned Effort – Budgeted based on a percentage of another (base)
related task (e.g., Quality Assurance)
– Earned Value must be taken in the same manner in which the work was planned
Determining Earned Value
25NASA PM Challenge 2010
Discrete Earned Value Techniques
Milestone
• Identifiable milestone
• Milestone weighting should correlate to the resources required to accomplish the task –1 milestone per month
• 3 or more accounting periods
Percent Complete
• Least desirable
• Somewhat subjective
• If used, apply quantitative back-up data
50/50 Method
• No more than 2 accounting periods
• 50% EV at start
• 50% EV at complete
0/100 Method
• 1 accounting period
• No EV at start
• 100% EV at completion
26NASA PM Challenge 2010
• Work that is dependent on or related to the performance of other discrete effort in direct proportion (i.e., Quality Control effort in production)
• Percentage and base task must be identified
• Take the percentage of the AE for the base task, and that is what is earned
Earned Value Techniques
Apportioned Effort (AE)
Production Effort
Inspection Effort
27NASA PM Challenge 2010
• Contract effort that does not produce any definable end product(s)– General or supportive activities; i.e., management, administrative, etc.
• Used when no other EV method is applicable
• Over-use will distort project performance measurement
• Budget is spread over the scheduled period of performance
• Value of work is earned based on passage of time
• Planned Work Budget (BCWS) = Earned Value (BCWP) – Never a Schedule Variance
Level of Effort (LOE)
Earned Value Techniques
28NASA PM Challenge 2010
Characteristics of a “Good” Milestone
• Tied to a specific product or event
• Within the authority of the person who will perform the work
• Clear, objective criteria for measuring accomplishment
• Directly related to the control account statement of work
• Weighted in relationship to the time-phased budget
• Expressed in calendar dates and identified to control account and
summary schedules
29NASA PM Challenge 2010
What are some potential causes of deviations from a Project Plan?
• Poor beginning estimates
• Poor understanding of requirements
• Lack of risk assessment/planning
• Requirements creep
• Poor management
• Unforeseen technology difficulties
• Jumping to conclusions about causes
30NASA PM Challenge 2010
• Estimate At Completion (EAC)– Actual direct and indirect costs allocable to the contract + the estimate of all
costs for authorized work remaining, or Estimate to Complete (ETC).
– Estimated cost to complete the entire scope of work
– Supported with a rationale
– EAC trends usually do not reverse
– Best case, worst case, most likely required in the CPR
• Comprehensive Estimate at Completion
– Bottoms-up estimate reviewing all remaining work and a thorough estimate of that work
– Required annually
Future Performance Predictors
31NASA PM Challenge 2010
EVM - Key Data Elements
Actual Cost Actual Cost of Work Performed (ACWP)
Performance MeasurementBaseline (PMB)
Budget atCompletion (BAC)
$ Planned ValueBudgeted Cost for Work
Scheduled (BCWS)
Earned Value (Accomplished)Budgeted Cost for Work Performed
(BCWP)
Estimate At Completion (EAC)
Estimate to Complete (ETC)
Time Now
Contract or Project Budget Base
Management Reserve (MR)
CompletionDate
CostVariance
ScheduleVariance
StartDate
Variance atCompletion
For ProjectedSlippage (check
schedule)
OverBudget
32NASA PM Challenge 2010
PART 3
Performance Reporting, Variance Analysis and Data Usage
33
NASA PM Challenge 2010
• First we need to analyze current trends (current performance indicators)– What are our problem areas?
– What are the causes of our problems?
• Forecast the future based on performance to date– How can we stay on plan?
– What are the impacts to the project?
– Any technical, cost, or schedule surprises?
– When will we finish all the work?
– What is the estimated cost at completion?
– Is it achievable?
• Indicate areas for management action– What should we do now to mitigate problems?
Where are we headed?
34NASA PM Challenge 2010
Project Data Trace (Storyboard)
Page 2 Organizational Owner
WBS #
Pro
ject
M
anag
er
Req
uire
men
ts
Des
ign
Dev
elop
men
t
Test
Pac
kagi
ng
Tech
nica
l S
uppo
rt
1.2.7 x1.2.8 x1.2.9 x1.3.1 x1.3.2 x1.3.3 x1.3.4 x1.3.5 x1.4.1 x1.4.2 x1.4.3 x1.4.4 x
IMS(Resource Loaded)
WBSDictionary
wInsight
Mgmt Reporting & Data Analysis
Planned(BCWS) &
Accomplished (BCWP)
Actuals(ACWP)
SOW
WBS
OBS
Time / AccountingSystem
RAM
CAPs
WAD
CPR
EVMSystem
XML
Contractor Example
35NASA PM Challenge 2010
• Cost Variance (CV): The difference between BCWP and ACWP
– CV = (BCWP - ACWP)– Favorable is positive (Cost underrun)– Unfavorable is negative (Cost overrun)
• Cost Variance Percent (%): The CV divided by the Earned Value times 100
CV% = (BCWP - ACWP)
• Schedule Variance (SV): The difference between BCWP and BCWS– SV=BCWP-BCWS
– Favorable is positive (Ahead of schedule)
– Unfavorable is negative (Behind schedule)
• Schedule Variance Percent (%): The SV divided by the BCWS times 100SV% = (BCWP - BCWS)
Variance Calculations
BCWPX 100
BCWSX 100
36NASA PM Challenge 2010
• Variance at Completion (VAC): The difference between BAC and EAC– VAC = BAC - EAC
– Compares what the work was budgeted (BAC) to what it is currently estimated to cost (EAC)
– Favorable is positive (Forecasted under run)
– Unfavorable is negative (Forecasted over run)
• Variance at Completion Percent (%): The VAC divided by the BAC times 100
Variance Calculations (conclusions)
BAC X 100VAC % = VAC
37NASA PM Challenge 2010
─ A clear explanation of the cause
─ A comprehensive impact assessment
─ A corrective action plan
─ A revised Estimate at Completion (EAC)• Cost
• Schedule
Variance Analysis Objectives
38NASA PM Challenge 2010
“Root Causes” of Cost Variances
Cost Variance
Labor
Material &
Equipment
Other Direct Cost
(ODC)
Rate Variance(Earned Value-Actual Rate) x Actual Hours
Usage (Efficiency) Variance(Earned Hours – Actual Hours) x Earned Rate
Price Variance(Earned Price – Actual Price) x Actual Quantity
Usage Variance(Earned Quantity – Actual Quantity) x Earned Price
Price Variance(Earned Price – Actual Price) x Actual Quantity
Usage Variance(Earned Quantity – Actual Quantity) x Earned Price
39NASA PM Challenge 2010
Analysis Considerations
Favorable and unfavorable variances should be carefully evaluated. The project analyst should examine and understand the reasons for underruns, as well as overruns because they could be masking a serious problem.• Establishing Thresholds for Reporting and Analysis
– Establish threshold reporting based on risk and tailored to WBS by specific dollar amount and/or variance percentage
– Identification of threshold reporting should also consider program phase, risk and WBS criticality • Good Variance Analysis Identifies
– The Problem – Cause of the Problem – Impact to the Program (Cost, Schedule and Technical) – Corrective Action – Get well date
• Areas to Address– Poor initial planning or estimating – Technical breakthroughs or problems – Cost (or usage) of labor, material, or Other Direct Costs higher or lower than planned – Inflation and new labor contracts – Front-end loading
• Reconciliation Analysis– Other budget/funding documentation – Labor resource planning documentation – Schedules
40NASA PM Challenge 2010
EVM Reporting
Contains Five Reporting Formats:
• Format 1 – Work Breakdown Structure
• Format 2 – Organizational Categories
• Format 3 -- Baseline
• Format 4 – Staffing
• Format 5 – Explanations and Problem Analysis– Total contract
– Cost & schedule variance analysis
– Other analyses (6 topics)
– OTB & OTS (as applicable)
Contract Performance Report (CPR) - Required EVM data report on contracts where full EVMS compliance is required
41NASA PM Challenge 2010
• Cost Performance Index (CPI): The average cost efficiency with which work has
been performed to date
– Formula: CPI = BCWP / ACWP
– If CPI > 1.0, Effort is under-running budget
– If CPI < 1.0, Effort is over-running budget
• Schedule Performance Index (SPI): The ratio of the budgeted value of work performed to the budgeted value of work scheduled
– Formula: SPI = BCWP / BCWS
– If SPI > 1.0, Effort is ahead of schedule
– If SPI < 1.0, Effort is behind schedule
– Is only an indicator
– More important early in the project
Performance Indicators
42NASA PM Challenge 2010
Current Status Indicators
• How much of the program SHOULD have been accomplished at this point?– Percent Scheduled = (BCWS / BAC) X 100
• How many of the forecasted funds have been SPENT at this point?– Percent Spent = (ACWP / BAC) X 100
• How much of the program has been ACCOMPLISHED to this point?– Percent Complete = (BCWP / BAC) X 100
• What is the budgeted value of the work remaining?– Work Remaining = BAC - BCWP cum
43NASA PM Challenge 2010
• To-Complete Performance Index (TCPI):
– Cost (CPI) efficiency required to complete the remaining work
– TCPIEAC= (BAC-BCWP) / (EAC-ACWP)
• Performance Indicator Comparison
– CPI versus TCPI– Cost efficiency to date compared to cost efficiency to complete the
work remaining
– Note spread point when more than 20% complete
– Thoroughly evaluate all point degradations, e.g. 1.42 versus .67
– Very helpful in reviewing EAC rationales
Performance Forecasting
44NASA PM Challenge 2010
Independent Estimate At Completion (IEAC)
• There are many ways to calculate an IEAC
• An example: Independent Estimate At Completion (IEAC)
– IEAC = Actuals + (Work Remaining/Performance Factor)
• EAC = ACWP + [(BAC-BCWP)/Performance Factor]
– SPI as an indicator of EAC is better in first 50% (early stages) of a project
– There is no substitute for GOOD program information
• Common Performance Factors
– Cost Performance Index (CPI)
– Schedule Performance Index (SPI)
– WEIGHTED (80/20 or some other ratio)
Future Performance Predicators
45NASA PM Challenge 2010
Using EVM Data as a Predictor *
• When a contract is more than 15% complete & more than 10% over-run:– The over-run at completion will be more than the over-run incurred to date
• The final cost variance (in dollars or as a percentage) will be worse than
the cost variance at the 20% completion point.
* Based on OSD database of more than 500 major DOD contracts since 1977
46
NASA PM Challenge 2010
PART 4
Summary
47
NASA PM Challenge 2010
EVM Process Overview
Define the Work Plan the Work Collect Results
Measure Workin Process
Analyze Variances
Internal ChangesExternal Changes
Change Control
Changes
Risk?
Work the Plan
Implement CorrectiveActions
(Earned Value)
An iterative process used throughout the life of a contract.
PMB
(Schedule/Budget)
48NASA PM Challenge 2010
• Provides Project Management with objective, accurate and timely data for effective decision making (much more than plan versus actuals)
• Early warning of issues
• Allows more efficient use of management resources, by identifying and
focusing on problem areas
– Detail planning at levels where work is performed allows management attention
to be directed to areas where significant problems are indicated
• Provides better capability to trace problems to their source
• Provides better historical performance data
• EVM is an “Industry Best Practice” for project management (ANSI/EIA-748 – Earned Value Management Systems)
Benefits of using EVM
49NASA PM Challenge 2010
Conclusions about EVM
• An integrated management process for gaining valuable insight into project performance.
• Only management process that effectively integrates technical, cost, schedule and risk.
• Provides valuable quantifiable performance metrics for forecasting at-completion cost and schedule.
• Promotes early visibility to cost and schedule variance sooner than any other practice---reduces the ‘shock syndrome’
• Focuses on significant technical, cost and schedule problems---avoids swatting mosquitoes while the elephants charge on by
• Establishes funding authority’s confidence in program management
50NASA PM Challenge 2010
Questions?
51NASA PM Challenge 2010