integrating risk with earned value
DESCRIPTION
Connecting risk with earned value management presented at the Colorado Springs 2009 SymposiumTRANSCRIPT
INTEGRATING RISK WITH EARNED VALUE
PIKES PEAK REGIONAL CHAPTER PROJECT MANAGEMENT PRODUCTIVITY TOOLS
SYMPOSIUM SATURDAY, FEBRUARY 28TH, 2009
GLEN B. ALLEMAN
The notion of integrating cost, schedule, technical performance, and risk is possible in theory. In practice care is needed to assure credible information is provided to the Program Manager.
Today's Learning Objectives Cost, Schedule, Technical
Performance, and Risk Are
Inseparable
All numeric values of cost,
schedule, technical
performance, and risk
assessment are random
variables drawn from some
underlying probability
distribution
Managing these variables
must be done through the
eyes of a Risk Manager 2
There are
usually two
phases to many
projects …
1. Too early to tell
2. Too late to stop
3
The Fundamental
Program Management Question
Can we plan with
sufficient accuracy, to
allocate resources to
reduce risk while
meeting requirements,
and obtain timely
feedback in order to
adjust plans and
performance goals?
4
Cost
Technical
Performance Schedule
Funding margin for
under performance
Schedule margin for over
target baseline (OTB)
Schedule margin for
underperformance or
schedule extension
Over cost or
under
performance
Over cost or
over
schedule
Over schedule
or under
performing
Dependencies Between The Inseparable Variables
5
Earned Value
Management
Establishes project
performance status
and extrapolates
that information to
understand future
trends and
allocation of
resources needed
to successfully
meet project
milestones
Programmatic and
Technical Risk
Management
Looks to the
unknown future to
identify risks and
recommend early
action to be taken
to limit the threat or
maximize the
exploitation of
opportunities
Technical
Performance
Measures
A time phased
progress plan for
the achievement of
the Technical
Performance
Parameters (TPP)
that indicate key
areas for risk
reduction and
program success
INCOSE G2SEBOK APM EV-Risk Working Group, 2007
Some Simple Definitions
6
Cost
Technical
Performance Schedule
Systems Engineering states
deliverables performance and
associated risks
Integrated Master Schedule &
Earned Value Management
tracks risk activities
Structure WBS contains risk
mitigations and retirement efforts
The Integration of Risk and Earned Value is a Systems
Engineering Paradigm
7
But Many Times, The Information from Cost,
Schedule, Techncial Performance, and Risk
Management Systems Gets Mixed Up When We
Try to Put Them Together
8
Every Program Manager Should Know…
When will this project finish?
What is the statistical confidence in this date?
Is the project’s schedule realistic and achievable?
What will it cost when we are done?
How is the project performing against its plan?
What deliverables are slipping?
How are we going to get the deliverables back on schedule?
What does past performance say about the future performance?
What is the impact on the schedule of any change requests?
Is there enough schedule slack and cost reserve to cover the risk?
Where is the risk in the schedule?
How can we get the work done sooner to reduce risk?
How can we recover from any foreseen delays?
How can we work around a problem?
9
SV
The First Step is to Start with Key Elements of Earned Value
B C W S
CV
The Work planned, performed, and the cost of
performing that work, is the common variable across
each Earned Value variable
A C W P
B C W P
10
EVM Analyst’s EAC Range
• Addresses Risk
• Based on CPI & SPI trend analysis not on
plugging numbers into a formula
• Incorporates technical performance, schedule
progress, CAIV and other program information
EAC Floor Performance Factor
EAC Ceiling Performance Factor
cum
cum cum
BAC BCWP BACEAC ACWP
CPI CPI
cum
cum cum cum
BAC BCWP BACEAC ACWP
CPI SPI CPI
Some Useful EV Information
http://www.daytonaero.com/Files/resource/31.pdf
11
ANSI 748B Says on Page 1
Plan all work scope for the program to completion.
Integrate program work scope, schedule, and cost objectives into a baseline plan against which accomplishments may be measured.
Objectively assess accomplishments at the work performance level.
Analyze significant variances from the plan and forecast impacts.
Provide data to higher levels for management decision making and implementation of management actions.
12
Some Criteria for Successful EVMS Beyond Full Compliance of the 32 Criteria
Define a Work
Breakdown Structure
Identify the
Organizations
doing the work
Integrate WBS and
OBS into a RAM
Schedule all
Planned Work
Indentify Products
and Milestones
Time Phase the
Budget
Record all Direct
Costs
Determine all
Variances
Sum These
Variances
Manage Action
Plans
Incorporate Changes 13
At The Same Time, Risk Management is
Commonly Misunderstood
It’s not about random chance, it’s about defining
mitigations and retirement plans in the presence of
uncertainty
14
1. Hope is not a strategy 2. No single point estimate of cost or schedule can be correct 3. Cost, Schedule, and Technical Performance are inseparable 4. Risk management requires adherence to a well defined process 5. Communication is the Number One success factor
Five Fundamental Principles of Risk Management
15
DoD’s Approach to Successful Risk Management
16
But ANSI 748B Also Says …
Identify physical products, milestones, technical
performance goals, or other indicators that will be
used to measure progress.
So why the disconnect between Risk and Earned
Value?
Isn’t the measure of risk, the rate of risk reduction
an “…other indicators…”
17
Define and
Organize the
Work
Establish MR
Issue Budget
Authorize Plan
Establish PMB
Authorize Work
Measure
Performance
Analyze Results
Plan Risk
Activities
Perform Risk
Assessment
Develop Risk
Handling Plans
Assign
Responsibilities
Monitor and
Communicate
Update Risk Register
Revisions & Change Control
I think you
should be more
explicit about
the steps here.
Integrating
Earned Value
and Risk
Management
18
SOME ACTIONABLE DETAILS FOR PUTTING THESE FOUR CONCEPTS TOGETHER
In Theory there is no difference between Theory and Practice.
In Practice there is — Yogi Berra
So much for our strategy of winning
through technical dominance
There are two types of “uncertainty” on any
sufficiently complex program
– Technical – uncertainty about the functional and
performance aspects of the program’s
technology that impacts the produceability of
the product or creates delays in the schedule
– Programmatic – uncertainty about the duration
and cost of the activities that deliver the
functional and performance elements of the
program, independent of the technical risk
We’re interested in connecting the two in the schedule and cost
model(s)
– When the technical uncertainty arises what is the impact on the schedule
and cost?
– When the schedule or cost uncertainty arises what is the impact on the
functional and performance aspects?
The Difference Between
Technical and Programmatic
Risk
20
Deterministic Versus
Probabilistic Deterministic
• Each activity has a planned value
• For the schedule each task has a
predecessor and a successor.
• The longest path through the
network is the critical path
• The total duration of the project is
a fixed value - it is deterministic
• The total cost is the sum of all the
activity costs
• Risks are defined and handled as
static entities
Probabilistic
• The program elements are not
random, but they are random
variables drawn from a probability
distribution.
• Three point estimates "can" be
used to describe task duration
random variables
• The total duration of the project is
a random number
• The total cost is a random number
• Risks are stochastic processes
that have probabilistic outcomes
for cost, schedule and technical
performance
21
Lack of predictive variance analysis
Untimely and unrealistic Latest Revised Estimates (LRE)
Progress not monitored in a regular and consistent manner
Lack of vertical and horizontal traceability cost and schedule data for corrective action
Lack of internal surveillance and controls
Managerial actions not demonstrated using Earned Value
Inattention to budgetary responsibilities
Work authorizations that are not always followed
Issues with Budget and data reconciliation
Lack of an integrated management system
Baseline fluctuations and frequent replanning
Current period and retroactive changes
Improper use of management reserve
EV techniques that do not reflect actual performance
Without these Principles The Program Train Wreck Starts When…
Mary K. Evans Picture Library
22
AN INTRODUCTION TO SOFTWARE ENGINEERING INSTITUTE’S CONTINUOUS RISK MANAGEMENT (CRM)
CRM is the Software Engineering Institute’s framework for managing risk in the context of system integration, COTS based product development, and the management of these activities.
Continuous Risk Management has Six Components
http://www.sei.cmu.edu/risk/index.html
24
Putting Continuous Risk Management Work
CRM Activity Project Representation
Identify Risk items with IMP/IMS #’s, CA/WP & resource
assignments
Analyze Risk management responsibilities assigned
Plan Mitigation plans with durations and resource
assignments
Track Status reported from Risk Management to IMS
Control Risk tasks reporting in weekly status process
Communicate Integrated Master Schedule (IMS) status reporting
as Physical Percent Complete
25
This All Comes Together in The Risk Registry
http://www.mitre.org/work/sepo/toolkits/risk/ToolsTechniques/RiskMatrix.html
26
PHYSICALLY CONNECTING EARNED VALUE WITH RISK MANAGEMENT
Some Historical Background†
Cost and schedule growth have been persistent
problems for decades
The use of Earned Value is common in DoD
environments
As is the use of Risk Management
It’s the integration that is missing
† “Integrating Risk Management with Earned Value Management,” NDIA
Report, www.ndia.org
28
The Core Problems Before We Start
Earned Value is a measure of quantity not quality
Risk measures the probability of an unfavorable
occurrence but rarely monetizes this outcome
Risk management focuses on dealing with future
events
Earned Value focuses on reporting past
performance
29
Starting with the Individual Elements
Identify Analyze Plan Track Control
Organization Identify risk
owners
Risk board
assessment of
impact
Mitigation and
retirement
owners
Connect PMB
with Risk registry
Vet members of
the leadership
Planning &
Budgeting
Identify
uncertainty and
response
Probability
impact in
schedule
PMB contains all
risk active
activities
Baseline the risk
registry
Transfer
mitigation to PMB
and close risk
Accounting
Assign risk to
CA/WP cost
baseline
Probability
impact on cost
Risk budgets
defined and
allocated
Risk Board
management of
cost impacts
Transfer through
EV change
control processes
Analysis
Asses impacts
and
dependencies
Risk registry
contains
probabilities of
cost and schedule
Risk Board and
IMS Board
impact analysis
Manage budget
assignment by
risk level
Revisions Keep historical
records
Update ETC from
risk management
Updated PMB
from risk
activities
Risk review Revise PMB
forecast
30
First, A (Notional) Big Picture
Risk ID Traceable to
work elements in the
IMS within the WBS
Risk reduction
waterfall metrics
connected to Earned
Value program
performance
Standard program
performance of TCPI
and IEAC
31
EV Data
EV Data
One Approach to Integrating Risk Management with Earned Value
Program Manager
Functional Managers
Work Package Managers
Individuals/Team Members
Identify
Analyze Review Prioritize
Evaluate Classify
Track
Plan Approve plans
Recommend actions Develop plans
Control Integrate across
functions Reprioritize Authorize project
resources
risks
Top N risks Decisions
Assign Responsibility
Required Indicators
Trends
Risk Status
Control Integrate Reprioritize Authorize Functional area
resources
EV Data
1
3 2
Top N risks
32
EXAMPLE SCHEDULE RISK ANALYSIS
Monte Carlo tools model the task durations with random variables for the durations and build “pictures” of the likelihood of a task completely on or before a specific date.
@Risk and Risk+ Sample Screens
34
Schedule for a Monte Carlo Risk analysis starts with a
credible schedule and defines the probabilistic behavior
of each activities and how it drives the deliverables
The Risk+ tool sets the upper and lower bounds of the possible durations 35
Date: 2/25/2009 3:34:12 PM
Samples: 300
Unique ID: 30
Name: Final Testing
Completion Std Deviation: 5.51d
95% Confidence Interval: 0.62d
Each bar represents 2d
Completion Date
Fre
qu
en
cy
Cu
mu
lative
Pro
ba
bili
ty
Tue 2/11/03Mon 1/20/03 Mon 3/3/03
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
0.02
0.04
0.06
0.08
0.10
0.12
0.14
0.16 Completion Probability Table
Prob ProbDate Date
0.05 Wed 1/29/03
0.10 Fri 1/31/03
0.15 Tue 2/4/03
0.20 Wed 2/5/03
0.25 Wed 2/5/03
0.30 Thu 2/6/03
0.35 Fri 2/7/03
0.40 Mon 2/10/03
0.45 Mon 2/10/03
0.50 Tue 2/11/03
0.55 Wed 2/12/03
0.60 Thu 2/13/03
0.65 Fri 2/14/03
0.70 Fri 2/14/03
0.75 Mon 2/17/03
0.80 Tue 2/18/03
0.85 Wed 2/19/03
0.90 Thu 2/20/03
0.95 Tue 2/25/03
1.00 Mon 3/3/03
The output of Risk+ is a Probability Distribution Function and a
Cumulative Distribution of all the possible dates that “watched”
activity could take.
The result is a picture of the Confidence that the target date of
2/10/3 – can be met.
It shows 40% – which is not good
36
So with this information we can ask …
What cost impact will there be?
What resource impacts?
What technical dependencies?
What mitigation or retirement plans must be in
place to increase the probability of success to
something greater than 40%?
What other interdependencies are there in the
program?
37
Program Management is Risky Business
Traditional Approaches Are Seriously Flawed
Risk Management Is Not About Preventing Risk
Risk Management Is About Managing In The Presence Of
Risk
This means managing Cost, Schedule, and Technical
Performance
38
Source Materials
Interfacing Risk & Earned Value: Management Reserve,
“Technical Performance Measurement, Earned Value, and Risk Management: An Integrated
Diagnostic Tool for Program Management,” Commander N. D. Pisano, SC, USN, Program
Executive Office for Air ASW, Assault, and Special Mission Programs (PEO(A))
Practical Risk Management: The ATOM Method, David Hillson and Peter Simon, Management
Concepts.
Project Risk Analysis and Management Guide, APM Publishing.
“Formal Risk Management,” DACS Gold Practices
“Quantify Risk to Manage Cost and Schedule,” Fred Raymond, Acquisition Review Quarterly,
Spring 1999, pp. 147–154
Probabilistic Risk Assessment Procedures Guide for NASA Managers and Practitioners, Office of
Safety and Mission Assurance, April 2002
Development Of Risk Management Defense Extensions To The PMI Project Management Body
Of Knowledge – Tutorial,” Edmund Conrow, Acquisition Review Quarterly, Spring, 2003.
Effective Risk management: Some Keys to Success, Edmund Conrow, American Institute of
Aeronautics and Astronautics, 2000.
39