guidance on which probability distributions for different levels of maturity - dr john ahmet...
DESCRIPTION
This is one of many excellent presentations given over the last three years of the eVa in the UK series. They can also be found in the archive at: http://evaintheuk.org/archive along with back-copy video footage in http://evaintheuk/pmchannel EVA19, the long established Earned Value conference, has this year described its theme as looking at a project management ‘ABC’ – Agile, Benefits and Complex. The four day event, which returns to the Armourers Hall, runs from the 19th to 22nd of May with the flagship conference being held on 20th and 21st May and workshops before and after. The conference will look at how this ‘ABC’ can be made to work within a portfolio and how agile fits into major and minor projects. It will investigate how to manage the relationship between portfolio benefits and project budgets, and whether complex projects even exist. Conference organiser and APM chairman, Steve Wake says: “Currently there is little evidence that this ‘ABC’ is being effectively deployed and managed. This conference aims to address that concern through EVA’s trademark blend of learning and professional development. Case studies and unusual presentations, delivered by top-notch speakers and experienced practitioners, will again engage and entertain the audience. We’ve used string quartets to illustrate points in the past and this year we will be using a Blues band for the first time.” Speakers across the two days include many familiar faces from the APM events programme including; Adrian Pyne of the APM ProgM SIG ‘Changing the project wasteland with a portfolio culture that works,’ APM Honorary Fellow Tim Banfield Director at the Major Projects Authority and Stephen Jones, Sellafield and Planning Monitoring and Control Specific Interest Group (PMC SIG) and Carolyn Limbert of the APM PMC SIG to talk about agile, benefits and complex. Peter Taylor, the Lazy Project Manager will be presenting on “The project manager who smiled” and the ever popular Stephen Carver will present the leadership lessons that can be learnt from Alfred the Great. In addition, there will be speakers from AIRBUS, TfL, Bloodhound, Heathrow T2 and London Tideway Tunnels. The conference will be supplemented by a number of workshops being held at the Chartered Institute of Arbitrators, Bloomsbury Square on Monday 19th and Thursday 22nd May 2014. 'eVa in the UK' http://evaintheuk.org is building a reputation, brand and a learning legacy for the Project Management Profession. The event series is now in its nineteenth year. It is almost as if it all kicked-off when Steve Wake was in short trousers and knights roamed the land on their chargers! #eva19 is an excellent example of Listening, Learning and Leading #apmLLL in action, and great opportunity for professional development. I would encourage anyone who is interested in 'Building a better Project Manager,' to take a look at the web site, and book your place and get involved.TRANSCRIPT
Guidance on which probability distributions should be selected across different levels of maturity
Sponsors:
Dr. John Ahmet ERKOYUNCU
Project Controls
11th June 2013
Agenda
• Introduction• Methodology• Literature Review• Survey Results • Data Analysis and Guideline• Validation• Conclusion and Future work
Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and
Future work
Introduction
Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and
Future work
Probability
Cost £500 700 900 1000
25%
Stochastic Methods
Three point estimating
Run a Monte Carlo Simulation
Apply a distribution to cost drivers
Methodology
Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and
Future work
• Searched key words• Identified main
journals
Literature Review
• Create questionnaire• Conduct SurveySurvey
• Analyse Case Studies• Brainstorm GuidelineData Analysis
• Analyse Case Studies• Create guideline tool
Guideline Development
Validation
Deliverable 1
Deliverable 2
Deliverable 3
Background
• The concept of maturity
• Cost estimate classification systems
• Probability distributions
5Aim and Objectives Methodology Literature
Review Survey Results Data Analysis Guideline Validation Conclusion and Future work
Concept of maturity
6
Data maturity:Your understanding and confidence regarding a cost element
Project maturity:Extent and accuracy and comprehensiveness of available information and
data
Aim and Objectives Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and
Future work
AACEI Classification system
AACEI : Association for the Advancement of Cost Engineering International
Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and
Future work
Probability distributions
• Basics
• Properties
• Continuous vs. Discrete
• Bounded vs. Unbounded
• Correlation
Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and
Future work
Probability distributionsContinuous vs Discrete
9
Probability
Cost £
Continuous DiscreteVS
Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and
Future work
Probability distributionsBounded vs. Unbounded
10
Probability
Cost £
Probability
Cost £
Unbounded BoundedVS
Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and
Future work
Commonly used continuous distributions
• Uniform• Triangular• Beta• Beta PERT
Probability
Cost £
MaxMin
Probability
Cost £Min Most Likely Max
Probability
Cost £Most Likely
+ Standard deviation
Probability
Cost £Min Max
+ 2 shape parameters
• Normal• Lognormal• Weibull• Exponential
Uniform
Triangular
NormalWeibull
Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and
Future work
Correlation
Basics:Relationship between two cost elements
Effects:Considerable impact on the cost estimate
Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and
Future work
Outcomes of literature review
• Identified the project phases depending on maturity
• Developed a comprehensive list of distribution and characteristic
Challenge: How to select a suitable probability distribution for cost elements across maturity phases?
Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and
Future work
Capturing current practice
Survey Aim
Understand the current practice in industry towards cost estimation, methodology
and project maturity classification systems.
• Online survey• ACostE members• Link project maturity with probability distribution selection• 76% large enterprise• 80% of respondents, have more than 5 years experience in cost estimation
Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and
Future work
Survey results I
6%
39%
2%
47%
6%No consideration of project matur-ity
Using subjective judgement rather than formal system
Using general cost estimate classi-fication system
Using customised cost estimate classification system
Others
Do you take into account the maturity of a project to develop an estimate?
There is no standard for cost estimate classification system.
Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and
Future work
Requir
emen
ts a
nalys
is
Projec
t Init
iation
Plannin
g
Conce
pt d
evelo
pmen
t
Detail
Dev
elopm
ent
Inte
grat
ion a
nd te
st
Imple
men
tatio
n
Opera
tion
and
main
tena
nce
Dispos
al0
5
10
15
20
25
30
In which Life cycle stage do you assess project maturity?
Survey results II
Early stage
Project maturity assessment seems to be critical during early project life cycle phase.
Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and
Future work
Survey results III
'''This table represents the project maturity phases.''' Least mature More mature
Name of the project maturity
Initiation/Conceptual/ Scoping Feasibility Preliminary design Detailed design Definitive/Execution
Percentage of project completion 0-2% 1-15% 10-40% 30-70% 50-100%
Purpose of estimate at project maturity level
Concept Screening FeasibilityBudget,
Authorization or Control
Control Check Estimate or Control
Expected accuracy range
- 20 to 50% / +30 to 100%
- 15 to 30% / +20 to 50%
- 10 to 20% / +10 to 30%
- 5 to 15% / +5 to 20%
- 3 to 10% / +3 to 15%
Project maturity classification system
Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and
Future work
Table is recommended for industries to develop their own project maturity classification system.
Survey results IV
1 2 3 4 50
2
4
6
8
10
12
Triangular
Uniform
Normal
Beta
Lognormal
Exponential
Which probability distribution do you use at different project maturity phases?
• Top three probability distributions used are Triangular, Uniform and Normal.
• The distribution selection does not seem to vary across phases
Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and
Future work
• No standard cost estimate classification systems are available currently in industry.
• Lack of awareness regarding selecting distribution• Guidelines are needed for industry.
Survey outcomes
Challenge: How to select a suitable probability distribution for cost elements across maturity phases?
Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and
Future work
Data Analysis
Aim
Support the creation of probability distributions selection guideline.
• Cases • Cost models from sponsors• Based on three points estimate• Different maturity levels
• Tools• Crystal Ball & Arrisca
• Monte Carlo simulation
Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and
Future work
• Define forecast
• Define assumptions
• Set simulation parameters
• Run Monte Carlo simulation
• Analyse the results
Data Analysis: Workflow
Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and
Future work
Data Analysis:Findings I
Symmetric datasetAsymmetric dataset
• Different probability distribution could lead to different cost estimation outcomes
• Symmetric dataset could have similar results with different probability distributions
• Some distributions have specific properties
Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and
Future work
Data Analysis:Findings II
Statistics Minimum Maximum 10-90% Interval
Changes - 9.23% + 10.89% + 326%
Estimation range changes when correlation is considered
Correlation could significantly change the cost estimation range
Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and
Future work
Data Analysis:Findings III
Comparison of outcomesTornado chart to determine the key drivers
Unsuitable distribution for the key cost drivers could lead to much worse outcomes
Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and
Future work
• Correlation should always be considered from the beginning
• Prioritise key cost drivers
• Dataset features should be considered
• Distribution parameters should be set with caution
• The selected distribution should be plotted and validated
Data Analysis:Conclusion
Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and
Future work
Guideline Overview
Aim
Guide the cost engineer to select the most suitable probability distribution for cost
elements.
• Guideline• Guideline tool• Guideline tool Demo
Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and
Future work
Guideline
Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and
Future work
Yes
Guideline Flow chart
Adapted from: Strategic Risk Taking: A Framework For Risk Management by Aswath Damodaran (2007)
Sensitivity Analysis: Identify key drivers
Apply correlation
Is the data continuous?
Yes
Is the estimate symmetric?
Yes
How confident are you in your boundaries?
Neg Binomial
Strong positive
Negative Skewness
No
Yes No
Can you estimate outcomes and probability?
Estimate probability distribution.
Is the estimate symmetric?
Yes No
Do you have most likely value?
What is the skewness of your distribution?
Yes No
Binomial Discrete Uniform Geometric Hypergeometric
Positive Skewness
Confident
Very confident
Less confident
Triangular,BetaPERT, Beta
Trigen, Logistic
No
Uniform
Triangular, Trigen, BetaPERT, Lognormal, Weibull, Gamma
Positive Skew
Triangular, Trigen, BetaPERT, MAX Extreme, Exponential
No
What is the skewness of your distribution?
Strong positive Skew
Negative Skew
Triangular, Trigen, BetaPERT, MIN Extreme
Trigen, Normal
Do you have a most likely value?
Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and
Future work
Data Maturity
Triangular
Beta-PERT
Beta Dat
a M
atur
ity
Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and
Future work
Guideline Tool
• Readme• Start• Continuous• Discrete• Guidance• Explanations
Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and
Future work
Guideline Demo
Is the estimate symmetric?
Yes
How confident are you in your boundaries?
Confident
Trigen, Normal
Do you have most likely value?
Yes
Is the data continuous?
Yes
Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and
Future work
Validation
Survey“Think it is quite a good document!”
Section Leader, BAE Systems Submarines
Data Analysis“No areas for improvement but areas which can be expanded in future research.”
Principal Reliability Specialist, BAE Systems ATC
Guideline“Yes I will be looking at that. I will be promoting this with other business units who haven’t had direct visibility of this.”
Principal Reliability Specialist, BAE Systems ATC
Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and
Future work
Conclusion
• Literature review• No consideration of project maturity in probability
distribution selection• Survey
• Cost estimation classification system• Data analysis
• Data features are the main influence on distribution selection
• Correlation assessment • Sensitivity analysis to prioritise cost drivers
Introduction Methodology Literature Review Survey Results Data Analysis Guideline Validation Conclusion and
Future work
Acknowledgements
• Andy Langridge, Price Systems
• Richard Parker, BAE Systems
• Tony Higham, BAE Systems
Thank you for your attention!