(risk-2510) modelling realistic outcomes using integrated cost … · modelling realistic outcomes...
TRANSCRIPT
RISK-2510
(RISK-2510) Modelling Realistic Outcomes using
Integrated Cost and Schedule Risk Analysis
Colin H. Cropley
RISK-2510
PLEASE USE MICROPHONE FOR ALL QUESTIONS AND COMMENTS!
2
RISK-2510
BIO of Colin Cropley
• Colin Cropley is Managing Director of Risk Integration Management Pty Ltd (RIMPL), an Australian company focused on large project quantitative project risk analysis
• A chemical engineer with over 40 years’ experience in project management, controls & risk management
• He has conducted risk management processes, schedule and cost risk analyses and training for many major companies since 2003
• He was Chairman of his state Primavera Users Group from 1997 to 2009 and has guest lectured in post-graduate project management courses since 1992
• He is a member of AACE International, the Australian Cost Engineering & Risk Engineering Societies and PMI
• Colin is convinced by the overwhelming scientific evidence that climate change is accelerating and that we have to change our behaviour urgently to avert the most serious consequences
3
RISK-2510
OUTLINE OF PRESENTATION PURPOSES OF THE PAPER
4
RISK-2510
Presentation Outline
• Purposes of the paper • Nature of this Integrated Cost & Schedule Risk Analysis
methodology (IRA) • When it is most effective • Extend IRA to IRRA to model operation of project assets • Example IRRA model: Generic Tollway PPP • Strengths and weaknesses of IRA/IRRA methodologies • How well IRA methodology addresses RP40R-08 criteria
for quantifying project risk and contingencies
5
RISK-2510
Purposes of the Paper
• To describe this Integrated cost & schedule Risk Analysis methodology (“IRA”)
• To relate it to the requirements of this project risk quantification methodologies conference track : – Referencing AACE® Recommended Practices – Contexts in which IRA has greatest advantages – Strengths and weaknesses and how well it addresses
RP 40R-08 Contingency Estimating – General Principles • To extend the IRA methodology beyond project startup to
cover the risks and uncertainties of asset operation • To illustrate the extended methodology modelling a Build Own
and Operate tollway project and using probabilistic NPVs and IRRs, forecast investment resilience
6
RISK-2510
WHAT IS THIS VERSION OF INTEGRATED COST & SCHEDULE RISK ANALYSIS
(IRA)? - COMPONENTS AND PROCESS
7
RISK-2510
IRA Components
• The IRA Methodology can be defined as “the simultaneous Monte Carlo Simulation (MCS) analysis of cost and duration uncertainty using a critical path schedule representing the project scope and strategy overlaid with the project cost estimate and with all the known significant project cost and duration impact risks mapped into the model.”
• As per the above definition, there are three components: – Critical path schedule representing project scope & strategy – Project Estimate, split into fixed (time independent) and variable
(time dependent) costs – The most significant known risks with time and cost impacts:
systemic risks and discrete and risk factor project risks • All uncertain parameters (durations, costs and risk impacts)
have probability distributions applied
8
RISK-2510
IRA Process – Flow Diagram
• The IRA Process follows the following flow diagram:
9
Receive / Critique Project Schedules,
Estimate & Risk Registers
Conduct Risk Assessment / Identification Workshops or
Interviews
Incorporate Required Schedule Changes Identified as Part of
above Steps
Incorporate Duration Uncertainties &
Duration Correlation Model
Incorporate Weather Model (if applicable)
Map Materials & Labour Costs to
Schedule Including Cost Correlation
Model
Map Risk Events & Risk Factors to
Appropriate Schedule & Cost Model
Elements
Analyse Model; Validation Checks
and Initial Reporting
Incorporate Project Team Feedback, Re-Analysis & Reporting
RISK-2510
IRA Process – Receive & Assess Model Inputs
• Receive or build CPM schedule to model project – Detailed Integrated Master Control Schedule, or – Summarised schedule representing logic & strategy – Technically well constructed
• Receive project estimate – Sufficient detail to represent differing risk profiles – Sharing same assumptions as schedule – Able to split accurately into fixed and variable costs
• Review Project Risk Register – Risks expressed for QRA (single cause, metalanguage) – All significant risks with time and cost impacts – Ensure systemic and discrete risks and risk factors covered
10
RISK-2510
IRA Process – Uncertainty/Risk Assessments
• Systemic risk due to organisational limitations leading to flawed project delivery must be assessed based on past relevant project performance: – Preferably by the organisation, or – By similar organisations in same sector and similar scale,
allowing for capabilities specific to this project • Duration and cost ranges and correlation models to be
assessed by Subject Matter Experts (SMEs) – Representing project uncertainties and – Any systemic risks not covered by above empirical data
• Discrete risk events from project risk register and above schedule and cost ranging assessed by SMEs – Assessed for suitability for Quantitative Risk Analysis
11
RISK-2510
IRA Process – Build the model
• Overlay the estimate on the schedule: – Use hammocks linked to, but separate from, project
tasks describing scope of work covered by the costs – Separate the fixed and variable costs, but both to be
linked to applicable tasks • Apply duration and cost ranges from SME assessments • Apply probabilistic weather calendars (if applicable) • Apply schedule and cost correlation models • Apply Systemic risk schedule and cost impacts to model • Map significant discrete treated project risks to model,
correlate as applicable
12
RISK-2510
IRA Process – Analyse and Report
• Treated risk-loaded IRA model validated to ensure it represents project, then Monte Carlo Simulation applied
• Results checked to ensure they make sense before being distributed to stakeholders for review; should include: – Date & Cost Histograms and summaries – Cruciality and cost tornado diagrams (what is driving)
• Review results with stakeholders; agree any changes • Re-analyse and re-review; repeat until acceptable • Complete with Quantitative Exclusion Analysis (QEA):
– Highest sensitivity drivers excluded, re-analysed; time and cost differences at agreed probabilities reported
• QEA allows for further treatment and risk optimisation • Full reporting plus discussion of results, conclusions and any
recommendations then follow 13
RISK-2510
WHEN IS IRA APPROPRIATE, MOST EFFECTIVE?
14
RISK-2510
When to use IRA
• IRA can be used when a project estimate, a project plan and a project risk register exist and are aligned
• This can be early in the development of a project, even at pre-Concept or Select Stage (see Risk-2592), where cost and time-to-market differences can be compared
• The comparative advantage of the methodology may be best just prior to Financial Investment Decision (FID): – When inputs most mature and should be fully aligned – When opportunities to optimise risk are high
• IRA can also be used during project execution to verify project on track or identify where changes needed
15
RISK-2510
QUANTIFYING CONTINGENCY FAILS TO ASSESS SIGNIFICANT CAUSES OF PROJECT FAILURES
- EXTEND IRA TO MODEL PROJECT ASSETS
16
RISK-2510
Many projects fail due to impacts on assets
• Frequent causes of loss of viability of a project are not threats to the project but to assets produced by the project. Examples include: – Demand and thus prices for most minerals and metals have
fallen away, bringing to an end, from around 2013, the mining boom projects that started from around 2003
– Huge increases in production of unconventional oil & gas in America halved prices of crude oil and natural gas and consequently, traded LNG, from 2014, causing cancellation of all projects not irrevocably committed
– Australian tollways failed due to unrealistic traffic forecasts – IT projects failed due to products overtaken by technology:
E.g., Minicomputers, Nokia & BlackBerry cell phones 17
RISK-2510
Need for modelling of Assets as well as Projects
• Clear need for ability to model not just the project but the operation of the produced assets of the project
• This would enable the resilience of the project plus assets to be tested by risks to: – Asset operating revenues, – Operating costs, as well as – The project itself.
• IRA methodology has been extended beyond project startup to cover asset operations as well, in the one model, to bridge the identified gaps in modelling
18
RISK-2510
Extension of IRA to IRRA
• IRA methodology, built on CPM integration of time and cost, can produce probabilistic cash flows
• Extending the modelling beyond startup of the project to incorporate operation of the assets for their economic life allows modelling of: – Revenue uncertainties and risk events – Operating costs uncertainties and risk events
• This is in addition to the project risks and uncertainties already being modelled
• Including revenue and costs enables probabilistic Net Present Values (NPVs) and Internal Rates of Return (IRRs) to be produced and scenarios to be evaluated
• This is “Integrated capital & operating costs, schedule & Revenue Risk Analysis” (IRRA), used in the following example
19
RISK-2510
EXAMPLE IRA / IRRA MODEL: THE GENERIC TOLLWAY PROJECT (“GTP”)
- A BUILD, OWN & OPERATE PUBLIC PRIVATE PARTNERSHIP PROJECT
20
RISK-2510
Objectives of GTP Model
• The GTP CPM-based model of the design, construction, operation and possible expansion of a tollway was created to demonstrate several capabilities of IRA/IRRA: 1. Capacity of CPM-based project risk quantification to assess
contingency and incorporate systemic risk; 2. Capacity of CPM-based modelling to respond to the
occurrence of risk events through the application of risk treatments including logic to alter responses if circumstances change during the simulation; and
3. Model viability of a project and the assets created by the project in alignment with AACE’s Total Cost Management (TCM) Framework
21
RISK-2510
Generic Tollway Project Description
• Generic Tollway nominally 10km in length running east-west from the western edge of city’s Central Business District (CBD) to a major arterial road at its western end
• Westernmost 2km of tollway consists of two tunnels: for three inbound lanes and for three outbound lanes
• Other 8km of tollway runs east from tunnels to western edge of the CBD, crossing over two major roads
22
Western EasternArterial CBD
Generic Tollway Project SchematicSection 1Tunnels
Section 2Major road A Major road B
2 km 8 km
RISK-2510
Generic Tollway Project Plan
• High level tollway project schedule created based on past actual projects. Summary barchart shown below:
23
RISK-2510
Generic Tollway Operations
• Schedule was extended past commencement of tolling to cover 25 years of operation
• Allowance made for growth in use of tollway leading to threat of increasing congestion and travel times
• Treatment is to add extra inbound and outbound lanes – Extra Lanes Project (ELP)
• Decision on applying to add extra lanes depends on volume of traffic per day and toll revenue growth – Could be after just 5 years operation, or not in 25 years
• Also a threat that Govt. may install commuter rail line parallel to GT, reducing commuter traffic and deferring ELP beyond 25 years
24
RISK-2510
Generic Tollway Operations Plan (portion)
• First nine of 25 years operation shown summarised below, including the possible extra lanes expansion:
25
RISK-2510
Extra Lanes Project (ELP)
• Included in Operations phase of schedule are ELP Construct, Commission & Operate (for 10 years) tasks that only exist if called by a macro that runs in each iteration
• The macro only initiates the ELP if tollway revenue in a year reaches a threshold specified as a modelling input
• Depending on the threshold, the ELP could be initiated in any tolling year from Year 1 to Year 16, or not at all in that iteration
• If it is initiated the ELP takes 3 years to be approved and built and operates for 10 years
• A growth formula for revenue is included and a high degree of uncertainty is provided around the revenue and operating costs
• The following slide shows the ELP Project and Operations below Base Operations 26
RISK-2510
ELP Project & Operations Plan
27
RISK-2510
Threat that Metro Rail Line may be built
• In the Schedule Cost Structure is a milestone, at the start of Base Operations Year 6, that the Government is considering building a rail line along the GT route
• This has probabilistic branching to two alternative tasks: – a decision after ~ 1 year not to proceed with the rail line – a five year activity to design and construct the rail line,
then operating tasks from Base Operations Years 11 to 25 • The probability of proceeding can be set at 0% to 100%;
not proceeding is [100%-Proceeding Probability] • If the rail line proceeds, the macro for the ELP stops the
ELP from being constructed at any time in that iteration
28
RISK-2510
Possible Rail Line Plan
• The following screen dump shows the possible rail line design & construction, then operation from Year 11 :
29
RISK-2510
Risks in GTP IRRA Model
• Three kinds of risk were included in GTP IRRA model: 1. Systemic Risk that project delivery organisation and team may
perform less than optimally delivering project; 2. Estimating uncertainty in time and cost - ranges assigned to line
items and task durations rather than single values; 3. Project specific risk events identified by project team as applying to
this project and operation of this asset. • Systemic Risk has been modelled based on the parametric
method advocated by Hollmann3, adapted for the transport sector, for use where previous performance is unavailable
• The Hollmann model is based on data asserted to cover all uncertainty other than scope changes and severe risk events, a major part of which covers organisational project delivery flaws
• Many (most?) organisations do not recognise systemic project delivery risk
30
3 Hollmann, JK, “Project Risk Quantification”: Chs 11, 15 & Appendix A
RISK-2510
Systemic Risk Modelling • Values have been substituted into a Spreadsheet
Template, made available with Hollmann’s book, for cost growth and for schedule slip, as shown below (by permission)
• It calculates mean slip and three point distribution values
31
RISK-2510
Adjustment to Hollmann Hybrid Method
• Hollmann recommends applying the parametric output costs and duration distributions to a single systemic risk activity of P50 duration and cost at the end of a CPM-based ICSRA schedule, using the example in RP 57R-094
• This simple approach also involves removing all ranging of cost and schedule uncertainty from the CPM model
• But it is not workable where the P10 duration of the systemic risk task is negative, as it is in this case
• Problem is overcome by applying a duration risk factor across all project tasks using three point distributions from the parametric modelling. This models time-dependent cost contingency due to time-cost inter-dependency of IRA model
• Time independent cost is modelled by a cost hammock over the project of fixed cost contingency distribution from parametric modelling. This is correlated 100% to duration risk factor
32
4 “Integrated Cost and Schedule Risk Analysis using MCS of a CPM Model”, AACE International®
RISK-2510
Project and Asset Specific Risk Events
• Treated risk events from the project risk register were mapped to the project schedule
• The aggregate Expected Value of these risks is not a significant threat to the project outcome
• But the major threats affect operations of the asset rather than the construction of the tollway, as noted earlier: – Threat that a commuter rail line may be built alongside – Threat of congestion and slower travel requiring Extra
Lanes (ELP treatment of risk) • These threats and treatments are intelligently applied in
the IRRA model, as described earlier
33
RISK-2510
Register of Treated Risks mapped to model
34
RISK-2510
Modelling Results: Project Duration • The Model produced a pessimistic forecast of
schedule contingency due to: – Probabilistic weather calendar not included deterministically – Pessimistic duration risk factor from systemic risk modelling
35
Data
Duration of:
PC - Tollway Project
Analysis
Iterations: 1000
Statistics
Minimum: 1287
Maximum: 1631
Mean: 1457
Std Deviation: 67.03
Bar Width: week
Highlighters
Deterministic (1274) <1%
10% 1369
50% 1456
90% 1548
1300 1400 1500 1600
Distribution (start of interval)
0.0
5.0
10.0
15.0
20.0
25.0
30.0
35.0
40.0
45.0
50.0
55.0
Hits
0% 1287
5% 1351
10% 1369
15% 1384
20% 1395
25% 1405
30% 1415
35% 1424
40% 1439
45% 1449
50% 1456
55% 1465
60% 1474
65% 1484
70% 1494
75% 1504
80% 1517
85% 1529
90% 1548
95% 1569
100% 1631
Cum
ulat
ive
Freq
uenc
y
GTP Rev10.1 $180m Extra Lanes Rev, 20% Rail Threat, Systemic & Treated RisksPC - Tollway Project : Duration
RISK-2510
Modelling Results: Project Capex • The Model also produced a pessimistic forecast of cost contingency
due to similar drivers as for schedule distribution • Costs are negative in cost & revenue model & $0 must be shown in negative range:
36
Data
Cost of:
PZ$.CC - GTP Capital Costs
Analysis
Iterations: 1000
Statistics
Minimum: ($3,651,400,869)
Maximum: ($2,708,894,444)
Mean: ($3,188,103,811)
Std Deviation: 176,512,583
Bar Width: $50,000,000
Highlighters
Deterministic (($2,625,000,0... <1%
10% ($2,959,561,948)
50% ($3,184,982,818)
90% ($3,418,499,211)
($3,000,000,000) ($2,000,000,000) ($1,000,000,000) $0
Distribution (start of interval)
0
10
20
30
40
50
60
70
80
90
100
110
Hits
0% ($2,708,894,444)
5% ($2,907,714,603)
10% ($2,959,561,948)
15% ($3,002,705,674)
20% ($3,030,880,728)
25% ($3,060,361,973)
30% ($3,089,056,949)
35% ($3,111,496,334)
40% ($3,138,824,232)
45% ($3,158,844,612)
50% ($3,184,982,818)
55% ($3,210,787,139)
60% ($3,235,425,842)
65% ($3,262,924,914)
70% ($3,282,514,255)
75% ($3,314,533,159)
80% ($3,342,230,158)
85% ($3,375,796,552)
90% ($3,418,499,211)
95% ($3,483,879,723)
100% ($3,651,400,869)
Cum
ulat
ive
Freq
uenc
y
GTP Rev10.1 $180m Extra Lanes Rev, 20% Rail Threat, Systemic & Treated RisksPZ$.CC - GTP Capital Costs : Cost
RISK-2510
Modelling Results: Drivers of Project Cost • Drivers of project duration and cost distributions
have been determined by Quantitative Exclusion Analysis:
37
RISK-2510
IRRA Scenario Analyses - Inputs
• Two scenarios were analysed to demonstrate the power of the IRRA methodology to compare different asset operating environments and their profitability
• Scenario 1: Low threshold revenue to add extra lanes, low probability of Rail Link being built: – $180 million annual revenue threshold input to macro to
trigger earlier decisions to add extra lanes to the tollway – 20% probability placed on the likelihood that the
government would build an adjacent commuter rail line • Scenario 2: High threshold revenue to add extra lanes,
high probability of Rail Link being built: – $400 million revenue input to the macro to delay decisions
to add extra lanes to the tollway – 80% probability set of an adjacent commuter rail line
38
RISK-2510
IRRA Scenario 1 Analysis – Probabilistic NPV
• Probabilistic NPV for optimistic Scenario 1 shown below (probability of NPV >= $x-value shown, 7% Discount rate)
• ~63% chance of the investment being profitable (positive NPV)
39
RISK-2510
IRRA Scenario Analyses –Compare Profitability
• Probabilistic NPVs for the two scenarios are compared below (probability of NPV < $x-value shown):
40
RISK-2510
STRENGTHS AND WEAKNESSES OF THE IRA AND IRRA METHODOLOGIES
41
RISK-2510
Key Strengths of IRA / IRRA Methodologies
1. Integrates estimating, planning, risk management and economic analysis (IRRA) to optimise time and cost outcomes of the project through simultaneous analysis of time and cost outcomes of the project and the assets (IRRA) and through evaluating the drivers that produce those outcomes.
2. Can enable project risk to be optimised through the integrated ranking of all drivers of project cost and their progressive optimisation.
3. Can evaluate major sources of project failure through examination of the viability of the assets created by the project and the effects on them of risks and uncertainties to project revenue as well as operating conditions, regulations and costs (IRRA).
4. Can improve schedule quality through exposing defects, gaps in scope and misalignment with the estimate.
5. Can facilitate cross-discipline communications and improve project management through collaboration
6. Quantifies effects of risks on the project and the asset (IRRA), focusing the PM team on the need for effective risk management
42
RISK-2510
Weaknesses of IRA / IRRA Methodologies
1. Requires estimate and schedule – thus not useful for early stages of project development, unless schedules have been drafted.
2. Requires a relatively good quality schedule to obtain useful results. 3. Requires significant levels of skills and experience to produce good
quality results. 4. No accepted standards defining procedures, processes or features
and functionality of tools required; depends on educated clients. 5. The methodologies are not empirically based, although historical
data has been referenced to assign repeat activities and previous similar analogous project logic and estimates (excludes hybrid approach incorporating systemic risk described here)
6. Hybrid approach used here assumes all uncertainty incorporated in systemic risk, but part of value of IRA is identifying different areas and magnitudes of uncertainty to reveal hidden risky critical paths.
43
RISK-2510
HOW WELL DOES IRA METHODOLOGY ADDRESS RP 40R-08 CRITERIA FOR
QUANTIFYING PROJECT RISK?
44
RISK-2510
How well IRA addresses RP 40R-08 (1 of 2)
45
RP 40R-08 Principle How well IRA Methodology meets Principle Meets client objectives, expectations and requirements
Over 30 projects analysed; 6 clients multiple users for up to 6 years confirming methodology meets client expectations and requirements. Similar methodologies by others confirm approach.
Part of decision and risk management process
IRA similar to NASA’s JCL 70 Process, part of NASA’s project budget development process.
Fit-for-use Refined since 2008 to be fit-for-purpose; now adapted to use parametric estimation of systemic risks
Starts with identifying risk drivers
Always involves collection of schedule and cost uncertainty information and risk factors where applicable then risk events, as inputs to analysis.
RISK-2510
How well IRA addresses RP 40R-08 (2 of 2)
46
RP 40R-08 Principle How well IRA Methodology meets Principle Links risk drivers & cost/schedule outcomes
Maps risk events, risk factors and probabilistic weather calendars to specific tasks. Also uses QEA to measure impact on schedule and cost of major sources of uncertainty in model.
Avoids iatrogenic risks
Documented IRA processes are repeatable and auditable, minimising chances of self-inflicted risks.
Employs empiricism
IRA methodology now incorporates a Systemic Risk Parametric method (refined by Hollmann) in IRA as a hybrid method. Further work required on this.
Uses experience / competency
Requires significant experience and competency to perform well – indicating power of methodology.
Provides probabilistic estimating results
Cost and schedule simulation output directly and simultaneously, supported by QEA ranking of cost and schedule drivers.
RISK-2510
CONCLUSION
47
RISK-2510
Conclusion
• IRA with parametric assessment of systemic risk complies with the principles of RP 40R-08
• Further development required to retain valuable area/ phase-specific ranging of duration and costs uncertainties to reveal “hidden critical paths”, yet incorporate parametrically derived systemic risk; clients to be educated to accept systemic risk
• CPM-based MCS methodology can incorporate decision-making during iterations to represent realistic treatment of risks. This requires use of macros, which no currently available CPM-based MCS software able to perform ICSRA provides – a market gap
• The Tollway project and asset modelling demonstrates the ability of IRRA methodology to optimise project and asset investment proposals and improve their resilience against asset threats not modelled in contingency–focused analysis
• The methodologies require sophisticated skills and experience, desirable attributes wherever high risk decisions are to be taken
48
RISK-2510
QUESTIONS/COMMENTS? (PLEASE USE MICROPHONE)
49