socio-economic analyses in perspective: uncertainties and ... · pdf filesocio-economic...
TRANSCRIPT
![Page 1: Socio-economic analyses in perspective: Uncertainties and ... · PDF fileSocio-economic analyses in perspective: Uncertainties and bias in decision support ... ”Chunnel” in 1987](https://reader034.vdocuments.us/reader034/viewer/2022052515/5a782d857f8b9aa3618e898b/html5/thumbnails/1.jpg)
Socio-economic analyses in perspective: Uncertainties and bias in decision support
Associate Professor, PhD Kim Bang Salling DTU Transport Traffic days in Aalborg 2012 – Special session: “Uncertainties in Transport Project Evaluation (UNITE)”
![Page 2: Socio-economic analyses in perspective: Uncertainties and ... · PDF fileSocio-economic analyses in perspective: Uncertainties and bias in decision support ... ”Chunnel” in 1987](https://reader034.vdocuments.us/reader034/viewer/2022052515/5a782d857f8b9aa3618e898b/html5/thumbnails/2.jpg)
DTU Transport, Technical University of Denmark 2
Project Plan of UNITE
Uncertainties in Transport Project Evaluation (UNITE): the five Work-Packages
(5) Evaluation methodologyWP5 project leader: Steen Leleur (DMG)
(4) Uncertainty calculation in transport modelsWP4 project leader: Otto Anker Nielsen (TMG)
(2) Organizational context of Modelling, an empirical study
WP2 project leader: Petter Næss (AAU)
(3) Uncertainty calculation of cost estimates
WP3 project leader: Bo Friis Nielsen (DTU Informatics)
(1) Systematic biases in transport models (recognized ignorance), an empirical studyWP1 project leader: Petter Næss (AAU)
![Page 3: Socio-economic analyses in perspective: Uncertainties and ... · PDF fileSocio-economic analyses in perspective: Uncertainties and bias in decision support ... ”Chunnel” in 1987](https://reader034.vdocuments.us/reader034/viewer/2022052515/5a782d857f8b9aa3618e898b/html5/thumbnails/3.jpg)
DTU Transport, Technical University of Denmark 3
How do we evaluate transport projects?
• Various existing guideline report: –Denmark, Sweden, UK, European Union, ....
• Socio-economic analysis by the use of Cost-Benefit
Analysis (CBA)
• Produces single point estimates such as Net Present Values (NPV), Benefit Cost Ratios (BCR), etc
• However, no common rule have been set in order to acommodate the uncertainties in CBA!
–Recent conducted PhD dissertation proved this point
![Page 4: Socio-economic analyses in perspective: Uncertainties and ... · PDF fileSocio-economic analyses in perspective: Uncertainties and bias in decision support ... ”Chunnel” in 1987](https://reader034.vdocuments.us/reader034/viewer/2022052515/5a782d857f8b9aa3618e898b/html5/thumbnails/4.jpg)
DTU Transport, Technical University of Denmark 4
Background & Motivation
• The Manual for socio-economic analysis in the transport sector (2003)
–Unique guidelines for evaluating transport infrastructure projects
–Lack of uncertainty handling –Expected revision 2012-2013
![Page 5: Socio-economic analyses in perspective: Uncertainties and ... · PDF fileSocio-economic analyses in perspective: Uncertainties and bias in decision support ... ”Chunnel” in 1987](https://reader034.vdocuments.us/reader034/viewer/2022052515/5a782d857f8b9aa3618e898b/html5/thumbnails/5.jpg)
DTU Transport, Technical University of Denmark 5
How do we evaluate transport projects?
• However, no common rule have been set in order to acommodate the uncertainties in CBA!
–Recent conducted PhD dissertation proved this point
![Page 6: Socio-economic analyses in perspective: Uncertainties and ... · PDF fileSocio-economic analyses in perspective: Uncertainties and bias in decision support ... ”Chunnel” in 1987](https://reader034.vdocuments.us/reader034/viewer/2022052515/5a782d857f8b9aa3618e898b/html5/thumbnails/6.jpg)
DTU Transport, Technical University of Denmark 6
The Case Study: HH-Connection • Connecting Denmark with Sweden: Scandinavian link
–Currently, close to the capacity limit on Oresund
HH-Connection (alternatives*)
Description (Alignment of connection)
Cost (million DKK)
Alternative 1 Tunnel for rail (2 tracks) person traffic only 7,700
Alternative 2 Tunnel for rail (1 track) goods traffic only 5,500
Alternative 3 Bridge for road and rail (2x2 lanes & 2 tracks) 11,500
Alternative 4 Bridge for road (2x2 lanes) 6,000
* Larsen, L.A. & Skougaard, B.Z. (2010). Vurdering af alternativer for en fast forbindelse Helsingør-Helsingborg, M.Sc. thesis, Department of Transport, Technical University of Denmark (in Danish)
![Page 7: Socio-economic analyses in perspective: Uncertainties and ... · PDF fileSocio-economic analyses in perspective: Uncertainties and bias in decision support ... ”Chunnel” in 1987](https://reader034.vdocuments.us/reader034/viewer/2022052515/5a782d857f8b9aa3618e898b/html5/thumbnails/7.jpg)
DTU Transport, Technical University of Denmark 7
The UNITE-DSS Modelling Framework Todays Outline
![Page 8: Socio-economic analyses in perspective: Uncertainties and ... · PDF fileSocio-economic analyses in perspective: Uncertainties and bias in decision support ... ”Chunnel” in 1987](https://reader034.vdocuments.us/reader034/viewer/2022052515/5a782d857f8b9aa3618e898b/html5/thumbnails/8.jpg)
DTU Transport, Technical University of Denmark 8
Results: Cost-Benefit Analysis
• Construction costs – by far the largest contributor of costs
• User Benefits – by far the largest contributor of benefits – Consists of Ticket revenue and time savings – Relies on the prognosis of future number of passengers i.e.
demand forecasts
HH-Connection (alternatives)
Cost (million DKK)
BCR NPV (million DKK)
Alternative 1 7,700 1.50 5,530
Alternative 2 5,500 0.16 -6,640
Alternative 3 11,500 2.71 28,240
Alternative 4 6,000 3.08 17,860
![Page 9: Socio-economic analyses in perspective: Uncertainties and ... · PDF fileSocio-economic analyses in perspective: Uncertainties and bias in decision support ... ”Chunnel” in 1987](https://reader034.vdocuments.us/reader034/viewer/2022052515/5a782d857f8b9aa3618e898b/html5/thumbnails/9.jpg)
DTU Transport, Technical University of Denmark 9
Are we telling the truth?!?!
Construction cost overruns
0%
200%
400%
600%
800%
1000%
1200%
1400%
1600%
1800%
2000%
Suez
Can
al
Sydn
eyO
pera
Hou
se
Con
cord
eSu
pers
onic
Aero
plan
eBo
ston
'sAr
tery
/Tun
nel
Proj
ect,
USA
Hum
ber
Brid
ge, U
K
Bost
on-
Was
hing
ton-
New
Yor
kG
reat
Bel
tR
ail T
unne
l,D
KA6
Mot
orw
ayC
hape
l-en-
le-
Frith
/Wha
ley
Shin
kans
enJo
etsu
Rai
llin
e, J
apan
Was
hing
ton
met
ro, U
SA
Cha
nnel
Tunn
el, U
K &
Fran
ceKa
rlsru
he-
Bret
ten
light
rail,
Ger
man
yØ
resu
ndAc
cess
link
s,D
K &
Swed
enM
exic
o ci
tym
etro
line
,M
exic
oPa
ris-A
uber
-N
ante
rre
rail
line,
Fra
nce
Cos
t Ove
rrun
s (%
)
Q: Have we learned anything from history?
”Chunnel” in 1987 £2,600 million (’85 prices) Completion 1994 £4,650 million (’85 prices) Total cost overrun of approx. 80%
”Øresund access link” in 1991 3.2 billion DKK (’90 prices) Completion 1998 5.4 billion DKK (’90 prices) Total cost overrun of approx. 68%
![Page 10: Socio-economic analyses in perspective: Uncertainties and ... · PDF fileSocio-economic analyses in perspective: Uncertainties and bias in decision support ... ”Chunnel” in 1987](https://reader034.vdocuments.us/reader034/viewer/2022052515/5a782d857f8b9aa3618e898b/html5/thumbnails/10.jpg)
DTU Transport, Technical University of Denmark 10
Theoretical anchoring The Transport Planning Phase: Adapted from the British Department for Transport (DfT) (2004)
Reference Class Forecasting: Optimism Bias
Inside View Outside View
”Uniqueness” of Project
”The Planning Fallacy”
Reference Class Forecasting
Forecasting of particular projects
Forecasting from a group of projects
(1) Identification of relevant reference
classes
(2) Establishing probability distribution
(3) Placing and comparing the
project
Optimism Bias UpliftsCurrent Situation
![Page 11: Socio-economic analyses in perspective: Uncertainties and ... · PDF fileSocio-economic analyses in perspective: Uncertainties and bias in decision support ... ”Chunnel” in 1987](https://reader034.vdocuments.us/reader034/viewer/2022052515/5a782d857f8b9aa3618e898b/html5/thumbnails/11.jpg)
DTU Transport, Technical University of Denmark 11
Optimism Bias and uplifts
• Deriving uplifts is highly dependet on large data-sets –Flyvbjerg from (AAU) has since 2003 developed a large
database –Unfortunately, it looks upon mega-projects
• The basis is Reference Class Forecasting i.e. statistical measurements on various project pools
Source: Flyvbjerg and COWI (2004)
![Page 12: Socio-economic analyses in perspective: Uncertainties and ... · PDF fileSocio-economic analyses in perspective: Uncertainties and bias in decision support ... ”Chunnel” in 1987](https://reader034.vdocuments.us/reader034/viewer/2022052515/5a782d857f8b9aa3618e898b/html5/thumbnails/12.jpg)
DTU Transport, Technical University of Denmark 12
Results : Optimism Bias Uplifts
• The BCR are lower, however, still point estimates towards DM –Moreover an advanced form of sensitivity analysis
• Imply to introduce risk analysis and Monte Carlo simulation
HH-Connection (alternatives)
Cost (uplifted) (million DKK)
BCR (orig.) (from slide 8)
BCR (uplifts): 80% uplift
Alternative 1 12,090 1.50 0.97
Alternative 2 8,640 0.16 0.10
Alternative 3 15,180 2.71 1.75
Alternative 4 7,920 3.08 1.98
![Page 13: Socio-economic analyses in perspective: Uncertainties and ... · PDF fileSocio-economic analyses in perspective: Uncertainties and bias in decision support ... ”Chunnel” in 1987](https://reader034.vdocuments.us/reader034/viewer/2022052515/5a782d857f8b9aa3618e898b/html5/thumbnails/13.jpg)
DTU Transport, Technical University of Denmark 13
The UNITE Project Database (UPD)
• The convention used is as follows: ( )( )
forecasted
forecastedactual
XXX
U100×−
=
Over estimation of Demand
![Page 14: Socio-economic analyses in perspective: Uncertainties and ... · PDF fileSocio-economic analyses in perspective: Uncertainties and bias in decision support ... ”Chunnel” in 1987](https://reader034.vdocuments.us/reader034/viewer/2022052515/5a782d857f8b9aa3618e898b/html5/thumbnails/14.jpg)
DTU Transport, Technical University of Denmark 14
• Demand forecasts (user benefits) are derived: – U is percent inaccuracy, – Xa is the actual traffic after the project is opened – Xf is the forecasted traffic on the decision to build
• Combination of two database samples
0
5
10
15
20
25
30
(-12
0;-1
00)
(-10
0;-8
0)
(-80
;-60)
(-60
;-40)
(-40
;-20)
(-20
;0)
(0;2
0)
(20;
40)
(40;
60)
(60;
80)
(80;
100)
(100
;120
)
(120
;140
)
(140
;160
)
(160
;180
)
(180
;200
)
(200
;220
)
(220
;240
)
Freq
uenc
y of
occ
uren
ce (%
)
Inaccuracies in demand forecasts (%)
Inaccuracies in demand forecasts (road projects)
Salling et al. (2012)
Flyvbjerg et al. (2003)
Nicolaisen et al. (2012)
![Page 15: Socio-economic analyses in perspective: Uncertainties and ... · PDF fileSocio-economic analyses in perspective: Uncertainties and bias in decision support ... ”Chunnel” in 1987](https://reader034.vdocuments.us/reader034/viewer/2022052515/5a782d857f8b9aa3618e898b/html5/thumbnails/15.jpg)
DTU Transport, Technical University of Denmark 15
The UNITE Project Database (UPD)
• The convention used is as follows: ( )( )
forecasted
forecastedactual
XXX
U100×−
=
Under estimation of costs
![Page 16: Socio-economic analyses in perspective: Uncertainties and ... · PDF fileSocio-economic analyses in perspective: Uncertainties and bias in decision support ... ”Chunnel” in 1987](https://reader034.vdocuments.us/reader034/viewer/2022052515/5a782d857f8b9aa3618e898b/html5/thumbnails/16.jpg)
DTU Transport, Technical University of Denmark 16
• Construction costs bias derived similarly: – U is percent inaccuracy, – Xa is the actual traffic after the project is opened – Xf is the forecasted traffic on the decision to build
• Combination of two database samples
0
10
20
30
40
50
(-10
0;-8
0)
(-80
;-60)
(-60
;-40)
(-40
;-20)
(-20
;0)
(0;2
0)
(20;
40)
(40;
60)
(60;
80)
(80;
100)
(100
;120
)
(120
;140
)
(140
;160
)
(160
;180
)
(180
;200
)
(200
;220
)
(220
;240
)
Freq
uenc
y of
occ
uren
ce (%
)
Inaccuracies in construction costs (%)
Inaccuracies in construction cost (road projects)
Salling et al (2012)
Flyvbjerg et al. (2003)
Nicolaisen et al. (2012)
![Page 17: Socio-economic analyses in perspective: Uncertainties and ... · PDF fileSocio-economic analyses in perspective: Uncertainties and bias in decision support ... ”Chunnel” in 1987](https://reader034.vdocuments.us/reader034/viewer/2022052515/5a782d857f8b9aa3618e898b/html5/thumbnails/17.jpg)
DTU Transport, Technical University of Denmark 17
Results (RCF): Monte Carlo simulation
![Page 18: Socio-economic analyses in perspective: Uncertainties and ... · PDF fileSocio-economic analyses in perspective: Uncertainties and bias in decision support ... ”Chunnel” in 1987](https://reader034.vdocuments.us/reader034/viewer/2022052515/5a782d857f8b9aa3618e898b/html5/thumbnails/18.jpg)
DTU Transport, Technical University of Denmark 18
Conclusions
• Feasibility risk assessment can be carried out by using historical experience stemming from RCF in order to obtain interval results
• An important aspect in RCF and UNITE is to set and validate input parameters. Hence, empirical data enter the assessment.
• The RCF approach has been illustrated on a case example concerning the construction of a new fixed link, the HH-Connection, between Denmark and Sweden.
• Clearly vital to include uncertainties within socio-economic analyses in order to validate results
![Page 19: Socio-economic analyses in perspective: Uncertainties and ... · PDF fileSocio-economic analyses in perspective: Uncertainties and bias in decision support ... ”Chunnel” in 1987](https://reader034.vdocuments.us/reader034/viewer/2022052515/5a782d857f8b9aa3618e898b/html5/thumbnails/19.jpg)
DTU Transport, Technical University of Denmark 19
Perspectives
• Recovering of further data (UPD) with regard to both the demand forecast uncertainty as well as the construction costs through large-scale research study
• Producing so-called decision conferences in order to achieve better input parameters to the UNITE-DSS Model combined with overconfidence theory allows for expert opinions (SIMSIGHT)
• More info on UNITE can be found: (www.transport.dtu.dk/unite)
• An international conference on the topic is scheduled in September 2013 – a specific call will be posted in the upcoming month.
![Page 20: Socio-economic analyses in perspective: Uncertainties and ... · PDF fileSocio-economic analyses in perspective: Uncertainties and bias in decision support ... ”Chunnel” in 1987](https://reader034.vdocuments.us/reader034/viewer/2022052515/5a782d857f8b9aa3618e898b/html5/thumbnails/20.jpg)
DTU Transport, Technical University of Denmark
SIMSIGHT: Decision Conferencing (DC)
20
• Producing so-called decision conferences in order to achieve better input parameters to the UNITE-DSS Model
• Enables to include Stakeholders and Decision-makers in an early stage, i.e. to include experts opinion on MIN and MAX values as entries to the Monte Carlo simulation
![Page 21: Socio-economic analyses in perspective: Uncertainties and ... · PDF fileSocio-economic analyses in perspective: Uncertainties and bias in decision support ... ”Chunnel” in 1987](https://reader034.vdocuments.us/reader034/viewer/2022052515/5a782d857f8b9aa3618e898b/html5/thumbnails/21.jpg)
DTU Transport, Technical University of Denmark 21
Results from DC and RSF
![Page 22: Socio-economic analyses in perspective: Uncertainties and ... · PDF fileSocio-economic analyses in perspective: Uncertainties and bias in decision support ... ”Chunnel” in 1987](https://reader034.vdocuments.us/reader034/viewer/2022052515/5a782d857f8b9aa3618e898b/html5/thumbnails/22.jpg)
DTU Transport, Technical University of Denmark
SIMSIGHT: Overconfidence
22
![Page 23: Socio-economic analyses in perspective: Uncertainties and ... · PDF fileSocio-economic analyses in perspective: Uncertainties and bias in decision support ... ”Chunnel” in 1987](https://reader034.vdocuments.us/reader034/viewer/2022052515/5a782d857f8b9aa3618e898b/html5/thumbnails/23.jpg)
DTU Transport, Technical University of Denmark 23
Perspectives
• Recovering of further data (UPD) with regard to both the demand forecast uncertainty as well as the construction costs through large-scale research study
• Producing so-called decision conferences in order to achieve better input parameters to the UNITE-DSS Model combined with overconfidence theory allows for expert opinions (SIMSIGHT)
• More info on UNITE can be found: (www.transport.dtu.dk/unite)
• An international conference on the topic is scheduled in September 2013 – a specific call will be posted in the upcoming month.
![Page 24: Socio-economic analyses in perspective: Uncertainties and ... · PDF fileSocio-economic analyses in perspective: Uncertainties and bias in decision support ... ”Chunnel” in 1987](https://reader034.vdocuments.us/reader034/viewer/2022052515/5a782d857f8b9aa3618e898b/html5/thumbnails/24.jpg)
DTU Transport, Technical University of Denmark 24
Thank you for your attention!
Affiliation:
Associate Professor, PhD Kim Bang Salling
Department of Transport Technical University of Denmark