transport appraisal in other countries: lessons for the nata refresh

8/8/2019 Transport Appraisal in other countries: lessons for the NATA Refresh

1/26

Transport Appraisal in othercountries: lessons for the NATARefreshReport - October 2007

A Report to the Department for Transport

Institute for Transport StudiesFACULTY OF ENVIRONMENT


2/26

Transport Appraisal in other countries: lessons for the NATA Refresh

Report - October 2007

2


Version 1.1


Peter MackieCharlotte Kelly

Institute for Transport Studies

University of Leeds

Leeds

LS2 9JT

Tel +44 (0) 113 3435336Fax: + 44 (0) 113 3435334


3/26



3

CONTENTS

1 Introduction ................................................................................................................................... 42 Assess whether the key HEATCO findings are correctly summarised.......................................... 5

2.1 Valuation of local and regional air pollution............................................................................ 82.2 Development of the rail appraisal guidance and the standard values for rail franchiseoperating costs ................................................................................................................................. 82.3 Extension of the NATA framework to cover aviation and maritime appraisal ......................... 92.4 Use of monte Carlo techniques and sensitivity testing in assessing optimism bias ............... 92.5 Value of time Meta Analysis.............................................................................................. 10

3 What did the HEATCO programme learn about how transport appraisal results are used toinform decision making in different countries ..................................................................................... 10

4 Danish (2002) and (2007) study results ...................................................................................... 105 Recent Studies of transport appraisal practice in other countries ............................................... 135.1 PIARC..................................................................................................................................135.2 CfIT High Speed rail: International comparisons............................................................... 145.3 OEI (NL)............................................................................................................................... 155.4 RAILPAG ............................................................................................................................. 16

6 Given the scope of the NATA refresh what lessons can be learned from all of these studies thatcan be incorporated into the NATA Consultation document?............................................................. 187 References.................................................................................................................................. 19


4/26



4

1 Introduction

The Institute for Transport Studies was commissioned by the Department for Transport to review theevidence base on transport appraisal in other countries to inform the NATA refresh process that iscurrently taking place. We were asked to:

Assess whether the key HEATCO findings are correctly summarised

Consider what the HEATCO programme learned about how transport appraisal resultsinform decision making in other countries

Look at how the conclusions from COWI (2002) and Danish Transport Research Institute(2007) compare with the findings of HEATCO

Assess whether there are any other recent studies that exist of transport appraisal practicein other countries and what are their conclusions?

Consider what lessons can be learned from these studies that can be incorporated into theNATA consultation document.

This report will look at each of these in turn.


5/26



5

2 Assess whether the key HEATCO findings are correctly summarised

The HEATCO project primary objective was the development of harmonised guidelines for projectassessment at an EU level. As part of this process the project undertook a survey of current practicein project appraisal in Europe detailed in Odgaard et al (2005) and HEATCO (2005a). A summary ofthis information is provided in table 1 and considers all 25 EU countries. This was an extension ofthe work originally completed by Grant Muller et al (2001). It should be noted that since the time thatthis has been produced appraisal practice has changed in some of the countries, for example, in theUK with Noise values (TAG 3.3.2) and carbon values (TAG 3.3.5) both introduced in 2006. Thissurvey was followed by HEATCO (2005b), which considered the state of the art of in projectassessment for the key elements of appraisal and considered in detail the key differences between

how the countries in the EU conduct their appraisal practice. One of the contexts for the HEATCOreport was the appraisal of international projects and other projects with transboundary effects,including the TEN projects.

Annex 1 is the section of the DfT note which describes the key differences in transport appraisalpractice between EU countries. This is an accurate reflection of the work that was conducted on theinitial surveys that looked at the 25 individual countries (presented in HEATCO, 2005a). However DfTdoes not appear to have reviewed the later HEATCO deliverables. We strongly recommend that amember of the NATA-refresh team reads HEATCO D5 (HEATCO 2006b) and Annex A to thatDeliverable (Shires and de Jong, 2006) since there are many tables of results which can help tobenchmark the UK against other EU member states. All the HEATCO deliverables can be

downloaded from http://heatco.ier.uni-stuttgart.de/

The findings summarised in the DfT note (taken from HEATCO) were:

there are significant regional differences in the approach to, and tradition for using, transportproject appraisals;

the appraisal frameworks used for roads are far more developed than those for rail, air, inlandwaterways and sea transport;

there is a lack of consensus about which elements to include within appraisals, especiallyenvironmental effects;

there is a lack of consensus on approaches to valuing impacts; and

there is a significant range of values used, especially for safety.

We believe for the majority of countries the list above is still a good general summary. Point 3 couldbe qualified with the statement that certain direct impacts such as construction and maintenancecosts and travel time and vehicle operating costs are routinely included.


6/26



6

Table 1 Coverage by country and element in case CBA is used1,2

Constructioncosts

Disruptionfromconstruction

Costsformaintenance,

operationandadministration

Passengertransporttime

savings

Userchargesandrevenues

Vehicleoperatingcosts

Benefitstogoodstraffic

Safety

Noise

Airpollution-local/regional

ClimateChange

Indirectsocio-economic

effects

Austria (*)

Belgium (*+)

Denmark (*)Finland (*)

France (*)

Germany (*)

Ireland (*)

Netherlands (*)

Sweden (*)

Switzerland (*)

UK (*)

Czech Republic (*)

Estonia (+)

Hungary (*)

Latvia (+)

Lithuania (*)Poland (+)

Slovak Republic (*)

Slovenia (*)

Cyprus (+)

Greece (+)

Italy (+)

Malta (*)

Portugal (+)

Spain (*)

Colour codes:

: Included with a money value

: Measured quantitatively, qualitatively or not included

North/West

E

ast

South

source: HEATCO(2005a)

The full HEATCO recommendations are provided in Annex 2. Table 2 presents a comparisonbetween the UK appraisal conventions and the HEATCO guidelines.

1 6 It appears that all countries include some effects with a money value for inclusion in CBA. Caution should be applied when interpreting thisresult, as CBA is not a requirement in all countries and as such is only used under some special circumstances. Furthermore, it should be noted that

the figure provides a summary for all modes, i.e. it might be that some effects are not included for some modes. (*) - reflects recommended/requiredapproach; (+) - reflects typical approach when CBA is used.

2 Since the production of this table some appraisal practice in countries have been updated.


7/26



7

Table 2 Overview of general assessment approaches (UK and HEATCO)

HEATCO UKAppraisal Period Planning and construction + 40 years 60 yearsDiscount Rate National Values of risk premium free

rate and sensitivity testing at 3%3.5% for the first 30 years, 3% for theremaining 30 years

Unit of Account Factor costs Market pricesCurrency Euro GBPMarginal costsof public funds

MCPF= 1 and use a cut off value ofRNPSS = 1.5

MCPF = 1 (all UK) (RNPSS = 1.5 inEngland only)

ScenarioDefinition

Reference scenario as do minimum Reference scenario as do minimum

Decision criteria NPV, BCR, RNPSS (Ratio of NPV andPublic Sector Support)

NPV, BCR

Treatment offuture risk anduncertainty

Sensitivity analysis at a minimum,Monte Carlos simulations as a moresophisticated approach

UK provides less prescriptive adviceand just recommends undertaking a riskanalysis

Equity issues(inter-generational)

Winners and losers tables at aminimum, distributional matrices as amore sophisticated approach

Disaggregation of impacts betweenstakeholders categories and mode toidentify winners and losers

Treatment ofindirect socio-economic

effects

Qualitative assessment at a minimum.Use of spatially computable generalequilibrium models where possible

Framework approach to appraisalbased around the 5 core objectives:environment, safety, economy,

integration, accessibility,Price base 2002, constant prices + PPP adjusted

prices2002

Treatment ofvalues over time

Adjustment on the basis of nationalGDP growth rates (elasticity = 1) exceptfor global warming and VTTS (0.7)

GDP growth ratesThe value of non-working time isassumed to increase with income, withan elasticity of 0.8. Working values oftime are assumed to grow in line withincome, with an elasticity of 1.

Turning to the individual elements of benefit, these are not easy to summarise succinctly, but among

the findings of HEATCO D5 were: at purchasing power parity, UK employers business values of time were around 10% above

the European average, UK non-work time 10% below and UK freight values close to average

safety values were typical for N and W Europe; some countries such as Norway, Denmarkand Switzerland are higher. The rest of Europe is much lower even at PPP. The relativitiesbetween fatal, serious and slight are typical.

D5 gives a recommended approach and values for NOx, NMVOCs,SOx,PM2.5s,and noise

D5 gives a recommended approach and values for carbon emissions but this predates theStern Report and is not recommended for use in the UK.

The DfT note identified the following areas from the HEATCO review and General Recommendationsfor areas where the NATA framework might be developed. These are:


8/26



8

1. The valuation of local and regional air pollution

2. The development of the rail appraisal guidance and the standard values for rail franchiseoperating costs

3. Extension of the NATA framework to cover aviation and maritime appraisal4. Use of Monte Carlo techniques in assessing optimism bias.

In addition to these areas we would also suggest considering the results from the value of timesavings meta analysis results that were produced as a result of the project (see annex 3) andsummarised below in section 2.5. HEATCO (2005b) reviewed the state of the art in projectassessment and HEATCO (2006) produced a proposal on the harmonisation of the appraisalpractices in practice and these reports will be used in the next section to summarise the findings fromHEATCO for the areas identified by the DfT note.

2.1 Valuation of local and regional air pollution

HEATCO (2005a) found that the majority of EU countries have values for PM, NOx, SO2, HC and COfor local/ regional pollution. The majority of these countries use the Impact Pathway approach tobase their monetary values. Other approaches include damage cost and avoidance costapproaches. The HEATCO recommendation for the valuation of local/ regional pollution was(HEATCO, 2006b):

To use country specific values taking into account local population density and regional climate. The

list of pollutants that should be covered are: Primary PM2.5 for transport emissions

NOXas precursor of nitrate aerosols and ozone

SO2 for direct effects and as precursor of sulphate aerosols

NMVOC as precursor of ozone

The HEATCO report provides cost factors for the UK for road transport and the methodology to useto calculate the monetary values.

HEATCO identified that in appraisal the main focus is on monetising impacts for noise, air pollutionand global warming. However other environmental impacts such as vibration, severance, visual

intrusion, loss of important sites, resource consumption, impairment of landscape, soil and waterpollution are included but rarely covered by assigning a monetary value. Countries such as theNetherlands are looking into values for some of these aspects, but have found that the resultingmonetary values can be very site specific (see section 5.3). The study by PIARC (2004) (see section5.1) also found that some countries are placing values on these other impacts.

2.2 Development of the rail appraisal guidance and the standard values for railfranchise operating costs

The HEATCO work did not look at standard values for rail franchise costs.


9/26



9

The HEATCO work did look at the elements that are included in rail appraisal in the 25 EU countries.

They identified that the appraisal framework for rail was less standardised than for roads. Thedevelopment of rail appraisal guidance is something that has been recently considered in RAILPAG (EU and EIB, 2005) Railway Project Appraisal Guidelines (see section 5.4) in more detail.

2.3 Extension of the NATA framework to cover aviation and maritime appraisal

The HEATCO survey found that several countries have entirely separate frameworks for analysingdifferent modes and only around one third of countries had formulated standard frameworks for theappraisal of air, inland waterway and sea transport projects. In the Nordic countries generalappraisal guidelines have been developed that are applied to all sectors (see section 4).

2.4 Use of Monte Carlo techniques and sensitivity testing in assessingoptimism bias

HEATCO (2005b) analysed the requirements in other countries for sensitivity analysis. This foundthat in most cases a sensitivity test is required on traffic growth and the value of time. In some casesa sensitivity test is also required for the discount rate, real GDP growth and investment and safetycosts. NATA currently has the following guidelines for sensitivity testing (TAG 3.9.2):

the level of patronage at which the net benefits of the scheme are zero. the level of cost overruns at which the net benefits of the scheme are zero; higher operating costs; the level of time savings (where appropriate) at which the net benefits of the scheme are

zero; and the responses of incumbent operators, such as fare and frequency responses of existing

bus operator responses to new light rail systems or continuing service of on-street stopsdespite the provision of new interchange facilities.

While the UK approach is in line with the majority of countries HEATCO made somerecommendations that take this further. The recommendations was that sensitivity analysis shouldbe the minimum used and Monte Carlo simulations used as a more sophisticated approach. Withregards to the specific impacts the following recommendations were made:

Sensitivity testing for the discount rate at 3%

For the National VTTS values it is recommended that an appraisal should be tested using+/-20% of the national VTTS values. This is different to the recommendations in TAG 3.9.2whereby it suggests testing the level of time savings (where appropriate) at which the netbenefits of the scheme are zero.

For the safety values the recommended sensitivity tests are v/3 and v*3, where v = value ofsafety per se.


10/26



10

2.5 Value of time Meta Analysis

One part of the HEATCO work was to produce an international meta analysis on values of travel timesavings (VTTS). The results are presented in Shires and de Jong (2006)see Annex 3. This workreached a number of conclusions. Which are summarised as:

A (largely cross-sectional) income elasticity of the VTTS of about 0.5 for business travel,0.7 for other passenger transport and 0.3 for freight.

Long distances are associated with higher VTTS for commuting and other purposes.Thecurrent M6 toll study is expected to shed further light on this.

SP and SP-RP studies give somewhat lower VTTS for employers business trips than thecost savings approach.

Significant effects for purpose (business, commuting) and mode (especially for air; less sofor bus relative to car and train).

Higher values of time in Southern and Eastern European countries, all other things(including GDP/capita) being equal.

3 What did the HEATCO programme learn about how transport appraisalresults are used to inform decision making in different countries

While HEATCO did not focus on how transport appraisal practice specifically informed decision-making the PIARC report (section 5.1) asked the countries whether the evaluation framework wasaccepted and how the results of the evaluation were used and the CfiT study (section 5.2) alsoconsidered this issue. The road directorate in Norway analysed the extent that CBA results affectedthe prioritisation of projects in 15 countries and found that in 5 out of 15 countries it was significant inexplaining the ranking (Danish Transport Research Institute, 2007:18)

Previous work for the EU in projects such as TRANSTALK has addressed this issue (Giorgi, 2002,Pearman, Mackie and Nellthorp,2003).Our opinion is that the countries with the best developed CBAtradition and evidence base are the Netherlands, Sweden and the UK. These countries also have

somewhat similar cultures regarding the role of CBA in decision support. There is therefore most tobe gained from a close comparison of decision support practices and the role of appraisal in thosecountries.

4 Danish (2002) and (2007) study results

Two reports have recently been published by the Danish Transport Research Institute consideringthe use of socio-economic assessments as the basis for decision-making in transport. The 2002report considered the appraisal practice in 7 countries (Finland, France, Holland, Norway, UK,Sweden and Germany). It then examined in more detail four of these countries (France, Norway, UKand Sweden). The 2007 report updated the previous report for the four Nordic Countries (Sweden,

Norway, Finland and Denmark). Additionally the second report considered how the four countries


11/26



11

appraisal practice compared to the HEATCO recommendations3. Table 3 provides a summary of

some of the components used in economic evaluation in the countries reviewed and compares it tothe HEATCO recommendation.

Table 3: Summary of DANISH 2002 and 2007 reportsHEATCO DNK

(2007)SWE(2007)

NOR(2007)

FIN(2007)

DEU(2002)

NLD(2002)

FRA(2002)

Discount Rate(real)

NationalValues ofriskpremium free rateand

sensitivitytesting at3%

6% 4% 4.5% (riskfree 2%,riskpremium2.5%)

5% 3% 4% 8%(5%)

Unit of Account factor market market market factor factor factor marketMarginal costsof public funds

- 20% 30% 20% - - - -

Decisioncriteria

NPV, BCR,RNPSS

NPV,BCR,IRR

NPV,BCR

NPV,BCR,FYB,IRR

BCR,NBCR

CBR NPV NPV,CBA,FYB,IRR

Local Airpollution Emissions

included

NoxSo2NMVOC

PM2.5/PM10

ParticlesNOXSO2

HCVOCs

ParticlesNOXSO2

HCVOCs

ParticlesNOX

ParticlesNOXSO2

COHC

ParticlesNOXSO2

COHCCarcin-ogens

ParticlesNOXSO2

COHC

PM10(onlyemission

withmonetaryvalue)

Valuationmethod

DC DC DC SP, DC DC - - DC

Global airpollution(valued)

yes yes yes yes yes yes yes yes

Valuationmethod

DC &AC AC AC AC DC AC AC Tax

Differentiationof passenger

VoT

Purpose -Passenger

work andnon worktrips

Purpose,type of

time,

Mode oftransport

Purpose.Type oftime,distance

Mode,purpose,

type of time,distance

Mode,purpose,

type oftime,

Purpose Mode,purpose

Mode,purpose,

type oftime,populationgroups,comfort,distance

Method WR/ CSAfor worktrips

WR, SP SP, WR SP, WR WR SP SP RP, SP,IC, WR

Last study 2006 1995 1997 1999/2000 1991 1998

DC = Damage cost, SP = Stated Preference, AC = avoidance costs, WR = wage rate, CSA = Costsaving approach, IC = International Comparisons

3 The 2007 report will be used as the basis of comparison for the Nordic countries the 2002 report will be used for the other countries


12/26



12

All of the countries use a socio-economic method to some extent in the decision making process.For the Nordic countries this very often provides an input into the national transport plans. The reportpaints a similar picture to the results in the HEATCO study, which also found that the road sector isthe furthest developed in terms of socio economic methodology.

The 2007 review identified a number of differences in the way that dealing with uncertainty wascovered. In Finland sensitivity analysis is conducted on the central parameters, but there is currentlyno formal requirement for doing so. In Sweden uncertainty analysis is dealt with on an ad hoc basisfor investment costs, traffic levels and the discount rate. This is different to Norway where it is aminimum requirement to conduct an uncertainty assessment for investment costs and traffic levels.4

The reviews also looked in more detail at how non-monetised impacts were dealt with in theappraisal. Currently there are no formal methods for dealing with these impacts in Denmark andFinland. Sweden has recently been working on the development of an impact profile. In Norway theexcluded non monetised effects are split into the following themes; local environment, recreationallife, natural environment, cultural relics and cultural environment, the landscape image, agricultureand fisheries, geographic and water resources, transport quality, local development patterns,accessibility for cyclists. Each of these themes are assessed firstly by value and then by extent, thenthese are combined to generate a society value. The results are then presented alongside the CBA.From the 2002 review it was identified that with the exception of the UK for the other countriessystematic methods for taking account of the non included effects did not appear to have beendeveloped (2002, p41). It was reported in (2007) that it was felt that impacts that are not monetisedand included in the CBA do not get sufficient attention in the decision making process. The Nordiccountries were all working to improve this element of their evaluation. In presenting the results of theappraisal the Norway and Swedish versions are very similar to the AST in the UK.

The review found differences between the countries in how the socio- economic evaluations areapplied in the different countries. For example in Sweden the socio economic evaluation is used toconduct an analysis of alternative potential plans for the next 10 year national transport plan (which isupdated every 4 years). This is used to provide evidence for the first of the three national transportpolicy objectives, which are: welfare economics, traffic safety and the environment and regionaldevelopment, which is then given to the politicians to decide on the framework. At a National

decision making level the results of the CBA cannot be directly recognised in the decisions made, butthey are an important element of the evidence given to politicians for making a case. The Swedishguidelines dictate that sector authorities should conduct post- analyses for major projects in excessof SEK 50-100million 2-3 years after opening. The Norwegian process is very similar to Sweden.One of the problems that was identified in Norway was a difficulty in comparing projects acrosstransport sub-sectors. A working group was set up to investigate the extent to which themethodologies used by the different sectors (road, rail, sea and air) were consistent. This work hasapparently improved consistency in assumptions and economic principles.

All of the countries considered had evaluation guidelines and have produced models to aid in theevaluation. A particular consideration to note is that in Norway all sectors have to follow the general

4 there is no change in the descriptions for the Nordic countries between 2002 and 2007 for uncertainty


13/26



13

guidelines (Norwegian Ministry of Finance, 2005). However because these guidelines were too

general this has led to the rail and road sectors developing their own guidelines. The rail guidelineswere based on the road guidelines, so there is a high degree of consistency. In Finland generalguidelines have been developed that should be followed in all sectors. One problem is that becausethey are very general there is a degree of freedom over which people can use them. In Denmark theguidelines for conducting socio economic analysis set out the principles for the different sectorshave been developed in conjunction with a working group from the road, rail and Danish TransportResearch Institute.

5 Recent Studies of transport appraisal practice in other countries

5.1 PIARC

In 2001 PIARC conducted a survey to consider the methods of transport appraisal for roads used inits member countries (PIARC, 2004). The survey found that like the HEATCO survey a wide rangeof appraisal frameworks were used with almost all countries using CBA in some part of theevaluation. The countries that returned the questionnaires in 2001 were; Australia, Canada, CzechRepublic, Denmark, France, Germany, Hungary, Japan, Mexico, Netherlands, New Zealand, Norway,Portugal, Switzerland, Sweden, South Africa, UK and USA. A table was produced within this reportdetailing in 2001 what impacts were included/ not included and which were included as a monetisedimpact or non monetised impact. The general story is that almost all countries use CBA as acomponent of the evaluation often linked with MCA or an environmental or socio- economic analysis.All countries include road user time costs, Vehicle operating costs and most include accident costs in

their economic evaluations. Externalities are generally non monetised, with the exceptions of thoselisted in Table 4.

Table 4 Modified from PIARC(2004) Table 2.2 (2001 survey)

Impacts Countries that monetise the impacts

Comfort France, New Zealand, South AfricaAir pollution local Czech Republic, Denmark, France, Germany, Japan, New

Zealand, Norway, SwedenAir pollution regional Denmark, France, Germany, Japan, Norway, SwedenAir Pollution - Global Australia, Denmark, France, Germany, Japan, New Zealand,

Norway, Sweden

Dust Nuisance Germany, New Zealand, Norway, SwedenCommunity Severance Germany,Impact on employment Canada, GermanyMitigation/ clean up Germany, Norway

PIARC (2004) considered what methods were used to compare projects and found like the otherstudies that there is great variation among the countries. For example in Japan CBA is used foracceptance or rejection of the project. Prioritisation depends on an MCA allowing for social factorsand the importance of implementation (p67). In New Zealand projects are initially prioritised based

on the BCR and then Governments stated transport priorities are considered alongside the BCR andmay lead to a re ranking. Whereas in Portugal decisions on which project proceeds are based more


14/26



14

on the plan to complete the main and complementary road networks than on the economic

evaluation. (p69). In Sweden economic evaluation is the tool used for deciding on which projectsshould be included in the 10 year plan and at the project feasibility stage. While South Africa useseconomic evaluation of road projects to select project options, prioritise competing projects and forglobal road network strategies(p21).

PIARC also considered whether there was any acceptance for the evaluation framework used ineach country. They found that generally there was acceptance for economic evaluation. In NewZealand prior to 2002 there was an almost perfect correlation between the BCR and implementation.This has recently been modified with the inclusion of government transport priorities. In Swedenthere is a good correlation between implemented projects and projects that did well on the appraisal.They found that exceptions were mainly for political reasons. This is a similar story to Denmark and

Mexico. In the Netherlands, high priority from evaluation does not necessarily guaranteeimplementation : public consultation for road projects in the Netherlands often leads to disagreementof the weighting of impacts(p71).

The key recommendations from this report are that PIARC member countries should:

Continue to refine and develop CBA and MCA methodologies

Cooperate in developing monetary values for environmental and social impacts and methodsfor their inclusion in project evaluation

Expand the project evaluation methodologies to cover all land transport modes

Give more attention to use of quantitative risk analysis for projects with significant risks or

which are controversial

5.2 CfIT High Speed rail: International comparisons

The Commission for Integrated Transport (CfiT) commissioned Steer Davies Gleave to consider whatchanges could/ should be made to Bitains appraisal criteria and processes to better capture thecosts and benefits of high speed rail. In doing this CfIT(2004) conducted 7 case studies in thecountries of Britain, France, Italy, Spain, Germany, Japan and Australia to identify what elements ofthe appraisal had hindered/ helped in the goal for high speed rail investment. The key differencesfound between the appraisals include:

All countries with the exception of Japan consistently use CBA for rail projects. Only Spain, Germany and Britain formally use MCA.

Regional economic benefits were quantified in Spain and Germany

Estimates of the employment generated by a project are included at least in part in Spain,Germany and France.

Britain does not count the benefits of high speed rail to non residents within the CBA.Germany applies a multiplier to international projects producing the reverse effect.

Spain and France count additional tax revenue as a benefit.

European countries are less good at assessing the wider economic impact of major transportprojects only Japan undertook this analysis in detail.


15/26



15

In terms of how the appraisal results inform the decision this review identified that in Germany the

appraisal process appears to directly determine whether projects are included in the nationaltransport plan. In Spain economic appraisal is required by law, but only determines the prioritisationof projects (the when and how) not whether the project should be undertaken at all. In the case ofJapan the approach is a combination of financial and economic impact appraisal, but equallyimportant are issues of national prestige.

One of the conclusions of this review was that in many respects British Appraisal Frameworkrepresents best practice. However a number of recommendations were proposed that relatespecifically to current appraisal practice and high speed rail, which were:

Values of time vary significantly between region and type of journey. It is recommended to

use project specific values where projects are large and distinctive in their journey timebenefits

The allowance for risk and optimism bias in the new green book for capital intensive projects isexcessive as applied to high speed rail.

Britain should consider undertaking analysis of the wider national economic impact of majorprojects such as high speed rail. Evidence from Japan and the Netherlands is that when thisassessment is undertaken the analysis of high speed rail economic impacts indicates them tobe higher than revealed by narrow CBA alone.

5.3 OEI (NL)

In 2000 the OEI guide was published in the Netherlands. (CPB and NEI, 2000). This is the Dutchstandard appraisal guidance on CBA, which states that a CBA must be carried out for all largegovernmental infrastructure projects and can be considered similar to NATA and its supportingmaterials such as WebTAG. A comparison with neighbouring countries indicated that theNetherlands spent a lot of attention on determining the relevance of indirect impacts (p. xi). TheNetherlands conducted an evaluation in 2002 of its appraisal practices based on the originaldocument produced in 2000. They found that there was a lot of support for the Overzicht EffectenInfrastrcture (OEI) and that people felt that it made the information transparent and was factual.However there were a number of areas where it was felt that improvements could be made to theguidelines. From this evaluation a number of process and content recommendations were made to

improve this guidance and these were reported in Buck Consultants Ltd (2002). A summary of thekey recommendations for process changes were:

An agreement should be made on the scope of a CBA

Should determine at which point in the planning CBA should be applied

Should Stimulate market widening for the exporting and testing of CBA

Join up the OEI and the MER (environmental assessment) in the appraisalThe key content recommendations were:

Share the knowledge of the guide with all stakeholders that are involved

Broaden the guide for a number aspects:o Expand the methodology for indirect effects

o Conduct monetisation studies for external effects


16/26



16

o Determine how to conduct the equity analysis (the distribution of effects between CBA

stakeholders) Develop an unambiguous and practically applicable definition for the business as usual or

reference case

More Input on freight appraisal

Set guidelines for a balanced presentation of the CBA.

One result of these recommendations was that the Dutch Ministry of Transport, Public Works andwater management together with the Dutch Ministry of economic Affairs started a researchprogramme to address some of these issues. The result of this was that a series of additionalguidelines (appendices to the guidance) were published. These included a study on the distributioneffects of CBA, risk, the monetisation of nature, water and soil and the value of reliability in transport.The distribution effects report (MVW, 2004a) was a methodological report, which concluded thatmore research was required before the guidelines could be updated. The risk report (MVW, 2004b)concluded among other things that a discount rate of 4% (risk free) should be used and that the riskpremium should be 3% as the default value. The nature, water and soil report (MVW, 2004c)provided an example of a case study where the external effects of nature, water and soil andrecreational changes had been monetised for a rail project. For example, in the CBA, wood, drinkingwater, clean surface water, recreational possibilities, recreational disturbance, residentialUnemployment and inheritance nature were all given values through a combination of commercialvalues (e.g. wood and drinking water) hedonic pricing for the recreational aspects and WTP for theinherited nature. One example is that recreational possibilities lost as a result of the project were

valued at 0.5 euros per trip per household. The conclusion of this work was that the monetisation ofthe impacts were very location specific and more work would be needed. The reliability report waspublished in 2005 (Rand Europe and Transport Research Centre, 2005) and was commissioned toresearch filling the gap in appraisal for including the effects of transport time reliability in the CBA.The report has published provisional values of reliability for passenger transport by car and publictransport (0.8 = car reliability ratio, 1.4 = public transport reliability ratio). The report concluded thatthere was no consensus on values of reliability for freight and that more work was needed in thisarea.

5.4 RAILPAG

The development of rail appraisal guidance is something that has been recently considered inRAILPAG (Railway Project Appraisal Guidelines) published by European Commission and EuropeanInvestment Bank (2005). The aim of this guidance was to provide recommendations for a commonframework for the appraisal of rail projects across the EU. It is based on the work carried out forTINA (Transport Infrastructure Needs Assessment) in 1999. The guidelines recommend an extendedCBA approach (p42) coupled with a stakeholder effects matrix to present the results. For eachproject the relevant effects and stakeholders have to be identified. A simplified stakeholder effectsmatrix is presented in table 5. Typical effects for the rail industry refer to user service (e.g. fares,travel time reliability etc...) operation (e.g. direct and indirect costs), assets (e.g. investment) andexternal effects (e.g. environmental impacts). The matrix is completed by filling in each cell with theNPV of the effect to the stakeholder. The NPV for the project can then be calculated by summing thevalue in each cell. Non monetised effects are dealt with using a colour coding system. A cell iscoloured green for mild effects, yellow for those that deserve some attention and red for those that


17/26


18/26



18

6 Given the scope of the NATA refresh what lessons can be learned from all ofthese studies that can be incorporated into the NATA Consultationdocument?

The conclusions we draw are as follows:

1. The UK remains at the forefront of the use of CBA to inform decision making. Whilethere are certain technical points on which UK practice differs from the European normand the HEATCO recommendations, this is not really a problem.

2. There has been an element of opaqueness since the revised Green Book in the

treatment of benefits to foreign travellers within the UK. These are far more importantin European appraisal than in the UK and a degree of consistency with Europeanappraisal practice in this area is desirable.

3. There also needs to be consistency among Western European States in the valuationof transboundary pollution and in particular carbon emissions.

4. If changes are envisaged to the treatment of financing costs in CBA, these might benefitfrom discussion with counterparts in Sweden, the Netherlands and the EIB.

5. The UK seems to be lagging some other countries by not yet using money values forlocal and strategic pollutants other than carbon. Addressing this properly will involvesignificant modelling and valuation challenges (eg PM2.5s)

6. More dubiously, other countries seem to have made progress in areas such as

biodiversity and natural resources which have always been considered difficult in theUK because of their context-specificity. It might be worth looking at how convincing theDutch work is.

7. There is scope for cross-fertilisation with the Dutch and Swedish on topics such as thevalue of travel time with income,distance and purpose and reliability values.

8. Continuing to articulate the role of CBA in decision support and policy analysis isessential. This is the strongest signal that appraisal is taken seriously, as an input todecision making.

9. Outside the studies reviewed here, we recommend continued liaison with Europeanacademics and policy makers in the approaches to estimating and valuing widereconomic benefits.


19/26



19

7 References

Buck Consultants Ltd (2002) Evaluatie OEEI-leidraadhttp://www.verkeerenwaterstaat.nl/Images/Evaluatie%20OEI-leidraad_tcm195-160558.pdf

CfIT (2004) High Speed rail: International comparisonshttp://www.cfit.gov.uk/docs/2004/hsr/research/

CPB and NEI (2000) Evaluation of Infrastructural projects; Guide for cost benefit analysis. Section 1:Main report. Research programme on the economic effects of infrastructure

Eddington R (2006) The Eddington Transport Study. Main Report: Transports role in sustaining theUKs productivity and competitiveness, December 2006 Stationery Office

European Commission and European Investment Bank (2005) Railway Project Appraisal Guidelines -RAILPAGhttp://www.railpag.com/index.php

Danish Transport Research Institute (2002) Use of socio-economic methods in selected countries.

Danish Transport Research Institute (2007) Appraisal methods in the Nordic Countries. Transport

Infrastructure assessment. Report 3.

Giorgi L et al (eds) (2002), Policy and Project Evaluation in Transport, Ashgate, Aldershot seeespecially chapter by W. Parsons

Grant- Muller S. M., P. Mackie, J. Nellthorp and A. Pearman (2001): Economic Appraisal of EuropeanTransport projects: The state of the art revisited. Transport Reviews 21(2) pp237-261

HEATCO (2005a) Current practice in project appraisals in Europe. Analysis of Country Reports.Deliverable 1.http://heatco.ier.uni-stuttgart.de/

HEATCO (2005b) State of the Art in project assessment. Deliverable 2.http://heatco.ier.uni-stuttgart.de/

HEATCO (2006a) Case Study Reports. Deliverable 6.

HEATCO (2006b) Proposal for Harmonised Guidelines. Deliverable 5http://heatco.ier.uni-stuttgart.de/

Ministerie van verkeer en Waterstat and Ministerie van Economische Zaken (2004a) Verdeling vanEffecten Infrastructuurprojecten. (Dutch distribution analysis)


20/26



20

Ministerie van verkeer en Waterstat and Ministerie van Economische Zaken (2004b)

Risicowaardering (Dutch risk analysis)

Ministerie van verkeer en Waterstat and Ministerie van Economische Zaken (2004c) Waardering vanNatuur, water en Boden in Mattschappelijke Kosten- Batenanalyses (Dutch nature, soil and waterreport)

Norwegian Ministry of Finance(2005) veilder for samfunnsokonomiske analyser, finansdepartmentet,2005, http://www.finans.dep.no

Odgaard T. , C. Kelly and J. Laird J (2005): Current practice in project appraisal in Europe. Analysisof Country reports. HEATCO Deliverable 1.

Pearman A.D., Mackie P, and Nellthorp J (eds) (2003), Transport Projects,Programmes andPoliciesEvaluation Needs and Capabilities, Asngate, Aldershot.

PIARC (2003) Economic evaluation methods for road projects in PIARC member Countrieshttp://publications.piarc.org/ressources/publications_files/1/628-09-07-VCD.pdf

RAND Europe and Transport Research Centre (2005) The value of reliability in transport. Provisionalvalues for the Netherlands based on expert opinion.

Shires J and De Jong G C (2006) An international meta analysis of values of time. Annex AHEATCO deliverable 5.


21/26



21

Annex 1: Differences in transport appraisal practice between EU countries

The EUs HEATCO programme recently reviewed transport appraisal practice across EU states. Itconcluded that:

Whilst all countries use cost-benefit analysis in some form, the guidelines about its applicationvary considerably across countries and between modes. Guidelines for Northern/Westerncountries are much more comprehensive than those for Southern/Eastern countries;

Appraisal frameworks for rail are less standardised than for road and only one third ofcountries have guidelines for air, inland waterways and maritime projects;

There are large differences between countries in terms of whether, and how, the main impactsof transport proposals are appraised. For example:

- most countries place monetary values on; construction costs, system operating andmaintenance costs, passenger and freight time savings, user charges/revenues andsafety impacts;

- only half of EU countries (and no Southern countries) monetise noise impacts. Only halfof countries monetise local/regional air pollution impacts;

-

most countries do not place monetary values on climate change effects and disruptionfrom construction impacts

- Eastern and Southern EU countries seldom place monetary values on environmentalimpacts.

there are differences between countries about which unit of account (market price or factorcost) is used, discount rates and appraisal periods;

only a few countries include distortionary impacts from tax financing and transboundaryeffects. However where this is done, there is a consensus on the approaches used;

differences exist about which elements of construction costs to include, residual lives andasset lives. Most countries adjust costs for optimism bias, but only a few do so using advancedmethods;

only half of the countries have standard figures for system operating and maintenance costs;

all countries include travel time savings and most (particularly Northern/Western EU countries)disaggregate by work/non-work and by mode. However there are differences in how countriesassess time savings. For work trips, the most popular approach is the cost saving approach,whilst non-work trips tend to be based on willingness to pay (particularly in Northern/Westerncountries) and a relationship to the wage rate;


22/26



22

all countries include vehicle operating costs associated with road transport and mainly to a

standard definition;

there is a significant range of monetary values for safety is used and no consensus on whethervalues should change over time. About one third of countries use a non-standard definition ofaccident types;

of those countries that place a monetary value on noise, nearly all are based on HedonicPricing studies and all include noise annoyance in the valuation, but only a few include healthcosts related to noise. Not all countries increase noise values over time;

for local/regional air pollution there is no consensus on which elements to include for thosecountries that include a monetary value. There is also no consensus on valuation techniques,though the impact pathway approach is most commonly used;

there are clear regional differences on the treatment of climate change effects. MostNorth/West countries value climate change effects, but few Eastern and Southern countries doso. There is no single approach for assessing the money value and a significant range ofvalues are used.

very few countries include environmental effects other than noise, local/regional air pollutionand climate change

very few countries include indirect socio-economic effects, such as agglomeration benefits,community severance and access to the transport system.


23/26



23

Annex 2: HEATCOs general recommendations for developing transport appraisal frameworks

1. Appraisal as a comparative tool. To estimate the costs and benefits of a project, one has tocompare costs and benefits between two scenarios: the Do-Something scenario, where the projectunder assessment is realised, and a Do-Minimum scenario, which needs to be a realistic base casedescribing the future development. If there are several project alternatives, one has to create ascenario for each alternative and compare them with the Do minimum case.

2. Decision criteria. We recommend the use of NPV (net present value) to determine, whether aproject is beneficial or not. In addition, depending on the decision-making context respectively thequestion to be addressed, BCR (benefit cost ratio) and RNPSS (ratio of NPV and public sector

support) decision rules could be used.

3. The project appraisal evaluation period. We recommend the use of a 40 year appraisal Period,with residual effects being included, as a default evaluation period. Projects with a shorter lifetimeshould, however, use their actual length. For the comparison of potential future projects, a commonfinal year should be determined by adding 40 years to the opening year of the last project.

4. Treatment of future risk and uncertainty. For the assessment of (non-probabilistic) uncertainty, weconsider a sensitivity analysis or scenario technique as appropriate. If resources and data areavailable for probabilistic analysis, Monte Carlo simulation analysis can be undertaken.

5. Discounting. It is recommended to adopt the risk premium-free rate or weighted average of therates currently used in national transport project appraisals in the countries in which the TEN-Tproject is to be located. The rates should be weighted with the proportion of total project financecontributed by the country concerned. In lower-bound sensitivity analyses, in order to reflect currentestimates of the social time preference rate, a common discount rate of 3% should be utilised. Fordamage occurring beyond the 40 year appraisal period (intergenerational impacts), e.g. for climatechange impacts, a declining discount rate system is recommended.

6. Intra-generational equity issues. We recommend, at minimum, that a winners and losers tableshould be developed, and presented alongside the results of the monetised CBA. Distributional

matrices for alternative projects might be created and compared amongst each other. Additionallystakeholder analyses should be undertaken as well. It is recommended to use local values to assessunit benefit and cost measures.

7. Non-market valuation techniques. If impacts in transport project appraisals cannot be expressed inmarket prices, but are potentially significant in the overall appraisal, we recommend that in theabsence of robust transfer values non-market techniques to estimate monetary values should beconsidered. We recommend that the choice of technique used to value individual impacts should bedictated by the type of impact and the nature of the project. However, Willingness to Pay (WTP)measures is preferable to cost-based measures. Values should be validated against existingEuropean estimates.


24/26



24

8. Value Transfer. Value transfer means the use of economic impact estimates from previous studies

to value similar impacts in the present appraisal context. Value transfers can be used wheninsufficient resources for new primary studies are available. The decision as to S3 HEATCO D5Summarywhether to use unit transfers with income adjustments, value function transfer and/or meta-analyses will depend on the availability of existing values and experience to date with value transfersrelated to the impact in question.

9. Treatment of non-monetised impacts. We recommend, at a minimum, that if impacts cannot beexpressed in monetary terms, they should be presented in qualitative or quantitative terms in additionto evidence on monetised impacts. If only a small number of non-monetised impacts can beassessed, sensitivity analysis may be used to indicate their potential importance. Alternatively, non-monetised impacts may also be included directly in the decision-making process by explicitly eliciting

decision makers weights for them vis-vis monetised impacts.

10. Treatment of indirect socio-economic effects. We recommend that if indirect effects are likely tobe significant, an economic model, preferably a Spatially Computable General Equilibrium (SCGE)model, should be used. Qualitative assessment is recommended, if indirect effects cannot bemodelled due to limited resources (high costs for the use of advanced modelling), insufficientavailability of data, or lack of appropriate quantitative models or unreliable results.

11. Marginal Cost of Public Funds. Our recommendation is to assume a marginal cost of public fundsof 1, i.e. not to use any additional cost (shadow price) for public funds. Instead, a cut-off value for theRNPSS of 1.5 should be used when relevant.

12. Producer Surplus of Transport Providers. We recommend to estimate (changes in) the producersurplus generated by changed traffic volumes or by the introduction and adjustment of transportpricing regimes.

13. Accounting procedures. a) Factor costs should be the adopted unit of account. This requiresmeasures expressed in market prices - which include indirect taxes and subsidies to be convertedto factor costs. b) We recommend to convert all monetary values into with a price level for a fixedyear. In this report, monetary values are given as 2002, i.e. with 2002 as base year. However, themonetary values should be adjusted with the Purchasing Power Parity (PPP) as explained in Annex

B, which also contains a table with PPP adjustment factors. However, these factors are only availablefor past years, whilst future PPP factors are likely to change as the economic growth rates differamongst countries. As we assume, that income and prices grow faster in Member States withcurrently low income, PPP factors will tend to converge closer to 1 in the future. Therefore, werecommend that two calculations are made one with and one without PPP adjustment assumingthat the true value will lie between the two results. c) Monetary values, i.e. preferences, for non-market goods like reduced risk of getting ill or reduced damage to the environment will increase withincreasing income; thus we recommend increasing monetary values based on GDP growth a tablewith possible country-specific GDP growth is given in Annex B.

14. Up-dating of values. The unit values supplied in this report represent the state-of-the-art for the

individual impacts addressed. Nevertheless, all values will be subject to change as new empiricalevidence becomes available and methodological developments take place. As a consequence, we


25/26



25

recommend that values are reviewed and up-dated on a regular basis e.g. after three years at

maximum.

15. Presentation of results. As far as possible, impacts should be expressed in both physical andmonetary terms. The results of the sensitivity analysis and the non-monetised impacts should bereported together with the central monetised results.


26/26



Annex 3: Shires and de Jong (2006)

The European Unions HEATCO research programme carried out a meta-analysis of value of time(VoT) studies. Below is a summary of the reports main conclusions.

The meta analysis estimated regression equations on almost 1,300 values of time from studiesaround the world, mostly studies conducted after 1990. The researchers also estimated models onmore than 130 values of time for freight transport. For both passenger and freight transport, theyestimated double logarithmic models with ordinary least squares. They also used fixed effects andrandom effects panel models that account for the fact that there were repeated observations for thesame country. The estimation results led to the following general conclusions:

A (largely cross-sectional) income elasticity of the VoT of about 0.5 for business travel, 0.7 forother passenger transport and 0.3 for freight;

Long distances lead to higher Value of Travel Time Savings (VTTS) for commuting and otherpurposes. Stated Preference and combined Stated Preference Revealed Preference studiesgive somewhat lower passenger VoTs than the cost savings approach. They found nosignificant effect of study method for freight transport;

The researchers found significant effects for purpose (business, commuting) and mode(especially for air; less so for bus relative to car and train);

They also obtained higher values of time in Southern and Eastern European countries, allother things (including GDP/capita) being equal.

The estimation results have been applied to each of the 25 EU countries to get VTTS by purpose,either for comparison against existing national values, or as a basis for our recommendations forcountries with missing VTTS.

transport appraisal in other countries: lessons for the nata refresh

Documents