managing for results in the road sector

Managing for Resultsin the Road Sector

16 May 2011 SingaporeAdrien Véron, EARD, ADB

Session Organization

Managing Road Sector Performance

Four Modules

Delivering to and with Road Users

Performance Contracting

Road Asset Management

Country Presentation Pakistan

Country Presentation Nepal

Group Discussion

Country Presentation Mongolia

Module 1: Managing Road Sector Performance

Key Learning Objectives

Understanding what results-based management can mean in the road sector

Being aware of the internal and external factors that affect performance

Learning about which performance indicators and when to use them

Getting to know about good practices in road sector performance management

Why Results-Based Management?

“A government that works better and costs less”

What is Results-Based Management?

“A government that does the right things right”

Birth Growth Upgrading Mature

Doing Managing Contracts

Accounting for Performance

Delivering Services

Government Government /IFIs

Government / Private

Government / Private / Users

Force-account Supervision, Contracting

Design, Maintenance

PPPs, Asset Management,

Focus

Funds

PrivateSector

Source: Adapted from OECD, 2003, Internal Performance Improvement of Road Administrations

Trends in Public Sector Management

Government Role

Management Models

Control

Separation political / execution levelsDecentralization Independence of agenciesContracting-out & privatizationTransparency & user participation

Defining performance and targetsAllocating budgets to performance targets and reportingPerformance agreements within agencies’ business plansDelegating authority to managersPerformance-based managers’ contracts

From control of inputs to outputsFrom pre-audit to post-audit Performance audits & evaluationFrom external control to internal audit

Planning / BudgetingFrom fixing problems to creating outcomesFrom one-year to multi-yearHigh-level program /output / sector budget allocations

SystemsIntegrated financial systemsInformation & management systems

Factors Influencing Road Sector Performance

Drilling Down Performance in a Road Maintenance Program

Source: Based on case study from Brazil

Clarity and Measurability of Objectives

• Incomplete• Unmeasured• Un-integrated• Unused

Efficiency in Planning and Funding

• Rigid budget structure• No budgeting for DD & supervision• Non-technical planning• Extra-budgetary practices• Unskilled civil servants• Poor program perception

Cost-Efficiency in Execution Mechanisms

• Cumbersome bidding processes• Slow payments / old contracts• Systematic 25% cost overrun• High monopoly bitumen price• Weak control of supervision • Corruption incentives

M&E Systems

• High rates of error• No management view• No dynamic / warning approach

Weak demand for results

Many veto playersPolitical investmentsPublic sector HRUnsolved issuesBudget rigidity

Paralyzing controlLeadership changesPolitical staff

Erratic demands for information

No capacity to measure indicators

Teething conflictsExcessive externalizationNascent planning

Ad hoc approach to projects

Overlapping responsibilities &multiple hierarchical lines

Separate systems

Ad hoc approach to information gathering

Factors Influencing Road Sector Performance

Drilling Down Performance in a Road Maintenance Program

Source: Based on case study from Brazil

Clarity and Measurability of Objectives

• Incomplete• Unmeasured• Un-integrated• Unused

Efficiency in Planning and Funding

• Rigid budget structure• No budgeting for DD & supervision• Non-technical planning• Extra-budgetary practices• Unskilled civil servants• Poor program perception

Cost-Efficiency in Execution Mechanisms

• Cumbersome bidding processes• Slow payments / old contracts• Systematic 25% cost increases• High monopoly bitumen price• Weak control of supervision • Corruption incentives

M&E Systems

• High rates of error• No management view• No dynamic / warning approach

Transport Council

Strong demand for results Priority program definitionPPPs

Performance audits Regular

performance reviews

Creation of institute

Long-term planningNew staff hiringProgram clean-up

Performance-based contracts

Unification of program structure

Single-point entry systemUser feedback and transparency

A Performance Framework for the Road Sector

Source: Adapted from OECD, 2003, Performance Indicators for the road sector

Assets HRSystems

Road Agency

Transport Sector

Processes - Manage organization - Gather information - Plan activities - Allocate resources - Procure & deliver products & services - Evaluate efficiency

INPUTSCapitalMaterialsLabor

GOVERNMENT GUIDANCE Objectives & goals Priorities Policies & regulations Resources

Outputs

Desired - Road system - Maintained assets - User services - Information

Undesired - Road works safety - Land occupation - Environmental degradation

Outcomes

Users - Accessibility - Road quality - User satisfaction - Congestion - Accidents

Community - Urban quality - Air pollution - Landscapes

Impacts

Employment

Market Shifts

Equity

Environment

Use of

Resources

Activity

Patterns

Land Use

Price Changes

Road User and Community Feedback

Performance Indicators: When?

Indicators as a key management tool

Performance

Indicators

ObjectivesAnd Goals

Road ProgramDevelopment

Road ProgramDeliveryRoad Program

Performance

Impacts

Observed &analyzed

ReflectedEvaluated

Monitored


SMART indicators: Specific Measurable Attributable Realistic

Targeted

Government Road Agency Road Users

HCM levels of ServiceAverage road user costsTransport cost / GDPAccessibility index

O&M costs by veh.kmTravel time and reliabilityQuality of traffic information to users

Level of satisfaction on traffic time and info.Hours of congestion delayUser charges / veh.km

Accessibility

Fatalities per populationAccidents by veh.km% of accidents involvingdrunken driver

% of traffic speedingSafety program assessments% of roads not meeting design standards

Unprotected road user riskTime from alert to treatment

Safety

Air quality standardsCumulative land taken %

Environmental programEmissions per capita for CO2, NOx, VOC, PM

% of pop exposed to noise levels > 65db% of pop exposed to air quality below standards

Environment

Regional distribution of roadsLaws for access by reduced mobility people

Balance of user charges / expenses by regionMarket research & customer feedback

Travel costs /user groupAccident risks /user groupSatisfaction with feedback capacity

Equity


SMART indicators: Specific Measurable Attributable Realistic

Targeted

Government Road Agency Road Users

Program Performance

Program Preparation and Delivery

Value of assetsEx-post value of CBATrends per programRoad expenditures / GDP

Pavement roughnessBearing capacity% of defected bridgesCongested roads – kmTruck overloading incidence

Surface conditionRest areas / 100 km% of main roads lightedFrequency of emergency and winter road closuresUser information system

Clarity of objectivesStability of leadershipDegree of completion of long-term programsSufficiency of maintenance funding

Forecast costs vs. actualCost of O&M per lane.kmOverhead percentStaff / lane.kmDelay from preparation to implementation of projects% of contracts with overruns and delays

User delay associated with maintenance and worksRoad works safety and nuisancesTransparency & competition


Good Practice (1)

New Zealand

Annual business plans and strategic plans

Baseline expenditure reviews to assess output prices

Annual performance and value-for-money auditing by auditor general

Multiyear fiscal planning

Statement of Intent and Performance Reports approved by Parliament

5-year performance based contracts for chief executives

See: http://www.nzta.govt.nz/planning/index.html

http://www.nzta.govt.nz/planning/index.html

Good Practice (2)

Japan

Performance plans and achievement reports

Data disaggregated at prefecture level

Customer surveys

Outcome-based budgets

Analysis of effectiveness

Internal motivation on outcomes through publicity and peer comparisons

Source: Adapted from T. Hishio, Y.Tsukada, T.Oba and M.Ohno, 2005, Outcome-oriented Performance Management of Road Administration in Japan. MLIT See also: http://www.mlit.go.jp/road/road_e/index_e.html

http://www.mlit.go.jp/road/road_e/index_e.html

Module 2: Delivering for and with Road Users


Understanding the different forms of user participation

Learning about specific participation mechanisms: Perception Surveys, Transformation Forums, Road Boards

Understanding the concepts of levels-of-service

Acknowledging Stakeholders

Who?

Road users

Communities affected

Downstream businesses

Road industry

Motorists’ associationsPublic transport users associationsTaxi AssociationsTruck operators associations

Local governmentsNGOs for special interests, inc:

- Pedestrians- Environment, etc.

Industrial / mining associationsChamber of CommerceTourism industryAgricultural groups

Professional AssociationsIndustry AssociationScientific and academic groups

Involving Stakeholders

Increasing Level of Public Impact

GOAL

COMMITMENT

TOOLS

INFORM CONSULT INVOLVE COLLABORATE EMPOWER

Source: Adapted from International Association for Public Participation www.iap2.org

To inform users and assist them in understanding problems

To obtain public feedback on analysis and alternatives

To work with the public to ensure public issues are considered

To partner with the public in all analysis and decision-making steps

To place decision-making power in the hands of the public

We’ll keep you informed

We’ll keep you informed and listen to your feedback

We’ll ensure your concerns are reflected and tell you how

We’ll always seek your advice and recommendations

We’ll do what you decide

• Reports• Websites• Open houses

• Comment• Call centers• Surveys• Public meetings

• Workshops• Deliberative meetings

• User advisory groups• Participatory decision-making

• Road Boards• Delegated decisions

Good Practice (1) Perception Survey

Governance in Kazakhstan / Armenia

Road Agency Governance Index

Source: C.Queiroz, A.Martinez, S.Ishibara, K.Homman, 2011, Road Asset Governance Filter: Case Study of Kazakhstan and Armenia, World Bank, DC.

Small-scale / low cost

Four dimensions:

Transparency, disclosure and accountability Transparent procurement systems Financial management systems Administrative procedures and anticorruption

Internal & external stakeholders

KazakhstanArmenia

Good Practice (2) Participatory Decision-Making

Mongolia, National Road Sector Capacity Development Roadmap

Source: ADB, 2010, Technical Assistance for the Preparation of a National Road Sector Capacity Development Roadmap, Manila

Diagnostic

• Capacity assessment

Collaborative

Preparation

• Shared diagnostic• Propose objectives• Evaluate options• Concrete propositions

Evaluation Consultatio

n

• Preparation of roadmap• Consultations on draft

Finalization

• Formal drafting• Presentation to Cabinet

Implementation

• Set-up task forces• Conduct legal changes• Trainings• Monitoring & Evaluation

WHO?

Consultant, ADB

Stakeholder groups

Facilitator

Road Agency +

Stakeholders

Ministry

Road Agency

Stakeholders

Consultants

How

long?

6 months 4 months 3 months 2-4 months 5 Years?

Good Practice (3) Road Boards

Second Generation Road Fund Criteria

Source: ADB, 2003, Roads Funds and Road Maintenance, an Asian Perspective, Manila, and World Bank, 1995, African Road Funds, What Works and Why?, Washington, DC

Road Fund only relies on road user fees to pay for roads O&M

Road Fund to act as purchaser, not provider

Road Fund to be managed by an independent Board

Road Board to recommend level and adjustments to road fees

Road Board to be composed in majority of user and business representatives

Road Board to have a secretariat and be managed under sound commercial practices

User fees to be directly deposited in the Road Fund

Road Fund to be concentrating on road maintenance

Levels of Service (LoS)

Managing the Roads for the Users

Source: CSS (2004) Framework for Highway Asset Management, County Surveyors Society

“Levels of Service describe the quality of services provided by the asset for the benefit of the customers.” “Levels of service are the manner by which the highway authority engages with the customer.” “Levels of service are about reflecting the customer’s interests in terms that can be measured and evaluated.”

Budgets

Plans

Works

Levels of Service

Road users satisfaction

Budgets

Plans

Works

Levels of Service

Road users satisfaction


Use of Levels of Service

Document and measure services provided

Rationally evaluate service versus cost trade-offs

Determine if adequate focus is given to what matters to the customer

Plan operational activities to support strategic goal to meet LOS

Type of Levels of Service

Condition of assetObjective / structural

Perceived by the users

Service aspirations

Safety

Performance (congestion, availability)

Environmental impacts

User services

Others.

Source: CSS (2004) Framework for Highway Asset Management, County Surveyors Society


Source: Hertfordshire’s Highway Asset Management Plan

Outcome

Safety of asset

Condition

Availability / Accessibility

Environmental impact

Customer service

Financial performance

Level of Service

Accident reduction

Safety related defects

Emergency management

Education, training and publicity

User perception

Performance measure

% of unauthorized objects on the network removed

% of potholes repaired within 24 hours

% of signs missing

No of accidents linked with snow and ice

% of accidents linked with asset condition

No of traffic signals not operating

Calculation method

% of hazardous objects removed within 2 hrs (contract response time) of notification, expressed in percentage

No of potholes repaired within 24 hours divided by the number reported faulty, for publicly reported faults, expressed as a percentage

Module 3: Contracting Maintenance for Performance


Being aware of the various ways to contract road maintenance

Understanding what is performance-based contracting and its benefits

Getting familiarity with performance specification issues

Getting familiarity with compliance monitoring issues

The Maintenance Contracting Menu

Hors-d’oeuvre• Low private sector involvement

Hors-d’oeuvre• Low private sector involvement

Entree• Performance contracts

Entree• Performance contracts

Desert• Full devolutionDesert• Full devolution

In-House MaintenanceIn-House Maintenance

RO

UTIN

ER

OU

TIN

E

EM

ER

GEN

CY

EM

ER

GEN

CY

PER

IOD

ICP

ER

IOD

IC

MA

NA

GEM

EN

TM

AN

AG

EM

EN

T

Salesman modelSalesman model

Partial competitionPartial competition

Rehabilitation and MaintenanceRehabilitation and Maintenance

Integrated / area modelIntegrated / area model

Brownfield concessionsBrownfield concessions

PPPsPPPs

Routine maintenanceRoutine maintenance

Framework modelFramework model

Bundling modelBundling model

Input based

Unit prices 1 year / seasonal

Unit prices 1 year / seasonal

Unit prices Indicative BOQs 3-5 years

Lump-sum / PBC 5-10 years


Lump-sum / PBC Unit / Global prices 5-10 years


Tolls / penalties 20-35 years

Tolls Lump-sum / PBC 20-35 years

Performance-Based Maintenance Contracts (PBCs)

What is a Performance-Based Maintenance Contract?

Source: Adapted from http://www-esd.worldbank.org/pbc_resource_guide/TrainingMaterial.htm

In PBCs, the contractor gets paid for delivering a service, not physical works

Service is considered delivered when expected performance indicators are reached:- Roads free of potholes

- Grass cut, etc.

Client specifies only performance targets, not methods and materials

If some performance indicators are not met, contractor does not get full payment for the related service (or gets penalties)

Lump sum payments are made periodically- some items based on unit prices- adjustment for change in factors (e.g. traffic, materials prices, inflation)

To meet performance targets, contractor is free to decide:- What to do- When to do- How to do- Where to do- To do himself or subcontract (with limits)

Contractor is responsible for Quality Assurance and data gathering

Procurement combines price and technical quality

Contracts last several years

http://www-esd.worldbank.org/pbc_resource_guide/TrainingMaterial.htm

Performance-Based Maintenance Contracts (PBCs)

What are the Benefits of Performance-Based Contracts?

Source: Adapted from World Bank, 2005, Performance-based Contracting for Preservation and Improvement of Road Assets, Washington, DC

Cost savings

Risk sharing

Incentives for innovation and good management Reduction in administrative expenses Possibility to reduce client staffing levels Greater flexibility to reward performance

Predictability of costs and financing

Reduction in variation orders Support for ensuring stable financingLife-cycle cost decisions

Focus on users / Levels of Service

Source: Finish Road Enterprise, 2002, Innovative Project Delivery Methods for Infrastructure, Helsinki

Cost Saving Potential

Compliance monitoring

Internal Q&A


Ensure performance standards are constantly met

Make available information to the Client:- summary sheets- daily logs of deficiencies and actions taken- inspection logs in accordance with QA plan

Implement a Quality Control Plan for constant monitoring

Inspections

Formal monthly inspections on selected number of roads- to verify summaries and inspection logs

Road user complaints

Informal inspections, at the client’s initiative, at any time and anywhere- Notification to contractor within ideally 24 hours of finding- Contractor to take remedial action within a certain time frame- In specified cases, result in penalties

Act upon deficiencies in a pro-active manner

It is Contractor’s responsibility to:

Setting-up Performance Standards: Group Exercise

Contract Type


National roads, 200km bituminous, 150km gravel

Contract for the rehabilitation and maintenance of the roads, 10 years

Traffic at 3,000 vehicles per day on the first stretch and 500 vehicles per day on the second

Road Agency Objectives

Minimize life-cycle and administrative costs

Satisfy users’ aspirations- accessibility

- comfort- speed- safety- information

Hilly terrain, with occasional snow-blocks and heavy rainfalls

Setting-up Performance Standards: Exercise Sheet

Performance Standard

Measurement Mechanism

Response time and corrective

action

Routine Maintenance

Periodic Maintenance


Module 4: Asset Management


Understanding Road Asset Management Planning and Life-Cycle Cost concepts

Understand the different components of a Road Asset Management System

Be aware of the various outputs produced by a standard HDM-IV based system

Be aware of data gathering tools, strategies and sustainability issues

Asset Management

How much are asset worth?

Source: Adapted from http://www-esd.worldbank.org/pbc_resource_guide/TrainingMaterial.htm and New Zealand Infrastructure Asset Management Manual

UK Highways = IBM

Japan Road Public Corporation = General Motors

Central and Eastern Europe = $550 billion

Typical developing country: up to 50% of GDP

What is Road Asset management?

“The combination of management, financial, economic, engineering and other practices applied to road assets with the objective of providing the required level of service in the most cost- effective manner” Basic Asset Management “…relies primarily on the use of an asset register, maintenance management systems, job/res- source management, inventory control, condition assessment and defined levels of service in order to establish alternative options and long-term cash flow predictions”

Advanced Asset Management “…employs predictive modeling, risk management and optimized renewal decision making techniques to esstablish asset licycle options and related long-term cash-flow predictions”

Asset Management Planning

Source: Adapted from World Bank, University of Birmingham, Senior Road Executives Program, http://www.worldbank.org/transport/training/birmingham-un/start.html

Organizational VisionCustomer expectations• Levels of Service• Cost of transport

Existing Assets

Non-Asset Solutions

• Capacity Building•Demand management

Government guidance• Financial• Environmental

Strategic Planning Process

Asset Management Process

New Assets

Surplus Assets

Maintain

Renew Upgrade Assets

Create Assets

Dispose Assets

Improved Performanc

e

Asset Management Benefits

What are the Benefits of Performance-Based Contracts?

Better understanding of service level options and requirements

Minimal (optimized) long-term costs for given LoS at the network level

Clear justification for budget requests


Better understanding and forecasting of asset-related options and costs

Managed risk of failure

Informed and objective (“scientific”) decision-making based on costs and benefits

Improved organization image

Asset valuation capacity

Life-Cycle Costs

Source: World Bank, University of Birmingham, Senior Road Executives Program, http://www.worldbank.org/transport/training/birmingham-un/start.html

Maintenance Strategies


Asset Condition

EXCELLENT

POOR

Deterioration

TIME

Level of Service Preventive treatments

Asset Management Planning

Management Cascade


Strategic Planning

• Mid-term budget needs to meet LoS• Forecast pavement condition• User cost forecasts• Compared NPV of alternative strategies, standards

Programming

• Annual budgets• List of road sections with recommended treatment / cost• Rolling program

Preparation

• Project design or formulation• Procurement

Works

• Performance standards• Invoices• Progress reports• Cost analyses

Data Collection

Road Asset Management System

Road Asset Management Systems Components


Data Collection

Database Decision support tools

Management Information

•Inventory •GIS coordinates•Riding quality•Surface distress•Pavement strength•Traffic flow/load•Brides•Furniture

• Forecasting models• Optimization models

-> HDM-IV

• Standard reports• Custom reports

Road Management Systems Sample Outputs


Long-term NeedsIntervention Strategies

Optimal Budget Allocations

Rolling Works Plan

Data Collection

What to collect? Factoring in data costs, skill needs and use

Source: Adapted from World Bank, 2006, Data Collection Technologies for Road Management, Washington

Collect only what you need

Collect at the lowest level of details you need

Collect when you need

IQL-5

IQL-4

IQL-3

IQL-2

IQL-1

IQL-5: High-level aggregate data- performance indicators

IQL-4: Data used for planning and management reportsIQL-3: Network-level survey data, simple data gathering methodsIQL-2: Data used for detailed designs and project-level decisions

IQL-1: Research level data, many attributes

Information Quality Level (IQL)

Data Collection: Cost/Performance trade-offs

Source: World Bank, 2006, Data Collection Technologies for Road Management, Washington

What have we learnt?

Managing Road Sector Performance

Four Modules

Delivering to and with Road Users

Performance Contracting

Road Asset Management

Country Presentation Pakistan

Country Presentation Nepal

Group Discussion

Country Presentation Mongolia

Thank [email protected]

managing for results in the road sector

Documents

road sector16

road maintenance programsource

performance targets

performance indicators

road sectorbeing aware

case study

good practices

external factors