electricity markets regulation - lesson 6 - efficiency assessments

Experience you can trust.http://www.leonardo-energy.org/training-module-electricity-market-regulation-session-6

Training on Regulation

A webinar for the European Copper Institute

Webinar 6: Efficiency Assessments

Dr. Konstantin Petrov / Dr. Daniel Grote

11.1.2009

http://www.leonardo-energy.org/training-module-electricity-market-regulation-session-6

11/01/2010 2

Agenda

a) TOTEX versus OPEX benchmarking

3. Application of efficiency results

a) Overview

2. Methods for efficiency assessments

c) Data Envelopment Analysis

b) Performance indicators

b) Efficiency convergence speed

1. Why measure efficiency?

c) Supporting schemes

e) Virtual network models

d) Parametric Approaches


11/01/2010 3


Regulation is needed in areas where

competition does not work (e.g. natural

monopolies - transmission, distribution

networks) to limit excessive pricing and

to set incentives for efficient

performance

Regulators apply benchmarking to

assess efficiency of regulated

companies for the purposes of

incentive regulation

Major Reasons

Cap regulation

Actual Cost

Current price levelCurrent price + InflationCurrent price + Inflation – productivity growth

Efficiency gains

time

Influenced by company

Influenced by company

Set by regulator


11/01/2010 44


Definition of efficiency

Efficiency = OutputsInputs

+ “Correction for Environment”

Distribution Company

e.g. # employees, fuel, operational costs,

Input Factors

e.g. # customers, delivered energy (kWh), peak load (kW)

Output Factors

e.g. firm size, network topology, climate, topography, terrain, task complexity

Environmental Factors


11/01/2010 5


Technological change (frontier shift): change in production technology within the

sector

Efficiency change (catch-up): change in efficiency of production

Change in the scale of production (scale efficiency)

Pure technical efficiency change

Allocative efficiency

Input mix allocative efficiency: producing same outputs with different mix of

inputs

Output mix allocative efficiency: producing different level of outputs with same

mix of inputs

Changes in operating environment

Reasons for efficiency changes


11/01/2010 6


Efficiency assessment and price control

Efficiency

Assessment

Efficiency

Scores

Efficiency

Improvement

Targets

Integration in

Price

Control

Allowed Revenue (Tariffs)Efficiency

Inte

rfac

e

Benchmarking

– Approach

– Sample

– Model Orientation

– Data Collection

– Data Validation

Conversion

– Convergence Time

– Convergence Profile

– Inefficiency Caps

– Efficiency Bands

Integration

– Chargeable Basis

– Capex Treatment

– Revenue Requirements

– Regulatory Formula


11/01/2010 7


Practical Relevance of Benchmarking and the X-factor

Reflects the regulatory view for anticipated efficiency improvement

Ensures ex-ante sharing of the anticipated efficiency gains between customers and

regulated companies

The X-factor is not a confirmation but rather indication of the anticipated efficiency

improvement

In some regulatory regimes the X-factor has a dual function:

Efficiency improvement

Revenue profiling


11/01/2010


Overview (1)

Benchmarking Methods

Partial Methods Total Methods

Non-parametric Parametric

Reference Networks (Virtual

Networks)

Index Methods

Data Envelopment

Analysis (DEA)

Stochastic Frontier Analysis

(SFA)

Ordinary Least

Squares (OLS)

Corrected Ordinary

Least Squares (COLS)

Total Factor Productivity

(TFP)

Uni-dimensional

ratios

Performance Indicators

Linear programming

Econometrics

Engineering Models

8

Total methods can be based on the average performance or the efficient frontier of comparable companies


11/01/2010 99


Overview (2)

Efficiency performance assessment (benchmarking) applied in various forms

Methods differ in the standard of comparison

No consensus among regulators as to which methodology is best

Sometimes different methods applied simultaneously for cross-checks

Frontier methods preferred by regulators, in particular DEA and SFA

– Parametric (econometric) models (Germany, UK)

– DEA analysis (Norway, the Netherlands, Germany, several countries in CEE)

– Reference network models (Spain, Sweden, Chile, Argentina)


11/01/2010 1010


Overview (3)

Efficiency Score

A B C D E

Measures of relative inefficiencies towards

best performance

Conversion (definition of

efficiency increase targets)

Companies


11/01/2010 11


Performance Indicators

Uni-dimensional ratios:

– Comparison of single performance indicators between firms

– Fails to account for the relationships between different input and output factors

Productivity (Managerial) Indicators

– GWh/Employee

– OPEX/GWh

– OPEX/Employee

– GWh/Line Length

Financial indicators

– Debt/Equity Ratio

– Return on Investment (ROI)

– Return on Capital Employed (ROCE)

Partial methods produce simple, easy to calculate straightforward indicators of performance

… but do not recognize trade-offs between different improvement possibilities or areas

Can only be used as a rough indication


11/01/2010 12


Index methods – Total Factor Productivity (TFP)

Total factor productivity (TFP) is a measure of the physical output of a regulated company produced by a given quantity of inputs

With multiple inputs (Y) and outputs (X), outputs are usually weighted by their revenue shares (sR) and inputs are weighted by their cost shares (sC)

Weights can be either static or dynamic (different weights used for each period) Extensively used in the US for utility regulation (both energy and telecoms) Data requirements can be harsh TFP does not provide any information about ‘infra-marginal’ efficiency improvement

possibilities; for this we need more articulated benchmarking techniques (frontier-based methodologies)

More suitable for an assessment of company performance over time than comparisons between regulated companies

n

jjj

C

m

iii

R

Xs

YsTFP

1

1

Input factors

Output factors


11/01/2010 13


Frontier methods

Frontier methods are based on the concept that all companies should be able to operate at

an optimal efficiency level that is determined by other efficient companies in the same

sample

These efficient companies are usually referred to as the “peer firms” and determine the

“efficiency frontier”

The “efficiency frontier” is formed from the observed performance of the companies in the

analyzed sample, as determined by the relationships between the inputs and outputs of the

sampled units

The companies that form the efficiency frontier use the minimum quantity of inputs to

produce the same quantity of outputs (input oriented model)

The “efficiency frontier” is used as a reference against which the comparative performance

of all other companies (that do not lie on the frontier) is measured

The distance to the efficiency frontier provides a measure for the inefficiency


11/01/2010 14


Data Envelopment Analysis (DEA) (1)

Output 1 Input 1

Input 2

Data Envelope

A

B

C

DE

most efficient companies

F

F’

Inefficiency

Input minimisation

Inefficiency

Output 2

Data Envelope

A

B

C

D

E

F

most efficient companies

F’

Output maximisation

G

G’


11/01/2010 15



Outputs

Inputs

A

B

C

constant returns to scale frontier

variable returns to scale frontier

F

F’

Variable returns to scale account for short-run scale inefficiencies In the long run, firms should optimally adjust their size so that constant returns to scale are achieved


11/01/2010 16



DEA is a non-parametric approach to calculate the relative Input-Output efficiency of a

regulated company

DEA benchmarks an individual company in relation to the best-practice (most efficient)

companies

Companies that are able to produce a given output at minimum cost or a maximum output

with a given input define the best-practice frontier that envelops all data points

Inefficiency is determined by the distance between the observed company and the best-

practice frontier

Calculation of inefficiency is done via a series of linear programming (mathematical

software needed)

The programs will output a series of efficiency scores, which may be normalized, ranked,

and split according to a number of components (scale, purely technical, allocative etc.)


11/01/2010 17



Advantages:

– Multi-dimensional method covering multiple inputs and outputs

– Establishes peer companies

– It does not require functional relationships between input and output factors

– Distinguishes between different types of inefficiency (scale, productive, allocative,

purely technical) in the presence of input (or output) price data

Disadvantages:

– The results could be influenced by random errors, measurement error or extreme

events

– Results depend on the selection of input and output factors

– Companies exhibiting “extreme” parameters will be classified as efficient “by default”

– Provides no information about statistical significance of the results

– Small samples and a high number of input or/and output variables can result in an

over-specification of the model and “made-up” results for efficiency scores (number of

efficient firms increases with the number of input and output variables)


11/01/2010 18


Parametric / Econometric approaches – Regression analysis

Corrected OLS (COLS)

Ordinary Least Square (OLS)

Most efficient observation

Input (Costs)

Output

Stochastic Frontier Analysis (SFA)


11/01/2010 19


Ordinary Least Squares (OLS)

Regression analysis: Mathematical relationship (functional form) that describes the

relationship between a dependent variable and one or more independent variables

Used to determine the values of parameters that cause the function to best fit a set of data

observations

The OLS regression line cuts across the observations by minimising the sum of the

squares of the distance (residual) between the line itself and each of the observations

Fit a line so that, at each point, the (regression) line is close to the corresponding observed

values, while minimising the sum of squared deviations from the line over all the

observable values in the sample

Efficiency frontier is based on the average cost function

OLS compares the (in)efficiency of an individual company with the average efficiency level


11/01/2010 20


Corrected Ordinary Least Squares (COLS)

Estimation of production or cost functions via Ordinary Least Squares

Use of regression residuals to characterise relative distances between observations in the

sample

Corrects the regression line by subtracting the largest negative residual (for a cost function)

from the OLS fit (shift the regression line to (unique) best-practice observation)

Measures the relative inefficiency of all other companies (points) from the line passing

through the largest negative residual (the most efficient company)

Allows to assess the significance of each network cost driver

No measurement of stochastic errors

Requires large data volume in order to create a robust regression relationship

Very dependent on data quality and, in particular, sensitive to outliers (the company

defining the frontier could just be an outlier!)


11/01/2010 21


Stochastic Frontier Analysis (SFA)

Uses same premises as COLS, but treats best practice as a “stochastic” process (a mix of

true efficiency and “random noise” effects)

Several statistical assumptions behind the errors

SFA requires a large sample size to be statistically relevant

In the presence of patchy and/or too small samples, COLS is relatively more reliable than

SFA (SFA cannot be drawn as a “frontier” line as COLS)

Less sensitive to inputs and/or outputs as DEA / COLS

Allows to assess the significance of each network cost driver

Considers stochastic errors explicitly

Complex and statistically demanding

Requires large data sets in order to create a robust regression relationship

Genuine inefficiency could be allocated to stochastic elements: scores might be too

generous (too high)


11/01/2010 22


Virtual network models

Artificially construct an efficient (engineering-designed) reference network according to

commonly accepted planning principles and taking into account technical and geographical

constraints

The regulated firm’s relative (in)efficiency is estimated by the firm’s performance in relation

to the virtual network

Virtual network models are not dependent on obtaining and analyzing data of “real”

companies

Does not require a significant set of comparable companies as benchmarks

Very complicated and difficult to specify

Model sensitive to changes in inputs

Reasons for the deviation from reference network might be beyond control of the company


11/01/2010 23


TOTEX versus OPEX benchmarking

Building Block Approach

Implemented as linked (coupled) cap regulation

Explicit projection of capex for the upcoming regulatory period

Separate checks and inclusion of investments

Sometimes formalised efficiency analysis based on controllable opex

• TOTEX Approach

– Implemented as unlinked (decoupled) cap or yardstick regulation

– Inclusion of (historic) capital cost into efficiency assessment modelling (total cost

analysis)

– Standardisation of capital costs for benchmarking purposes

– (Planned) investment for the regulatory period not taken into account for the annual

allowed revenue


Building blocks (UK, Australia, Central and Eastern Europe) supported by:

– Efficiency carry over schemes

– Sliding scale schemes

Total cost approach (Germany, Norway, the Netherlands, Austria) supported by:

– Quantity terms (pre-specified cost drivers) incorporated in price control formulas

– Explicit investment allowances

– Inefficiency caps

TOTEX versus OPEX benchmarking

24



11/01/2010 25


Efficiency convergence speed

Allowed revenue

Initial level Initial one-off cut

1 2 3 4 5 Regulatory period

Proportional decrease

The X-factor prescribes the rate of change in the company’s prices or revenues, reflecting

the expected transition from the existing price level towards the efficient price level

Regulator to decide whether existing price level serve as starting point for the regulatory

formula or whether one-off cut of the initial price

Advantage of initial one-off cut,

prices can be brought to more

realistic levels at once

Large one-off adjustments quickly

eliminate inefficiencies at the

beginning, but decrease incentives

for further efficiency improvements

by the company


11/01/2010 26


Supporting Schemes

Inefficiency caps (Austria, Germany)

– Germany: Minimum (individual)

efficiency score – 60 %

– Austria: Max. (individual) efficiency

increase – 5.45 %

Sliding scale (Norway, 1997-2001 and

2002-2006)

– Base return with dead band plus

caps/ collars

Efficiency bands (Norway, 1997-2001)

Germany

(2009-2013)

Norway

(1997-2001)

KA b,0

Year 1 Year 10

KA dnb,t Permanently

Controllable

costs (base year)

KA vnb,0 Temporary non-

controllable costs

Max. 60% of total costs after deducting of

(permanently) non-controllable cost

(proportionally over 10 years)

-

non-controllable costs

-

15%

Profit floor level(tariff increase)

Profit cap level(tariff reduction))

Dead band

8,3%

2 %Profit floor level(tariff increase)

Profit cap level(tariff reduction))

Dead band


Summary

There are several benchmarking techniques and no consensus amongst regulators as to

which methodology is best

Data quality and model specification are fundamental for successful and defensible

outcomes

Benchmarking is an indication and not a confirmation of efficiency position

Integration of benchmarking results should take into account its imperfections and the

specifics of the price control design

Experience you can trust.http://www.leonardo-energy.org/training-module-electricity-market-regulation-session-6

End of webinar 6

KEMA Consulting GmbH

Kurt-Schumacher-Str. 8, 53113 Bonn

Tel. +49 (228) 44 690 00Fax +49 (228) 44 690 99

Dr. Konstantin Petrov

Managing Consultant

Mobil +49 173 515 1946 E-mail: [email protected]

electricity markets regulation - lesson 6 - efficiency assessments

Technology