approach paper ebrd energy sector strategy (2014- 2018 ... · 1.3 scope the ess covers the...

APPROACH PAPER

EBRD Energy Sector Strategy (2014-2018) - EvD Interim Review

September 2017

EBRD EVALUATION DEPARTMENT

EvD Approach Paper for the Interim Review of the EBRD Energy Sector Strategy (2014-2018) 1

The Evaluation department (EvD) at the EBRD reports directly to the Board of Directors, and is independent from the Bank’s Management. This independence ensures that EvD can perform two critical functions, reinforcing institutional accountability for the achievement of results; and, providing objective analysis and relevant findings to inform operational choices and to improve performance over time. EvD evaluates the performance of the Bank’s completed projects and programmes relative to objectives.

EvD’s Special Studies review and evaluate Bank activities at a thematic or sectorial level. They seek to provide an objective assessment of performance, often over time and across multiple operations, and to extract insights from experience that can contribute to improved operational outcomes and institutional performance.

This approach paper has been prepared by EvD and is circulated under the authority of the Chief Evaluator. The views expressed herein do not necessarily reflect those of EBRD Management or its Board of Directors. Responsible members of the relevant Operations teams were invited to comment on it prior to internal publication. Any comments received will have been considered and incorporated at the discretion of EvD. Whilst EvD considers Management’s views in preparing its evaluations, it makes the final decisions about the content of its approach papers.

Nothing in this document shall be construed as a waiver, renunciation or modification by the EBRD of any immunities, privileges and exemptions of the EBRD accorded under the Agreement Establishing the European Bank for Reconstruction for Development, international convention or any applicable law.

Under the supervision of the Chief Evaluator, Joe Eichenberger, this Approach Paper was prepared by Chiara Bocci, Senior Evaluation Manager, supported by consultancy services provided by Climate & Energy Solutions. Inputs have been provided by Evaluation Department staff: Barry Kolodkin, Regina Husakova, Saeed Ibrahim, Alejandra Palma and Stephanie Crossley.

The Approach Paper was prepared in coordination and collaboration with EBRD Management focal points in the EBRD Energy Business Group.

© European Bank for Reconstruction and Development, 2014

One Exchange Square

London EC2A 2JN

United Kingdom

Web site: www.ebrd.com

All rights reserved. No part of this publication may be reproduced or transmitted in any form or by any means, including photocopying and recording, without the written permission of the copyright holder. Such written permission must also be obtained before any part of this publication is stored in a retrieval system of any nature.


Contents

1. Introduction ...................................................................................................................... 4

1.1 Objectives 4

1.2 Rationale for inclusion in the work programme 4

1.3 Scope 4

2. EBRD Energy Sector Strategy (2014-2018).................................................................... 5

2.1 Key elements 5

2.2 Theory of change 7

2.3 Portfolio analysis 9

3. Approach and process .................................................................................................. 12

4. Evaluation questions ..................................................................................................... 14

5. Data collection tools and sources ................................................................................ 15

6. Challenges and limitations ........................................................................................... 17

7. Administrative arrangements ....................................................................................... 18

7.1 EvD team 18

7.2 Peer review 18

7.3 Management review 18

7.4 Timetable 19

7.5 Budget 19

Annex 1 Evaluation matrix ................................................................................................ 20

Annex 2 Portfolio analysis ................................................................................................ 24

Natural Resources 29

Power & Energy 33

Technical cooperation (TC) 39

Annex 3 Documents consulted ........................................................................................ 43

Annex 4 People met .......................................................................................................... 45


List of figures

Figure 1: EBRD’s Energy Sector Strategy (2014-2018) Theory of change – simplified version ................... 8

Figure 2: EBG NCBI per country (Dec 2013 – Apr 2017) ............................................................................. 9

Figure 3: Review approach ......................................................................................................................... 12

Figure 4: EBG Disbursements (Dec 2013 – Apr 2017) in € million ............................................................. 26

Figure 5: Net Cumulative Bank Investment EBG (Dec 2013 – Apr 2017) .................................................. 27

Figure 6: EBG NCBI per country (Dec 2013 – Apr 2017) in million € ......................................................... 28

Figure 7: Natural Resources – industry distribution by number of operations (Dec 2013 – Apr 2017)

.................................................................................................................................................................... 30

Figure 8: Natural Resources – industry distribution by volume (Dec 2013 – Apr 2017) .................... 30

Figure 9: Natural Resources - Net Cumulative Bank Investment and number of projects (Dec 2013 – Apr

2017) ........................................................................................................................................................... 31

Figure 10: Power & Energy – industry distribution by number of operations (Dec 2013 – Apr 2017)

.................................................................................................................................................................... 35

Figure 11: Power & Energy – industry distribution by volume (Dec 2013 – Apr 2017) ...................... 35

Figure 12: Power & Energy - Net Cumulative Bank Investment and number of projects (Dec 2013 – Apr

2017) ........................................................................................................................................................... 37

Figure 13: Donors for energy related technical cooperation activities marked as non-transactional (2013-

2017) ........................................................................................................................................................... 42

List of tables

Table 1: EBG Financial Frameworks approved by the EBRD Board (Dec 2013-Apr 2017) in chronological

order............................................................................................................................................................ 24

Table 2: EBG NCBI and operations (Dec 2013-Apr 2017) ......................................................................... 25

Table 3: EBG portfolio class (Dec 2013-Apr 2017) ..................................................................................... 25

Table 4: EBG disbursements (Dec 2013-Apr 2017) ................................................................................... 26

Table 5: Natural Resources – country distribution per industry (Dec 2013-Apr 2017) ................................ 29

Table 6: Natural Resources – geographic distribution (Dec 2013-Apr 2017) ............................................. 30

Table 7: Natural Resources – Transition Impact at approval (Dec 2013-Apr 2017) ................................... 31

Table 8: Natural Resources – SEI /GET and Environmental Category (Dec 2013-Apr 2017) .................... 32

Table 9: Power & Energy – country distribution per industry (Dec 2013-Apr 2017) ................................... 33

Table 10: Power & Energy – geographic distribution (Dec 2013-Apr 2017) ............................................... 36

Table 11: Power & Energy – Transition Impact at approval (Dec 2013-Apr 2017) ..................................... 38

Table 12: Power & Energy – SEI and Environmental Category (Dec 2013-Apr 2017) ............................... 38

Table 13: Natural Resources operations – transactional TCs per country ................................................. 39

Table 14: Natural Resources operations – donor distribution of transactional TCs .................................... 39

Table 15: Power & Energy operations – transactional TCs per country ..................................................... 39

Table 16: Power & Energy operations – donor distribution of transactional TCs........................................ 40

Table 17: Energy related technical cooperation activities marked as non-transactional (2013-2017) ........ 41


Abbreviations

DFI Development Finance Institution

EBRD European Bank for Reconstruction and Development

EBG Energy Business Group

E2C2 Energy Efficiency and Climate Change (team)

EPG Economics, Policy and Governance

EQ Evaluation Question

ERCCA Energy, Russia, Caucasus & Central Asia

ESD Environment and Sustainability Department

ESS Energy Sector Strategy

EvD Evaluation Department

GET Green Economy Transition

LTT Legal Transition Team

NCBI Net Cumulative Bank Investment

NR Natural Resources

OECD DAC Organisation for Economic Co-operation and Development - Development Co-operation Directorate

OPI Operation Performance Indicator

RO Resident Office

SEI Sustainable Energy Initiative

SEMED South Eastern MEDiterranean

SRI Sustainable Resources Initiative

TC Technical Cooperation

ToC Theory of Change


1. Introduction

1.1 Objectives

This Approach Paper sets out the focus, scope and methodological approach proposed by the EBRD’s

Evaluation Department (EvD) for the Interim Review of the EBRD Energy Sector Strategy (ESS)1.

Consistent with the Operations Manual Management is invited to comment and provide guidance.

The general objectives of the Review are:

To provide the EBRD Board and Management with an independent, evidence-based

assessment of the Bank’s current Energy Sector Strategy;

To identify findings and where appropriate make recommendations useful to Management and

the Board in drafting the next Energy Sector Strategy due by the end of 2018.

1.2 Rationale for inclusion in the work programme

One of EvD’s core strategic objectives indicated in the Evaluation Department Work Programme for 2017-

19 is to embark in sector strategy evaluations timed to feed analysis and evidence into

Board/Management reviews or updates. “The current Energy Sector Strategy dates from late-2013 and is

scheduled to be updated and approved by the EBRD Board by late-2018.” EvD’s review “will provide an

evidence-based strategic and operational assessment directly relevant to multiple initiatives (e.g., Green

Economy Transition) and country strategies, and to Board and Management review. The evaluation will

among other things encompass the operationalisation of strategic priorities, assess evidence of results

relative to objectives, and review issues related to execution and delivery of the strategy.”2

1.3 Scope

The ESS covers the time-frame 2014-2018 and sets EBRD’s general direction and identifies how it will

prioritise its activities in the sector. The ESS covers all the Bank’s activities in electricity generation,

transmission, distribution and supply and hydrocarbon extraction, processing, transportation, distribution

and supply. Hydrocarbons for this purpose include oil, gas and thermal coal.3 Thus this Review focuses

primarily on energy supply-side activities. End-use activities have been and/or are covered by the

Sustainable Energy Initiative (SEI), the Sustainable Resources Initiative (SRI), the Green Economy

Transition (GET), and other EBRD’s Strategies and Policies.4 Thus energy efficiency activities are only

covered if they relate to energy sector facilities, such as power plants, networks or refineries.

This Interim Review follows the standard OECD DAC evaluation principles and criteria. The evaluation will

focus on the following three questions:

i. Did the ESS set appropriate priorities for the EBRD?

ii. Did the EBRD adequately and consistently operationalise the ESS?

iii. What results have emerged from the ESS implementation so far?

1 As approved by the EBRD Board of Directors on 10 December 2013 2 As approved by the EBRD Board of Directors on 13 December 2016 3 page 6 4 Among others: the Mining Operations Policy; the Transport Strategy; the Municipal and Environmental Infrastructure Strategy.

http://www.ebrd.com/cs/Satellite?c=Content&cid=1395238499084&d=&pagename=EBRD%2FContent%2FDownloadDocumenthttp://www.ebrd.com/cs/Satellite?c=Content&cid=1395254811504&d=&pagename=EBRD%2FContent%2FDownloadDocumenthttp://www.ebrd.com/cs/Satellite?c=Content&cid=1395254811504&d=&pagename=EBRD%2FContent%2FDownloadDocument


In order to provide meaningful answer to those questions, the evaluation team has identified a number of

criteria and indicators – specified in Annex 1.

For the purpose of this review, EvD will cover the period running from approval of the ESS in December

2013 until April 2017.

In terms of activities implemented, EvD will consider projects (investments and technical assistance) and

other activities (such as policy dialogue) as implemented by the Energy Business Group (EBG) and its

three teams with support of other departments in EBRD HQ and resident offices (including among others:

Energy Efficiency and Climate Change – E2C2; Legal Transition Team – LTT; Environment and

Sustainability Department – ESD; Economics, Policy and Governance – EPG; etc.).

2. EBRD Energy Sector Strategy (2014-2018)

This section provides an overview of the key aspects of the EBRD Energy Sector Strategy. This section

also provides key figures of ESS implementation through a preliminary portfolio analysis.

2.1 Key elements

The 2013 EBRD Energy Sector Strategy replaced the EBRD Energy Operations Policy dated 2006. This

was in response to major developments occurring in the sector, and the new strategic directions set by the

Capital Resource Review 4 (2010-2015) which called for a “shift towards an energy efficient low carbon

economy supporting energy security and economic competitiveness”5 as well as the need to align energy

sector operations with the 2008 EBRD’s Environmental and Social Policy.6 The ESS was also drafted at a

time when the EBRD was starting reformulating its approach to results and shifting from a sector focus to

a country focus.7

EvD extracted the key elements of the ESS as presented in the document itself and approved by the

EBRD Board.

5 Capital Resource Review 4: 2011-2015 as approved by the Board of Directors at its Meeting on 23/24 March 2010, p. 38 6 as approved by the Board of Directors at its meeting on 12 May 2008 7 In July 2013 a document about Nomenclature was circulated to clarify the terminology used in Bank documentation including the definition of the terms strategy, policy, initiative and framework. Eventually in September 2014 the Board Financial and Operations Policies Committee discussed a document on The Architecture of Transition Impact Results Frameworks in the Bank.


Box: Key Elements of EBRD Energy Sector Strategy8

The ESS is built around energy efficiency and rooted in EBRD’s transition mandate and its

direction is set by the persistent transition challenges in its countries of operations.

The ESS emphasises the need to take a “holistic approach to energy systems”, and

“complements the Bank’s SEI which defines the Bank’s approach to the wider energy

systems” including energy use, albeit strictly from a climate perspective.

The ESS defines the Bank's role in the energy sector as “promoting the transition to the

policies, assets, institutions, actors and regulations that comprise a market oriented energy

sector, which will in turn deliver sustainable, secure and affordable energy services.”

The ESS “sets EBRD’s general direction and identifies how it will prioritise its activities in

the sector.” (…) “The detailed focus for each country and region is set, within the parameters

of the ESS, in the Country Strategy prepared for each country of operations.”

The ESS’s operational approach is organised around seven themes, rather than sub-sectors

(as in the previous Energy Operations Policy), namely:

o Energy Efficiency and Demand Side Measures;

o Building deep and liquid energy markets;

o Rethinking energy systems;

o Low carbon transition;

o Cleaner energy production and supply; fossil-fuelled generation;

o Setting standards and best practice;

o The wider role of the energy sector.

The ESS states that “the Bank will not finance any greenfield coal-fired power plant

except in rare circumstances, where there are no economically feasible alternative energy

sources.” In September 2014 the Bank circulated its methodology for the assessment of coal

fired generation projects.

The ESS specifies integrated approach as instrument enabling the Bank in some cases to

“draw together additional resources, technical assistance and policy dialogue coordinated with

a series of targeted investments to address specific transition gaps.”

As a monitoring framework, the ESS defines five Operation Performance Indicators (OPIs):

o OPI1: Private participation;

o OPI2: Cost reflective pricing;

o OPI3: Energy efficiency;

o OPI4: Carbon intensity;

o OPI5: Interconnections/energy trade.

8 All elements indicated in the box are quotes from the ESS itself.


2.2 Theory of change

Based on the key elements of the ESS as approved by the EBRD Board of Directors, EvD has elaborated

a theory of change (ToC) that was not included in the ESS documentation, but implicit in the paper. The

ToC reflects purely the view of the chain of results as indicated in the ESS itself – and it does not reflect

any EvD’s interpretation of it. In essence, it depicts the causal pathway that, according to the information

provided in the ESS, leads to achieving the expected short, medium and long term results. The ToC

elaborated by EvD has been shared with and validated by the Energy Business Group and will be used by

EvD as element for the analytical framework of this Review.

The ToC in its simplified version is presented in Figure 1 in the form of an expected impact diagram, which

differentiates between the levels of intended hierarchy of results (outputs – outcomes – impacts) and the

anticipated causal links between them.9 Correspondingly, the ToC has been used for the development of

the evaluation question and related judgement criteria focused at ESS effectiveness (achievement of

results) as specified in section 3.

The ToC also indicates at outcomes level where the ESS has strong links with SEI and its targets. Also, it

contains reference to the five OPIs identified in the ESS.

The ToC identifies five main outcomes for the ESS that are closely inter-related, namely:

A - Improved energy and resource efficiency / lower energy intensity on the supply and demand

sides

B - Improved environmental performance

C - Enhanced quality and security of supply

D - More open and better functioning markets

E - Improved transparency, governance, skills, and standards/practices.

9 An expanded version of the ToC has been developed by EvD. It identifies also risks and assumptions, and specifies in details all level of results.


Figure 1: EBRD’s Energy Sector Strategy (2014-2018) Theory of change – simplified version

Physical outputs

(i) In the power

sub-sector

(ii) In the oil and

gas sub-sectors

/ extractive

industries

(iii) In energy end-

use sectors

(outside the

energy sector)

(iv) In nuclear

safety

Policy and regulatory

changes completed

ImpactsOutcomesInputs

EBRD’s financing

resources (debt, equity,

etc.)

Donor funding for EBRD

technical assistance,

capacity building and

policy dialogue

EBRD’s staff expertise

Financing projects:

• Direct financing

(medium and large

operations)

• Indirect financing

via PFIs or direct

lending to MSMEs

(SBI) and

investment funds

• Coal-fired power

plant only in rare

circumstances

OutputsActivities

EBRD’s attributes i.e.

ability to foster and

support reforms

Policy dialogue and

cooperation to promote

energy market reforms,

cost-reflective tariffs,

setting standards and

best practice, etc.

Higher standards

and best practices

adopted

Secure, affordable

and sustainable

energy

Improved energy and resource

efficiency / lower energy intensity

on the supply and demand sides

T (SEI3) (OPI3)

More and better functioning

markets

(OPI1, OPI2, OPI5)

Improved environmental

performance

T (SEI3) (OPI4)

Equity and debt co-

financing resources of

private and public

sponsors and co-financiers

(including IFIs)

Mobilisation of co-

financing (syndication,

CIF, EU, etc.)

Transactional technical

assistance

Integrated approach

Enhanced quality and security of

supply

T (SEI3) (OPI5)

Improved transparency,

governance, skills, and

standards/practices


2.3 Portfolio analysis

The population taken into consideration for the purpose of this Interim Review is the portfolio of the

Energy Business Group (EBG), composed by three teams: Natural Resources, Power & Energy Utilities,

and Energy Russia Caucasus & Central Asia (ERCCA).

The timeframe under consideration is December 2013, when the ESS was approved, until April 2017.

For the purpose of the analysis of the population of investments, the evaluation team did consider all

operations approved by the Board and eventually signed in the timeframe under consideration. The

evaluation team did not consider investments classified as ore mining, as falling under the umbrella of the

EBRD Mining Operations Policy.

All information is specified in Annex 1 as directly derived from EBRD official databases. Key elements are

summarised below:

The Board has approved seven financial frameworks in different countries and regions

(Poland, SEMED, Turkey, Georgia, Kazakhstan, and Greece) for different purposes, mainly to

develop renewable energy projects. Of these, one was an Integrated Approach approved in

Poland about renewables. See Table 1.

A total of 85 operations have been approved by the EBRD Board of Directors and eventually

signed. One was cancelled after signing.

The total amount of Bank investment over the 84 operations is €4.5 billion, divided as €1.7

billion for the Natural Resources team (23 projects) and €2.8 billion for the Power & Energy

team (61 projects). See Table 2 and Figure 5.

This represents 8.9% in terms of number of projects, and 18.9% in terms of volume of the EBRD

investments in the same timeframe.

Only two investments have been structured as equity; the other 82 as loans (Table 2).

60 projects (71%) of the projects are classified as private, the remaining 24 (29%) as state

(Table 3).

Across the two sub-sectors of natural resources and power & energy, the countries that

received more investments are Turkey, Kazakhstan, Ukraine, Azerbaijan, Egypt, and

Jordan. However, as indicated in Figure 6, the composition of the portfolio is very different in

those countries and further specified in the description below.

Figure 2: EBG NCBI per country (Dec 2013 – Apr 2017)

Source: EvD elaboration on Tableau based on DWH data


Natural Resources (excluding ore mining projects)

23 projects for a total amount of €1.7 billion, accounting for 37.8% of the population.

The vast majority of projects are classified as private (19), only four as state - see Table 3.

In terms of industries (by volume) – see Table 5 Figure 7 and Figure 8:

o 36% to oil and gas extraction projects, mainly in Azerbaijan, Egypt and Kazakhstan.

o 27% to support activities for oil and gas operations mainly in Ukraine and Egypt.

o 17% to pipeline transformation of natural gas, mainly in Kazakhstan.

o 9% to petroleum refineries mainly in Turkey.

o 9% to pipeline transportation in Ukraine.

In terms of geographic distribution – see Table 6 and Figure 9:

o Eastern Europe and Caucasus account for 52% because of the investments in

Azerbaijan (two projects with one client) and Ukraine (two projects).

o Central Asia accounts for 19% because of the investments in Kazakhstan (two

projects).

o SEMED accounts for 13% because of the investments in Egypt (five projects).

o Turkey, counting also as a region, accounts for around 8% (two projects).

o Investments in Greece accounts for 5% (two projects).

In terms of transition impact, the projects were rated with good or strong good, with high risk

– see Table 7

12 projects out of 23 contribute to the SEI target - cleaner energy production (Table 8).

The majority of the projects were classified B, three only A, as environmental category (Table 8).

A total of €2.2 million have been committed with transactional grants (project preparation and

implementation) mainly devoted to support operations in Greece and Ukraine (see Table 13

and Table 14).

Power & Energy

61 projects for the total amount of €2.8 billion, accounting for 62.2% of the population.

33% of projects are classified as state (20), and 77% (41) as private - see Table 3.

In terms of industries – see Table 9, Figure 10 and Figure 11:10

o 49% of volume to alternative energy across 62% of the total number of operations

(38 projects).

o 27% of volume to electric power generation across 13% of number of operations

(eight projects).

o 14% of volume to electric power distribution in a same number of eight projects,

i.e. 13% of the number of operations.

o 8% of volume to electric power transmission, control and distribution across six

projects, i.e. 10% of the number of operations.

o 2% of volume to natural gas distribution in one project, i.e. 2% of the number of

operations.

In terms of geographic distribution – see Table 10 and Figure 12:

o South-Eastern Europe accounts for 29% with investments mainly in Albania,

Bulgaria, Romania and Serbia.

10 In September 2017 the Energy Business Group re-classified the industry of nine operations from Electric Power Generation to Alternative Energy. The implications of this change will be taken into consideration, if any, in the Review itself.


o SEMED accounts for 20% because of the investments in Egypt (one project) and

Jordan (eight projects).

o Turkey, counting also as a region, accounts for around 20% (seven projects).

o Central Asia accounts for 15% because of the investments in Kazakhstan (six

projects).

o Investments in Central Europe (6%) are concentrated in Poland (four projects) where

the Integrated Approach for renewables has been implemented.

o Eastern Europe and Caucasus accounts for 6% with investments in Georgia (five

projects) and Ukraine (four projects).

In terms of transition impact, four have been rated excellent, 17 strong good and 30 good,

with high risk (Table 11).

58 projects out of 61 contribute to the SEI targets – corporate energy efficiency, cleaner

energy production, or renewable energy (Table 12).

48 projects were classified B, 13 A as environmental category (Table 12).

A total of €22.3 million have been committed with transactional grants (project preparation

and implementation) mainly devoted to support operations in Tajikistan, Kazakhstan, Albania,

Egypt, Kosovo, Morocco, Tunisia and Kyrgyz Republic (see Table 15). The main donor is

the EBRD Shareholder Special Fund, followed by Austria, Italy, SEMED Multi-Donor, Climate

Investment Fund - CIF, Japan (see Table 16)

A total of €48 million of investment co-financing (via CIF and Global Environment Facility -

GEF) was provided to support seven operations in Georgia, Kazakhstan, Morocco, Mongolia,

and Ukraine

Policy dialogue

From the information gathered by the evaluation team, donor funded activities aimed at the broader

category of policy dialogue (including: capacity building, institution building, legal and regulatory

reforms, research) have been initiated and implemented on broad energy related topics by EBG, E2C2,

LTT, ESD, etc. Key data are provided below and will be further analysed by the evaluation team during

the ESS Review – see Table 17 and Figure 13:

A total of €17 million for 77 assignments (see Table 17)

Policy dialogue activities took place mainly in Egypt, Kazakhstan, Turkey (see Figure 13)

Activities included support activities to the Regulators, assistance to relevant Ministries on

energy related legislation and action plans, transparency in extractive industries, and broader

research activities.

Additional policy dialogue activities carried out by the Bank in the sector and not funded by donors will

complement the data outlined above and in Table 17 and Figure 13. Information about those activities will

be collected during the Review as provided by the teams involved – as not captured systematically in any

EBRD system.


3. Approach and process

This Interim Review follows the standard OECD DAC evaluation principles, and addresses its subject

through selected aspects of the standard evaluation criteria (in particular relevance, effectiveness,

efficiency).

The review process adopts a systematic approach, through which different tools and data collection

methods are used to gradually construct answers to evaluation questions and to formulate conclusions

and recommendations. The process is structured along three main phases as outlined in Figure 3.

Figure 3: Review approach

A. Development of analytical framework

In this phase, the outcome of which is represented by this Approach Paper, the analytical framework

(evaluation matrix) for the Review is developed. The key building blocks underpinning the selection of key

issues to be included in the evaluation matrix are the understanding of the strategic objectives of the ESS,

its actual implementation, and context.

The objectives and key elements of the ESS as approved in December 2013 are described in section 2

and schematically represented via the theory of change (Figure 1).

EvD has preliminary reviewed the actual implementation of the ESS through the analysis of the portfolio

data of the EBG and the TCs as well as policy dialogue activities – as harvested during this Approach

Paper phase. The summary of the portfolio analysis is presented in section 2.3 and detailed in Annex 1.

The portfolio analysis facilitated among others the preliminary selection for countries to be visited and

areas of interest (see section 5).

The evaluation team explored the key context of the ESS, both internally within EBRD and externally in its

interaction with stakeholders, to be able to identify the main issues of interest for the Review. This was

pursued through preliminary document analysis (see overview in Annex 3), and through first exploratory

Theory of

change

Context

overview

Portfolio analysis

Evaluation matrix= Evaluation Questions,

Judgment criteria, Indicators

Conceptualisation of the evaluation framework

Structuring of evaluation questions

B. Data collection and analysis

Strategy-

level

documents

Country and

programme

level

documents

Context

documents

Stakeholders

’ views

Facts,

Findings

Analysis,

Evaluative

judgments

C. Synthesis (output = ESS

Review Report)

Answers to EQs

Conclusions

Recommendations

A. Development of analytical framework (output = Approach Paper)

Portfolio

analysis

TC MIS data


interviews with key stakeholders (see overview in Annex 4). This allowed further development of the

evaluation questions.

Based on these inputs the evaluation matrix of evaluation questions and sub-questions (judgment criteria

and indicators) was developed (see section 4 and Annex 1). It is intended to serve as the analytical

framework to guide and structure data collection through the various tools and data collection methods

(see below). While no significant revisions of the matrix are foreseen, the evaluation matrix might be

slightly amended during the subsequent phases of the Review to reflect new inputs substantive for

achieving the objective of the evaluation.

B. Data collection and analysis

Once the analytical framework is completed, data is collected by the evaluation team via the various tools

and methods for data collection (see section 5). Data collection through all tools is structured along the

evaluation matrix, to allow for the analysis of the data across the different tools and, to the extent possible,

for verification and cross-checking (triangulation) of data from different sources. The evaluation will

combine secondary (desk) sources with fresh data collection through interviews of relevant stakeholders

in the countries of operations, EBRD HQ, and the main donors’ HQs.

C. Synthesis

The synthesis phase is then devoted to constructing answers to the evaluation questions and formulating

conclusions and recommendations on the basis of the data collected throughout the process. The final

output of the synthesis will be a comprehensive report to be circulated to the audience of the evaluation.

The timing is specified in section 7.4.


4. Evaluation questions

The main evaluation questions and their underlying rationale are briefly outlined below and are to be read

together with challenges and limitations outlined in section 6. The full evaluation matrix, which will be used

to guide the collection and structuring of data, is presented in Annex 1.

The evaluation team has identified three evaluation questions to guide this Interim Review of the ESS

which broadly correspond to the OECD/DAC criteria of relevance, efficiency and effectiveness.

EQ1: Did the ESS set appropriate priorities for the EBRD?

In order to answer to this evaluation question, the evaluation team has identified a number of judgment

criteria associated with the theme of ‘relevance’. In particular, the evaluation team will assess whether at

the time of drafting the ESS the Bank: (i) correctly identified the challenges and needs of the sector in its

countries of operations; (ii) derived adequate priorities for its interventions from this analysis of challenges

and needs; (iii) adequate considered for the EBRD specific business model and unique value added

(“institutional additionality”) in the identification of priorities in the ESS; (iv) took proper account potential

for complementarity and cooperation with other players; (v) whether the Bank in doing all of this

incorporated the inputs from relevant stakeholders, including but not limited to host governments, industry

players, other DFIs and international organisations, and CSOs; and (vi) whether elements of internal

learning were derived by the Bank from the 2006 Energy Operations Policy and previous relevant sector

evaluations.

EQ2: Did the EBRD adequately and consistently operationalise the ESS?

The evaluation team will be focusing on whether the EBRD set up an adequate operational organisation

(broadly defined) and allocated sufficient and adequate resources in the ESS to enable its successful

implementation (eventually assessed in terms of emerging results via EQ3). Answering this evaluation

question could include an assessment of: (i) an adequate result framework for the ESS; (ii) adequate

operational approaches, including the use of grants; (iii) adequate alignment of subsequent Country

Strategies with the ESS (as the detailed focus for each country and region is indicated in the ESS to be

set in the Country Strategy prepared for each country of operations); (iv) the adoption of an internal

organisation, and allocation of resources (including human resources), and (v) identification of internal

processes enabling the Bank to achieve the results identified in the ESS.

EQ3: What results have emerged from the ESS implementation so far?

This evaluation question is directly related to actual EBRD operations approved, signed and implemented

since the ESS came into force at the end of 2013. Bearing in mind this is an Interim Review of an on-

going Strategy, the evaluation team will seek through this question to carry out a preliminary assessment

of the progress(es) that the Bank has made in reaching its intended results in the energy sector through

its investment operations, technical cooperation and policy dialogue activities. As explicitly indicated in the

Theory of Change, the ESS medium-term results could be conveniently summarised by five expected

outcomes – which are closely interlinked among them. The evaluation team will collect data to gauge

progress toward the emerging results related to the five outcomes. The evaluation team will also update

the five OPIs identified in the ESS – bearing in mind limitations in terms of attribution. Some measures

could only be partial as it will be based on a selection of countries and only elements of EBRD’s

contribution could be harvested. In light of the harvested findings, the evaluation team will seek to identify

the major factors that could influence the achievement of the expected results.


5. Data collection tools and sources

A series of tools will be used for collecting, structuring, processing and analysing data. Some have already

been used in the preparation of this Approach Paper (such as preliminary review of documents, initial

interviews, and portfolio analysis). This section presents the tools and collection methods foreseen for the

use in all the phases of the evaluation as outlined in section 3.

EvD will mix the collection methods as fits the purpose of the specific judgement criteria identified to

respond to the three evaluation questions (see the evaluation matrix in Annex 1).

Interviews

First exploratory interviews with key stakeholders have already been conducted in the course of the

preparation of this Approach paper (see Annex 4). Further interviews, structured in line with the evaluation

matrix, will be conducted both in EBRD HQ and in the countries selected for field visits. Some interviews

may be conducted via phone or videoconferencing as necessary. The team will seek to gather the views

in particular of the following key stakeholders (not exhaustive):

EBG in HQ and ROs

Heads of ROs

Representatives of other relevant EBRD teams, including: E2C2, EPG/OCE, ESD, Vice

Presidency Policy, Donor Co-financing, Civil Society Engagement Unit, Legal Transition Team,

Internal Audit, Corporate Strategy, Budget, Operational Strategy and Planning, Operational

Effectiveness and Efficiency, etc.

Representatives of EBRD shareholders and donors

EBG clients

Relevant counterparts/stakeholders at country level (including local and national authorities,

regulatory agencies, sector associations, civil society organisations, etc.)

Representatives of DFIs, bilateral donors, etc.

Country visits

Operationally, the ESS was meant to provide the parameters based on which the EBRD would set

priorities at country level. The evaluation team intends to visit selected countries of operations to

complement the desk work. The objective is to gather information to feed into the answers to evaluation

questions at the country level, and to explore the different contextual determinants of ESS implementation

and success. The rationale is that the Bank is predominantly seeking to address transition challenges at

country level. It is important to emphasise that the country visits will not be aimed to be stand-alone

evaluations of EBRD country strategies.

The evaluation team will agree with Management on two countries to visit from a short-list of four

selected on the basis of a number of criteria, namely: Georgia, Jordan, Kazakhstan, and Turkey. Selection

criteria included:

The number of projects (and volume) signed: critical mass is essential to achieve the

outcomes pursued by the Bank through the ESS and the projects and other activities that deliver

these;

Diversification of the energy sector portfolio in the country in terms of: private and public

sectors; Power & Energy and Natural Resources projects, range of sub-sectors spanned by the

projects, etc.;


Contribution to ESS expected outcomes, based on a preliminary assessment by the

evaluation team for the purpose of this Approach Paper of the likelihood that each signed

project will contribute to each of the five expected outcomes;

Transition Impact ratings at approval: a higher proportion of projects rated good or higher

denotes a high level of potential impact in the country and higher likelihood to achieve one or

more of the expected outcomes of the ESS;

The existence of substantial technical cooperation and policy dialogue-related

interventions;

Countries belonging to different regions characterised by different transition and generally

early and intermediate transition stages.

Logistical arrangements will be discussed and coordinated with RO teams in advance, and success of the

missions will depend to a large degree on the cooperation of the EBG colleagues in HQ and EBRD

Resident Offices staff, especially in terms of provision of contact details for local clients and other

stakeholders.

Specific themes of interest

To complement data collection and structuring along strict geographical lines, other aspects will be

investigated. Given the importance given by the ESS to themes (see section 1) the evaluation team also

consider the Bank interventions in the sector through cross-cutting or thematic lens. For instance, a

preliminary topic of interest will be the use of the so-called ‘Integrated Approach”, and the specific case

that is most advanced in implementation is the Integrated Approach to Polish Renewables.

Depending on resources implications, this Interim Review will consider possible implications on the

interpretation of the ESS in light of the on-going EBRD’s revision of the measurement of transition impact.

Other topics for thematic case studies could be added to the evaluation where they would help to answer

the evaluation questions and resources and time will allow.

Portfolio analysis

Preliminary analysis of the EBG portfolio (excluding ore mining projects) over the period December 2013-

April 2017 has already contributed to the preparation of the methodology (see Annex 1) and for the

preliminary selection of countries for field visit. During the Review, it might be further developed as

needed to contribute to further data collection. The same approach applies to the population of grants

available in the period under evaluation. Data analysis will also be used for selecting stakeholders to be

interviewed, in collaboration with the EBG and country teams.

Documentary analysis

Similarly, preliminary reading of key ESS-related and contextual documents has already been carried out

for the purpose of preparation of this Approach paper (see Annex 3). Further documents will be collected

and consulted to complement information relevant to the issues outlined in the evaluation matrix. This will

include but not be limited to the following documents:

EBRD Board-approved documents relating to activities in the energy sector, particularly but not

limited to the EBG and SEI/GET, including strategies, policies, programme documents, technical

cooperation and non-technical cooperation grants; minutes of EBRD Board of Directors meeting

and its committees discussions on energy related matters; Board information sessions; etc.

EBRD Country Strategies approved after December 2013, etc.

EBG projects-level documents, including TC documents and available reports, etc.

Strategic EBRD documents such as: CRRs, SCF, SIPs, etc.


Contextual documents and statistics relevant to EBRD countries of operations, including EBRD

Assessment of Transition Challenges as included in the annual Transition Reports, national

development strategies, relevant documents of OECD, DFIs, IEA and other international

organisations, etc.

Previous relevant evaluation work, including:

o 2011 Special Study on Power and Energy Sector Review,11never discussed at the EBRD

Audit Committee. Management comments were provided in the framework of the 2012

Follow-up of Evaluation Recommendations by Management12

o 2014 Special Study on EBRD’s Experience with Policy Dialogue in Ukraine, Case study on

Power & Energy and Case study on Energy Efficiency13

o 2015 Evaluation of the EBRD’s Sustainable Energy Finance Facilities (SEFFs)14

o 2016 Evaluation of Four Wind Energy Projects15

o 2016: Evaluation of EBRD’s Use of Subsidies16

o Sector related validations and reviews of Operations Performance Assessments.

Documents from other DFIs and relevant energy sector strategies and policies.

6. Challenges and limitations

A number of challenges and potential limitations for the study are acknowledged, which could affect to

various degrees the outcome of the evaluation.

Ownership of the ESS

Roles and responsibilities are not defined in the ESS. EvD has made the assumption that the EBG is the

main key responsible department although the ESS affects the entire EBRD business, as the energy

sector itself touches upon many projects directly or indirectly.

EBRD strategic planning context

Since the ESS was approved in December 2013, the EBRD internal context has developed. Changes

introduced are affecting the ESS and its implementation to an unknown extent which is going to be

verified by the evaluation team during this Interim Review.

Results contribution

The evaluation team will seek to identify and assess the contribution of EBRD to observed results, mainly

through qualitative analysis. The source of information may come from the EBRD itself, and from direct

interviews carried out as part of the evaluation with relevant stakeholders. In addition, views and opinions

will be collected as a part of direct interviews in the countries visited.

Availability and quality of available information

Experience from past EvD Special Studies shows that the necessary documentation may not always be

fully available to the evaluation team. In addition, Special Studies of large programmes rely to some extent

on available monitoring and evaluation data and their quality and comprehensiveness. Lastly, the

availability of EBRD current clients for interviews and their willingness to share information with the

evaluation team may affect the data collection, as may the availability of other counterparts and

stakeholders.

11 Circulated in November 2011 12 As approved by the EBRD Board of Directors on 27 November 2012 13 Discussed at the Audit Committee on 6 May 2014 14 Discussed at the Audit Committee on 25 January 2016 15 Discussed at the Audit Committee at its meeting on 25 February 2016 16 Discussed at the Audit Committee on 7 November 2016


Staff turnover

Related to the previous issue is the challenge of staff turnover and institutional memory. It is a common

occurrence that key staff changes both internally in EBRD, and externally within local stakeholders and

counterparts. This may hinder the collection of data of older date, as institutional memory may not be

preserved.

7. Administrative arrangements

7.1 EvD team

The evaluation team is composed of Ms Chiara Bocci, EvD Senior Evaluation Manager. As appropriate,

specific contributions may be made by other EvD colleagues, in particular for specific data collection in

countries of operations if efficient. The evaluation team will benefit from the services provided by a

consultancy team – to be contracted based on the needs of this Review on specific aspects.

7.2 Peer review

The evaluation team will select one external peer reviewer. The expert will provide an independent review

of the final draft of the study with respect to its quality, in particular soundness of the analysis, appropriate

evidence base and presentation of findings, links between findings, conclusions and recommendations,

etc.

7.3 Management review

The draft Approach Paper has been circulated to the designated counterparts in Management as per the

Operations Manual 8.5, namely to the two Focal Points (Director Strategy & Policy Coordination in VP

Policy, and Managing Director Portfolio Business Group) who co-ordinated with the relevant

departments/teams for circulation of the draft Approach Paper and shared consolidated comments with

EvD. Management Focal Points and Reviewers for the study include:

Focal Points:

Managing Director, Banking Portfolio

Director, Country Strategy Coordination and Results Management

Management Reviewers:

Managing Director, Energy & Natural Resources

Director, Head of Energy, Russia, Caucasus & Central Asia

Director, Head of Natural Resources

Director, Head of Power&Energy Utilities

Director, Strategy, Energy & Natural Resources

Managing Director, OSP, Energy Efficiency & Climate Change

Director, Head of Energy Efficiency Climate Change

Managing Director, Environment & Sustainability

Managing Director, Economics Policy & Governance

Director, Sector Economics & Policy

Director, Legal Transition

This list may be added to during the course of the evaluation.


7.4 Timetable

Milestone Date

Study starts April 2017

Draft Approach paper circulated to Management July 2017

Approach paper approved August 2017

Field visits September/October 2017

Draft circulated to internal peer reviewers November 2017

Draft cleared by Chief Evaluator for circulation to external peer

reviewers

December 2017

Draft cleared by Chief Evaluator for Management Comments December 2017 / January 2018

Final approved by Chief Evaluator January 2018

Final distribution to Board February 2018

7.5 Budget

This study will require resources for EvD staff, consultancy team, travel costs and external consultant for

peer review, all within EvD approved budget for 2017-18.


Annex 1 Evaluation matrix

Evaluation questions Judgement criteria Indicators Sources of Data

1. Did the ESS set appropriate priorities for the EBRD?

1.1 Clarity of ESS’s identification of sector challenges (and broader context)

Existence and use of methodology to assess sector challenges aimed at drafting ESS

Peer-review process

ESS Preparatory analysis to ESS OCE/EPG documents External literature

1.2 Clarity of priorities/themes of intervention in the ESS derived from sector challenges and aligned with EBRD strategic planning documents

Coherence between sector/needs analysis and priorities

Prioritisation based on CRR4, SCF, SEI 3 and SRI and other strategic planning documents

Identification of specific methodologies to address specific challenges (e.g. related to coal)

ESS EBRD CRR4, SCF SEI 3, SRI Other strategic planning documents

1.3 Adequate consideration for EBRD specific business model and unique value added (“institutional additionality”) in the identification of priorities in the ESS

EBRD institutional attributes reviewed

ESS includes an ex-ante assessment on how EBRD’s interventions are likely to be additional

Risks

ESS Preparatory analysis to ESS

1.4 The ESS adequately identifies the potential for and implications of complementarity with other key players

Existence of mapping of other IFI/donors on-going or planned activities in the sector and indication of possible complementarity actions and activities


1.5 ESS incorporates adequately inputs of all relevant stakeholders

Existence of processes (such as specific meetings/hearings) for expression and/or contribution by stakeholders of their views

Actual incorporation in the ESS of points raised/inputs made by stakeholders during the consultation or otherwise

ESS and Report on the Invitation to Public to Comment Minutes and other documents from consultation meetings

1.6 ESS incorporates elements of internal learning

Findings and lessons from the previous Energy Operations Policy

Findings and lessons from previous relevant sector evaluations


2. Did the EBRD adequately and consistently operationalise the ESS?

2.1 Results of the ESS are captured in an adequate results framework to ensure inter alia the ESS is on track

Existence of explicit or implicit architecture of results linked to priorities

Existence of adequate OPIs and targets

ESS and underlying documents

2.2 The ESS identifies resources to implement the adequate operational approaches (e.g. integrated approach) vis a vis priorities

References to operational approaches per priority/theme of intervention

Identified integrated approach

ESS and underlying documents



Use of technical assistance, subsidies and concessional resources

Identification of resources in the ESS

2.3 EBRD Country Strategies objectives and targets are aligned with the ESS

EBRD Country Strategies incorporate sections about the energy sector with results and indicators aligned with ESS priorities and OPIs

28 Country Strategies approved after December 2013

2.4 Adequate resources and capacities (human, financial) have been allocated to ensure implementation of the ESS

Δ Bank’s headroom

Δ headcount in Energy Business Group and other departments implementing the ESS

Δ consultancy budget

Δ technical cooperation budget

Existence of action plans

Energy Group Scorecard Other internal EBRD documents Other IFI energy sector strategies/policies

2.5 An adequate internal organisation (team roles and responsibilities, processes, etc.) has been put in place in order to best implement the ESS

Management tools for cross-team collaboration processes and procedures

Processes and procedures between HQ and ROs

Accountability mechanisms towards the Board

Structured mechanism of learning and knowledge management

Internal EBRD available documents Other IFI energy sector strategies/policies

3. What results have emerged from the ESS implementation so far?

3.1 Progresses towards improved energy and resource efficiency / lower energy intensity on the supply and demand sides

Contribution of projects to ESS objectives and SEI as reflected by:

Δ in absolute energy consumption per capita (OPI 3)

Energy savings (Toe)

Addition of RE generation capacity

Δ of share of renewable energy in the energy mix

Δ of energy intensity (Toe/GDP)(OPI 3)

Δ of CO2 emissions per capita (OPI 4)

Δ of carbon intensity (MtCO2/ kwh)(OPI 4)

Progresses made by policy dialogue activities (donor funded, EBRD funded, and part of daily EBRD activities in HQ and countries of operations)

Sources utilised by the Bank for the initial calculation and compilation of the OPIs

External literature

International databases (World Bank, IRENA, IEA)

Project documentation (investments, TCs and policy dialogue)

Interviews and documents from EBRD colleagues

3.2 Progresses towards improved environmental performance


OPI 4 indicators above

GHG emissions reduction (Mt CO2-equ.)

Δ of operators adopting best practices and standards for nuclear electricity production and waste disposal (e.g. IAEA safety standards) (see 3.5)

Project documentation provided by EBG and E2C2 (investments, TCs and policy dialogue)



Progresses made by policy dialogue activities

3.3 Progresses towards enhanced quality and security of supply


Indicator of quality of electric power grid: o Availability (continuity of supply), o Technical properties (voltage quality) o Commercial quality (timeliness of services to customers)

Indicator of quality of gas power grid: o Supply (technical operational quality) o Composition (natural gas quality) o Commercial quality (timeliness of services to customers) o Loss reductions, collections rates improvements

Addition of RE generation capacity (see 3.1)

Δ of share of renewable energy in the energy mix (see 3.1)

Δ of renewable energy consumption (% of total final energy consumption)

Δ of cross-border energy trade: o Proportion of energy exports over total energy production (OPI 5) o Proportion of energy imports over total energy consumption (OPI

5) o Aggregate interconnection capacity (MW) (OPI 5, see 3.4)


International databases: CEER, IRENA, World Bank


Monitoring reports

3.4 Progresses towards more open and better functioning markets

Investment in energy with private participation (current US$)

% of domestic electricity generation under private sector control (OPI 1)

% of domestic gas production under private sector control (OPI 1)

% of domestic oil production under private sector control (OPI 1)

% of refining capacity under private sector control (OPI 1)

% of the national electricity network under private sector control (OPI 1)

% of the national gas transmission network under private sector control (OPI 1)

% of electricity consumers switching suppliers

Change in number of countries with operating exchanges

% of total electricity consumption served by private companies (OPI 1)

% of total gas consumption served by private companies (OPI 1)

International databases: World Bank

Project documentation

Policy dialogue documentation



Total gas pipeline interconnection capacity (in bcm/y) (OPI 5)

Number of electricity interconnectors (OPI 5)

Total oil pipeline interconnection capacity (Mtoe)

Δ of proportion of energy prices, weighted by consumption, that is either liberalised or, if regulated, at levels that do not imply any pre-tax subsidies (OPI 2)

Δ Number of countries with unbundled power and gas industries

Δ Number of countries with (deemed) independent power and gas regulators

Δ Number of countries providing transparent and non-discriminatory third party access to gas and power grids


3.5 Progresses towards improved transparency, governance, skills, and standards/practices

Contribution of projects to ESS objectives as reflected by:

Δ Number of operators (esp. in the oil and gas sector) having adopted transparency practices (e.g. EITI principles) in a country

Δ Number of companies having adopted CSR policies (re. social, environmental and governance practices) in a country

Δ Safer NPPs/ total number of NPPs in a country


International databases



Annex 2 Portfolio analysis

The population taken into consideration for the purpose of this Review is the portfolio of the Energy

Business Group (EBG), composed by three teams: Natural Resources, Power & Energy Utilities, and

Energy Russia Caucasus & Central Asia (ERCCA).

The timeframe under consideration is December 2013, when the ESS was approved, until April 2017.

For the purpose of the analysis of the population the evaluation team did consider all operations approved

by the Board and eventually signed in the timeframe under consideration. The evaluation team did not

consider investments classified as ore mining, as falling under the umbrella of the EBRD Mining

Operations Policy.

From December 2013 until April 2017, the EBRD Board of Directors has approved seven financial

frameworks, including one Integrated Approach, as listed below.

Table 1: EBG Financial Frameworks approved by the EBRD Board (Dec 2013-Apr 2017) in chronological order

Framework name

Board approval

Banking team

Short Description Country (Sub-) operations* approved

Integrated Approach to Polish Renewables

15/10/2014 Power & Energy

To address key transition challenges of the Polish renewable energy sector through policy dialogue and targeted outcome-

driven engagements and associated investments

into private sector renewable energy projects

and electricity grid investments

Poland 4

Genesis Framework


Bond Framework covering four vertically integrated

Polish energy groups

Poland 2 (cancelled)

SEMed Private Renewable Energy Framework


To provide finance to private developers of

renewable energy generation projects in

SEMed

Regional 1

PLUTO - Early stage geothermal support framework


Financing for early stage geothermal development

Turkey 2

Georgian Low Carbon Framework


Support the Government strategy to foster low

carbon generation, cover the country’s seasonal winter demand and to

support private ownership in the energy sector

Georgia 2

Kazakhstan 14/12/2016 Power & Finance the modernisation Kazakhsta 3


Framework name

Board approval

Banking team

Short Description Country (Sub-) operations* approved

Renewables Framework

Energy and strengthening of the grid in order to enhance

the integration of RES into the grid system

n

Greek Renewable Energy Framework


Policy dialogue and financing for renewable

energy projects

Greece 1

Source: EvD elaboration on DTM data

*Operations approved as part of Integrated Approach financial frameworks are classified as stand-alone

For the purpose of this analysis, the evaluation team has considered all approved investments and

analysed data available for the signed ones. 85 operations have been signed in the period under

consideration, and one signed was cancelled. Therefore the total population under consideration is

composed by 84 operations.

Table 2: EBG NCBI and operations (Dec 2013-Apr 2017)

Sector Team Net Cumulative Bank Investment (€ million)

% Number of operations

Average size (€ million)

Natural Resources 1,721 37.8% 23 75

Debt 1,721 37.8% 23 75

Power & Energy 2,831 62.2% 61 46

Debt 2,650 58.2% 59 45

Equity 180 4.0% 2 90

TOTAL 4,551 100% 84 54

Source: EvD elaboration on DWH data

In the same timeframe the EBRD approved 949 investments for a total amount of €24,128,913,558, thus

making count the EBG (exclusive of the ore mining projects) its 8.9% of it in terms of number of projects,

and 18.9% in terms of volume.

Table 3: EBG portfolio class (Dec 2013-Apr 2017)

Class Natural Resources Power & Energy TOTAL

Private 19 41 60

State 4 20 24

TOTAL 23 61 84



Table 4: EBG disbursements (Dec 2013-Apr 2017)

Class Disbursements

(€ million)

To be disbursed

(€ million)

NCBI

(€ million)

Signed 0 815 815

Disbursing 1,267 859 2,126

Repaying 1,494 104 1,598

Completed 12 0 12

TOTAL 2,774 1,778 4,551


Figure 4: EBG Disbursements (Dec 2013 – Apr 2017) in € million

Source: EvD elaboration based on DWH data


Figure 5: Net Cumulative Bank Investment EBG (Dec 2013 – Apr 2017)



Figure 6: EBG NCBI per country (Dec 2013 – Apr 2017) in million €



Natural Resources

The description of the 23 projects signed by the Natural Resources sector team is available in the tables

and graphs below.

Table 5: Natural Resources – country distribution per industry (Dec 2013-Apr 2017)

Industry Op # NCBI (€) % total volume

Average size (€)

Oil and Gas Extraction 8 613,581,326 35.7% 76,697,666

Azerbaijan 2 411,390,959 23.9%

Egypt 3 108,685,774 6.3%

Greece 1 18,284,043 1.1%

Kazakhstan 1 64,908,351 3.8%

Romania 1 10,312,200 0.6%

Support Activities for Oil & Gas Operations

5 464,139,050 27.0% 92,827,810

Bulgaria 1 10,000,000 0.6%

Egypt 2 111,313,251 6.5%

Greece 1 68,565,160 4.0%

Ukraine 1 274,260,639 15.9%

Pipeline Transportation of Natural Gas (inc storage)

2 297,340,441 17.3% 148,670,220

Kazakhstan 1 256,340,441 14.9%

Moldova 1 41,000,000 2.4%

Petroleum Refineries 2 157,370,442 9.1% 78,685,221

Estonia 1 20,240,122 1.2%

Turkey 1 137,130,320 8.0%

Pipeline Transportation (Ukraine) 1 150,000,000 8.7% 150,000,000

Gasoline Stations 4 33,746,152 2.0% 8,436,538

Georgia 2 25,140,559 1.5%

Kyrgyz Republic 1 1,325,593 0.1%

Turkey 1 7,280,000 0.4%

Coal Mining (Mongolia) 1 4,571,011 0.3% 4,571,011

TOTAL 23 1,720,748,421 100% 74,815,149



Figure 7: Natural Resources – industry distribution

by number of operations (Dec 2013 – Apr 2017)

Figure 8: Natural Resources – industry distribution by

volume (Dec 2013 – Apr 2017)


Table 6: Natural Resources – geographic distribution (Dec 2013-Apr 2017)

Region & Country Op # NCBI (€) % total volume

Eastern Europe and Caucasus 7 901,792,156 52.4%

Azerbaijan 2 411,390,959 23.9%

Georgia 2 25,140,559 1.5%

Moldova 1 41,000,000 2.4%

Ukraine 2 424,260,639 24.7%

Central Asia 4 327,145,396 19.0%

Kazakhstan 2 321,248,792 18.7%


Mongolia 1 4,571,011 0.3%

Southern and Eastern Mediterranean 5 219,999,025 12.8%

Egypt 5 219,999,025 12.8%

Turkey 2 144,410,320 8.4%

Turkey 2 144,410,320 8.4%

Cyprus and Greece 2 86,849,202 5.0%

Greece 2 86,849,202 5.0%

South-Eastern Europe 2 20,312,200 1.2%

Bulgaria 1 10,000,000 0.6%

Romania 1 10,312,200 0.6%

Central Europe and Baltics 1 20,240,122 1.2%

Estonia 1 20,240,122 1.2%

TOTAL 23 1,720,748,421 100.0%



Figure 9: Natural Resources - Net Cumulative Bank Investment and number of projects (Dec 2013 – Apr 2017)


Table 7: Natural Resources – Transition Impact at approval (Dec 2013-Apr 2017)

TI TI Risk

High Medium TOTAL

Excellent - - -

Strong Good 5 - 5

Good 10 1 11

Moderate Good - 1 1

Satisfactory - - -

Marginal - - -

SUB-TOTAL 15 2 17

N/A* - - 6

TOTAL 23


* 6 projects does not have TI rating because either are sub-operations in a Framework or because they are an extension of an existing operation


Table 8: Natural Resources – SEI /GET and Environmental Category (Dec 2013-Apr 2017)

SEI Category / Env Category A B C None TOTAL

3 – Cleaner Energy Production 2 10 - - 12

No SEI/GET 1 7 1 2 11

TOTAL 3 17 1 2 23



Power & Energy

The description of the 61 projects signed by the Power and Energy sector team is available in the tables

and graphs below.

In September 2017 the Energy Business Group re-classified the industry of nine operations from Electric

Power Generation to Alternative Energy. The implications of this change will be taken into consideration, if

any, in the Review itself.

Table 9: Power & Energy – country distribution per industry (Dec 2013-Apr 2017)

Industry Op #

NCBI (€) % of total volume

Average size (€)

Alternative Energy (geothermal, ocean wave, solar, wind)

38 1,385,140,411 48.9% 36,451,063

Cyprus 2 10,350,000 0.4% 5,175,000

Estonia 1 42,000,000 1.5% 42,000,000

Georgia 5 133,563,640 4.7% 26,712,728

Greece 1 50,000,000 1.8% 50,000,000

Jordan 7 188,698,491 6.7% 26,956,927

Kazakhstan 2 81,545,272 2.9% 40,772,636

Mongolia 1 22,855,053 0.8% 22,855,053

Montenegro 2 52,700,000 1.9% 26,350,000

Morocco 2 86,824,364 3.1% 43,412,182

Poland 4 167,501,425 5.9% 41,875,356

Romania 1 73,426,313 2.6% 73,426,313

Serbia 1 3,100,000 0.1% 3,100,000

Tajikistan 1 45,710,107 1.6% 45,710,107

Turkey 4 405,905,746 14.3% 101,476,436

Ukraine 4 20,960,000 0.7% 5,240,000

Electric Power Generation 8 770,640,036 27.2% 96,330,004

Albania 1 118,000,000 4.2% 118,000,000

Bulgaria 1 80,000,000 2.8% 80,000,000

Egypt 1 173,698,405 6.1% 173,698,405

Jordan 1 67,469,926 2.4% 67,469,926

Kazakhstan 2 105,663,555 3.7% 52,831,777

Serbia 1 200,000,000 7.1% 200,000,000

Turkey 1 25,808,150 0.9% 25,808,150

Electric Power Distribution 8 385,237,547 13.6% 48,154,693

Bosnia and Herzegovina 1 7,500,000 0.3% 7,500,000

Bulgaria 1 69,600,000 2.5% 69,600,000

Kazakhstan 1 13,077,467 0.5% 13,077,467


Industry Op #

NCBI (€) % of total volume

Average size (€)

Kyrgyz Republic 1 4,000,000 0.1% 4,000,000

Romania 2 137,180,203 4.8% 68,590,101

Turkey 2 153,879,877 5.4% 76,939,939

Electric Power Transmission, Control & Distribution

6 231,383,494 8.2% 38,563,916

Bosnia and Herzegovina 1 8,500,000 0.3% 8,500,000

FYR Macedonia 1 37,000,000 1.3% 37,000,000

Kosovo 1 30,000,000 1.1% 30,000,000

Romania 1 8,821,259 0.3% 8,821,259

Tajikistan 1 100,562,234 3.6% 100,562,234

Tunisia 1 46,500,000 1.6% 46,500,000

Natural Gas Distribution (Kazakhstan)

1 58,122,076 2.1% 58,122,076

TOTAL 61 2,830,523,563 100.0% 46,402,026



Figure 10: Power & Energy – industry distribution by

number of operations (Dec 2013 – Apr 2017)

Figure 11: Power & Energy – industry distribution by

volume (Dec 2013 – Apr 2017)



Table 10: Power & Energy – geographic distribution (Dec 2013-Apr 2017)

Region & Country Op # NCBI (€) %

South-Eastern Europe 15 825,827,775 29.2%

Albania 1 118,000,000 4.2%

Bosnia and Herzegovina 2 16,000,000 0.6%

Bulgaria 2 149,600,000 5.3%

FYR Macedonia 1 37,000,000 1.3%

Kosovo 1 30,000,000 1.1%

Montenegro 2 52,700,000 1.9%

Romania 4 219,427,775 7.8%

Serbia 2 203,100,000 7.2%

Southern and Eastern Mediterranean 12 563,191,186 19.9%

Egypt 1 173,698,405 6.1%

Jordan 8 256,168,417 9.1%

Morocco 2 86,824,364 3.1%

Tunisia 1 46,500,000 1.6%

Turkey 7 585,593,773 20.7%

Turkey 7 585,593,773 20.7%

Central Asia 10 431,535,764 15.2%

Kazakhstan 6 258,408,370 9.1%


Mongolia 1 22,855,053 0.8%

Tajikistan 2 146,272,341 5.2%

Central Europe and Baltics 5 209,501,425 7.4%

Estonia 1 42,000,000 1.5%

Poland 4 167,501,425 5.9%

Eastern Europe and Caucasus 9 154,523,640 5.5%

Georgia 5 133,563,640 4.7%

Ukraine 4 20,960,000 0.7%

Cyprus and Greece 3 60,350,000 2.1%

Cyprus 2 10,350,000 0.4%

Greece 1 50,000,000 1.8%

TOTAL 61 2,830,523,563 100.0%



Figure 12: Power & Energy - Net Cumulative Bank Investment and number of projects (Dec 2013 – Apr 2017)



Table 11: Power & Energy – Transition Impact at approval (Dec 2013-Apr 2017)

TI TI Risk

High Medium TOTAL

Excellent 4 1 5

Strong Good 17 - 17

Good 30 3 33

Moderate Good - - -

Satisfactory - - -

Marginal - - -

SUB-TOTAL 51 4 55

N/A* - - 6

TOTAL 61


* 6 projects does not have TI rating because either are sub-operations in a Framework or because they are an extension of an existing operation

Table 12: Power & Energy – SEI and Environmental Category (Dec 2013-Apr 2017)

SEI Category / Env Category A B C None TOTAL

1 - Corporate Energy Efficiency - 1 - - 1

3 – Cleaner Energy Production 3 15 - - 18

4 - Renewable Energy 10 29 - - 39

No SEI - 3 - - 3

TOTAL 13 48 - - 61



Technical cooperation (TC)

Transactional TCs

For the purpose of this Approach Paper the evaluation team has analysed the Board approval document

for each investment to understand if transactional technical cooperation resources were expected to be

used for the purpose of the achievement of the projects’ results. Transactional TCs include resources

marked mainly for: project preparation, project implementation support, and capacity building.

The evaluation team has considered only the transactional TCs linked to investments part of the

population under consideration (investments approved from December 2013 until April 2017). The

information provided below does not include all other TCs implemented by the EBRD in the period under

review, but related to investments approved before December 2013 or not approved yet.

Table 13: Natural Resources operations – transactional TCs per country

Country €

Greece 1,352,465

Ukraine 583,218

Regional 82,465

Mongolia 72,306

Egypt 56,195

Kosovo 17,060

TOTAL 2,163,709

Source: EBRD databases

Table 14: Natural Resources operations – donor distribution of transactional TCs

Donor €

Not available 1,585,000

Ukraine Multi Donor 292,918

EBRD SSF 157,290

Japan 57,131

EU 56,195

ETC Fund 15,175

TOTAL 2,163,709


Table 15: Power & Energy operations – transactional TCs per country

Country €

Tajikistan 7,577,486

Kazakhstan 2,785,922

Albania 2,705,640


Country €

Egypt 1,918,049

Kosovo 1,880,000

Morocco 1,595,000

Tunisia 1,274,000

Kyrgyz Republic 1,096,410

Turkey 397,598

Serbia 324,130

Georgia 276,790

Ukraine 212,999

Bosnia and Herzegovina 94,970

Jordan 66,320

Cyprus 47,500

Regional 11,400

TOTAL 22,264,214


Table 16: Power & Energy operations – donor distribution of transactional TCs

Donor €

EBRD SSF 8,188,118

Austria 5,476,212

Italy 2,197,640

SEMED Multi Donor 1,728,965

CIF 1,445,978

Japan 1,000,000

UK 998,920

EU 397,598

Norway 396,000

GEF 231,283

Not available 138,500

Germany 65,000

TOTAL 22,264,214



Non-transactional – policy dialogue

For the purpose of tacking stock of policy dialogue activities carried out in the framework of the ESS, the

evaluation team has considered data provided by EBG, E2C2, LTT, ESD, and EBRD TC databases. As in

2013-2014 the EBRD has modified the way in which TC information are captured, the evaluation team has

done its best effort to consolidate those information and assumes that what has been provided covers the

almost the entirety of activities carried out. A snapshot of the policy dialogue activities is provided below.

The evaluation team will harvest more during the implementation phase of the Review.

Additional policy dialogue activities carried out by the Bank in the sector and not funded by donors will

complement the data outlined above and in Table 17 and Figure 13. Information about those activities will

be collected during the Review as provided by the teams involved – as not captured systematically in any

EBRD system

Table 17: Energy related technical cooperation activities marked as non-transactional (2013-2017)

Country € committed Number of TCs

Egypt 4,577,360 10

Regional 2,605,015 9

Kazakhstan 1,248,083 5

Turkey 1,043,479 6

Montenegro 877,212 4

Kyrgyz Republic 770,000 1

Mongolia 765,000 5

Bulgaria 750,000 1

Tajikistan 679,952 3

Georgia 667,999 4

Morocco 530,000 4

Jordan 471,450 2

Ukraine 398,840 3

SEMED 350,100 2

Moldova 349,900 2

Romania 300,220 3

Bosnia and Herzegovina 249,280 2

Albania 210,860 1

Poland 199,575 3

Tunisia 194,990 2

Belarus 180,000 1

Kosovo 157,205 1

FYR Macedonia 150,000 1

Serbia 139,990 1

Croatia 61,652 1

TOTAL 17,928,162 77

Source: EvD elaboration based on EBRD documents and databases


Figure 13: Donors for energy related technical cooperation activities marked as non-transactional (2013-2017)

Source: EvD elaboration based on EBRD documents and databases


Annex 3 Documents consulted

Documents related to ESS approval process and implementation

Year Date Title

2013 - Summary of comments received on the existing Energy Operations Policy (2006) and the forthcoming Draft Energy Strategy in the period from November 2012 to April 2013

14 March Information Session: Energy Strategy - key issues

27 June Energy Sector Strategy

12 September Minutes of the Meeting of the Financial and Operations Policies Committee of 11 July 2013

15 November Energy Sector Strategy

15 November Energy Sector Strategy: report on the invitation to the public to comment

3 December Minutes of the Meeting of the Financial and Operations Policies Committee of 25 November 2013

3 December Energy Sector Strategy

4 December Report by the Chairman of the Financial and Operations Policies Committee on Energy Sector Strategy

10 December Energy Sector Strategy

2014 29 January Minutes of the Board Meeting of 10 December 2013

25 September Methodology for the assessment of coal fired generation projects

- Energy Sector Strategy Operational Performance Indicators

2016 07 April Energy Sector Strategy Update

1 August Minutes of the Meeting of the Financial and Operations Policies Committee of 14 April 2016

Relevant EBRD Strategic documents

Year Date Title