Evaluation Office

Evaluation for Environmentally Sustainable Development

Rob D. van den Berg

Shanghai, October 2007

Evaluation Office

What is the GEF?

The Global Environment Facility is an independent financial mechanism for multilateral environmental agreements that helps developing countries protect the global environment while promoting sustainable development. 32 countries contribute to the GEF trust fund.

GEF partnerships unite international organizations, governments, NGOs, scientists and the private sector.

177 countries are members of the GEF.

Evaluation Office

Projects funded by the GEF

Since 1991, the GEF has funded more than 2000 projects in over 151 countries.

Over $ 6 billion in GEF grants.>$20 billion in co-financing.

Evaluation Office

Global Environment Facility

Ultimate goal is to achieve global environmental benefits:– Reducing the risks of climate change– Stemming biodiversity loss– Preventing ozone depletion– Safeguarding international waters– Eliminating Persistent Organic Pollutants– Preventing land degradation

Evaluation Office

Evaluation Office

STAP

Donor Replenishment

Group

NGOs

Assembly

CBD

UNFCC

POPs

CCD

MultilateralFund of MontrealProtocol

InternationalWaters

IAs/EAs

UNDP

UNEP

WB

ADB

AfDB

EBRD

FAO

IDB

IFAD

UNIDO

Council

CEO/Chair

GEF Secretariat

Evaluation Office

GEF links to Multilateral Environmental Agreements

The designated financial mechanism for the– Convention on Biological Diversity (CBD)– Convention on Climate Change (UNFCCC)– POPs Convention

A designated financial mechanism for the– Convention on Combating Desertification

(UNCCD) The GEF collaborates closely with other treaties

and agreements to reach common goals (International Waters, Montreal Protocol)

Evaluation Office

GEF “Implementing Agencies”

World BankUNEPUNDP

Investment projects

Global regional/ and trans-boundary projects

Technical assistance/

capacity building projects

Evaluation Office

GEF “Executing Agencies”

FAO

UNIDO

IFAD

African Development Bank

Asian Development Bank

European Bank for Reconstruction and Development

Inter-American Development Bank

Evaluation Office

Key GovernmentDepartment

Other GovernmentDepartments

Stakeholders• NGOs

• Private Sector

• Civil Society

• Other Institutions

1 / 04

(Global Conventions)

Implementing Agencies

Political Focal PointOperational Focal Point

Executing Agencies

Evaluation Office

Six Focal Areas of the GEF

Biodiversity Climate Change International Waters Ozone Depletion (only countries in transition) Land Degradation Persistent Organic Pollutants – POPs

Evaluation Office

Total GEF $6,126.72Total Co-Financing $20,225.00TOTAL $26,351.72

$2,200

$2,055

$858

$516

$181

$156

$157

Biodiversity

Climate Change

International Waters

Multi-focal Area

Ozone

POPs

Land Degradation

Current GEF Portfolio (in millions of US dollars, January 2006)

Evaluation Office

Resource Allocation Framework (RAF)

In Climate Change and Biodiversity: 70% of the funds to go to individual countries – 25% to groups and 5% to Global and Regional Projects

Allocation based on past performance (WB index) and on potential for global environmental benefits

RAF will be extended to other focal areas in GEF-5 Mid-term evaluation of RAF upcoming Questions and problems noted:

– Slow start of RAF– Countries with low capacity will not be able to access GEF sufficiently– Indexes for benefits not sufficiently cognizant of vulnerability and of

marine resources

Evaluation Office

Unrealistic expectations

GEF does not have the means to directly influence global environmental problems

Example in Climate Change:– in order to change trends, reductions of 110-150

billion metric tons of GHG are needed per year– currently the GEF has contributed less than 1% over

12 years… In many focal areas, the GEF has a catalytic

role…

Evaluation Office

Less money in real terms

0

500

1,000

1,500

2,000

2,500

3,000

3,500

Replenishment New money In '94 dollars

GEF1

GEF2

GEF3

GEF4

Evaluation Office

GEF Evaluation Office

Evaluation Office

OverallPerformance

Study

Thematic andcross-cuttingevaluations,

impact assessments,country portfolio reviews

APR

Portfolio and program reviewsFocal area indicators

Annual PIR, Proj ects-at-risk systems,Supervision

Project indicatorsMonitoring

Mid-term evaluations

STAPadvice

enabling environment

oversight

M&E policy

COUNCI L

The M & E pyramid…

Evaluation Office

The GEF Evaluation Office

Mission statement:“Enhancing global environmental benefits

through excellence, independence and partnership in monitoring and evaluation”

We carry out our work with: ImpartialityProfessionalismTransparency

Evaluation Office

Five Evaluation Criteria 1 (2)

1. Relevance – the extent to which the activity is suited to local and national development priorities and organizational policies, including changes over time.

2. Effectiveness: The extent to which an objective has been achieved or how likely it is to be achieved.

3. Efficiency: The extent to which results have been delivered with the least costly resources possible. Also called cost-effectiveness or efficacy.

Evaluation Office

Five Evaluation Criteria 2 (2)

4. Results: The positive and negative, and foreseen and unforeseen, changes to and effects produced by a development intervention. In GEF terms, results include direct project outputs, short- to medium term outcomes, and longer-term impact including global environmental benefits, replication effects and other, local effects.

5. Sustainability: The likely ability of an intervention to continue to deliver benefits for an extended period of time after completion. Projects need to be environmentally as well as financially and socially sustainable.

Evaluation Office

Monitoring and Evaluation Policy

Purpose of M&E in the GEF: – Promote accountability: results, effectiveness,

processes and performance– Promote learning, feedback and knowledge

sharing as basis for decision making on all levels

The Policy is based on existing norms and practices for M&E; it ensures that M&E in the GEF aspires to the highest international standards

Evaluation Office

Minimum requirements

1. Project Design: All projects will include a concrete and fully budgeted M&E plan

2. Application of Project M&E: Project monitoring and supervision will include implementation of the M&E plan

3. Project Evaluation: Each Full Sized Project (and Medium Sized Project) will be evaluated at end of implementation

Evaluation Office

Use of evaluations

Evaluation Use Follow-up (MAR)

OPS3 Replenishment OPS4

Local benefits study Project preparations and oversight APR

Biosafety Better focus on countries needs New GEF biosafety strategy

CPE Costa Rica Better interaction with countries – CR: better RAF strategy

In RAF

Annual Performance Report

Improvements terminal evaluations, M&E planning and oversight

APR

Joint Activity Cycle Evaluation

Redesign of activity cycle Council in June

Incremental Costs Reformulation of ICA Council in June

Evaluation Office

GEF Evaluation OfficeWork Program 2007-10

2007-08

Capacity Development Evaluation

Annual Performance Report Annual Report on Impact RAF mid-term review 4 Country Portfolio

Evaluations in Africa Catalytic Role of the GEF GEF Focal Areas Evaluation of Partnership

and umbrella projects Small Grants Programme

2009-10

Country Portfolio Evaluations

Annual Performance Report

Annual Report on Impact Six Focal Area evaluations Fourth Overall

Performance Study

Evaluation Office

Development of results indicators

Results: outputs, outcomes, impacts Outputs: the products, capital goods and services

which result from a development intervention; may also include changes resulting from the intervention which are relevant to the achievement of outcomes

Outcomes: the likely or achieved short-term and medium-term effects of an intervention’s outputs

Impacts: positive and negative, primary and secondary long-term effects produced by a development intervention, directly or indirectly, intended or unintended.

Evaluation Office

Output and outcome

Output indicators are usually non-problematic and part of the regular monitoring, and final outputs are reported in the terminal evaluation

– Indicators should be “SMART” Outcomes: indicators should be chosen carefully to

measure achievement of desired outcomes– Focal area task forces have worked on “tracking tools” and

indicators to be used throughout their portfolios GEF Evaluation Office supports development of

indicators

Evaluation Office

GEF “SMART” performance indicators:

Specific: Capture the essence of the desired result by clearly and directly relating to achieving an objective, and only that objective.

Measurable: Specify indicators unambiguously so that all parties agree on what the system covers and there are practical ways to measure the indicators and results.

Achievable and Attributable: Identify what changes are anticipated as a result of the intervention and whether the result(s) are realistic. Attribution requires that changes in the targeted developmental issue can be linked to the intervention.

Relevant and Realistic: Establish levels of performance that are likely to be achieved in a practical manner, and that reflect the expectations of stakeholders.

Time-bound, Timely, Trackable, and Targeted: Track progress in a cost-effective manner at desired frequency for a set period, with clear identification of the particular stakeholder group to be impacted by the project or program.

Evaluation Office

Impact of the GEF

Impact of GEF interventions is usually beyond time horizon

Sometimes also beyond the physical boundaries of an intervention

Impact will have to be established in collaboration with others (UNEP, scientific community, WRI, IUCN, etc.)

But if no impact indicators are adopted, no collaboration will be possible and no impact can be attributed to the GEF when it finally occurs