ms. anna viggh senior evaluation officer 22 september 2015 evaluation of the gef civil society...

Ms. Anna VigghSenior Evaluation Officer22 September 2015

Evaluation of the GEF Civil Society Organizations

(CSO) Network

Introduction

• GEF IEO has begun consultations to inform the Evaluation of the GEF CSO Network.

• Today’s presentation includes a brief overview of: CSO Network History Purpose of the CSO Network Membership and Structure Evaluation Objectives Proposed Methods/Tools Identified Limitations Knowledge Management & Communication Plans Management of the Evaluation Time Frame

• Approach Paper was circulated with the GEF Partnership and is currently being finalized. Final will be posted on IEO website.

The GEF CSO NetworkHistory

• From its inception, the Pilot Phase, CSOs are acknowledged as critical stakeholders in the GEF.

• Formalized and systematic consultations became a part of the restructured GEF, including mention in the Instrument of the importance of CSO participation (Para 5) and ability to act as executors (Para 28).

• GEF restructuring resulted in a process of accreditation by the GEFSEC and semi-annual consultations amongst the Network and with Council, which are in still in place today.

• Any accredited NGO was thus automatically a member of a “GEF NGO Network”. Council replaced the accreditation with a membership system (2009) to be operated by the Network itself in a similar manner with member CSOs eligible to attend Consultations and Council/Assembly meetings

The GEF CSO NetworkPurpose

• Members of the Network are focused on influencing Council proceedings and facilitating participation of NGOs in the Council, NGO Consultations and Program implementation.

• The Network has developed a Vision, Mission and Objectives (v.2014):

Vision: “A dynamic civil society influencing policies and actions at all levels to safeguard the global environment and promote sustainable development.”Mission: “To strengthen civil society partnership with GEF by enhancing participation, contributing to policy and stimulating action.”

• Objective 1: To enhance the role of civil society in safeguarding the global environment

• Objective 2: To strengthen GEF Program implementation through enhanced partnership with civil society

• Objective 3: To strengthen the GEF NGO Network Capacity

The GEF CSO NetworkMembership

• The Network membership is currently comprised of 466 member CSOs. Of these:

189 are in the Africa Region representing 37 countries; 113 in Asia and the Pacific representing 32 countries; 79 in Europe representing 27 countries; and, 85 in the Americas representing 24 countries

Africa; 189; 41%

Asia; 113; 24%

Europe; 79; 17%

Americas; 85; 18%

CSO Network Membership

The GEF CSO NetworkStructure and Governance

• The structure of the Network has come about as a result of self-regulating initiatives,

Governance

OutreachStrategy

Management

GEF-Related Conventions

Central Focal Point

16 Regional Focal Points

3 Indigenous People’s

Representatives

Central Africa

Eastern Africa

Northern Africa

Southern Africa

Western Africa

Pacific

South Asia

North East Asia

South East Asia

West Asia

CaribbeanSouth

America

Mesoamerica

North America

East Europe and Central Asia

Europe

Asia and Pacific Africa

Americas

Coordination Committee

GEF CSO Network Evaluation (2005)

• Independent Review of the GEF NGO Network was presented to GEF Council in October 2005.

• Conclusions:– Model of NGO engagement at that time (both regional and country-level) was ineffective

– Network lacked a long term vision

– GEFSEC and Council had no long-term strategy for engaging the Network

– Insufficient Resources and a need for capacity building were major obstacles

– Secretariat, Council, Agencies and NGO community have a vested interest to re-energize the Network

• Recommendations:• Increase the network’s accountability and effectiveness by strengthening the network’s management, increasing

accountability in the application of the network’s Guidelines, re-focusing the accreditation process, and strengthening outreach to NGOs;

• Establishing an active partnership between the NGO Network and the GEF Secretariat and Council; and

• Providing support, financial and otherwise, to build the network’s capacity.

Evaluation Objectives

• Building on the 2005 review, this evaluation will answer the following key questions:

1. To what extent is the CSO Network meeting its intended goals and strategic objectives and adding value to the GEF Partnership and its membership?

2. How are Network features contributing to the effective and efficient functioning of the Network?

3. What are the implications for the next phase of the development and evolution of the CSO network?

Literature Review

• The increase in number and influence of CSO networks worldwide post the 1990s have allowed for their activities to be the subject of greater scrutiny.

• There is a growing body of literature on network formation, development, capacity building and evaluation. Evaluators have begun to develop frameworks for understanding networks using a mix of methods and tools:

• Pre-Evaluation Phase entailed beginnings of review of the literature on networks and their evaluation.

Key Questions (1)Key Evaluation Questions Network

Elements Example Evaluation Questions Information Sources Possible Approaches

Is the CSO Network meeting its intended goals and strategic objectives and adding value to the GEF partnership and its members?

Network Objectives as set by the GEF Council:i. Preparing for and reporting on

the GEF Council meetings and NGO Consultations to the wider CSO community at the national, regional & international levels

Network Objectives as set by the CSO Network:ii. To enhance the role of civil

society in safeguarding the global environment

iii. To strengthen global environmental policy development through enhanced partnership between Civil Society and the GEF

iv. To Strengthen the GEF NGO Network Capacity

Connectivity

What GEF-relevant information (knowledge products, presentations, reports, etc.) is flowing through the Network to its membership and other stakeholders?

- Network Documents- Data / Results from Surveys,

Interviews, and other primary sources (e.g. workshops)

- (Online) Self-Assessment- Interviews and Focus

Groups/Focused meetings with key stakeholders

- Cost / Level of Effort Overview Assessment

- Social Network Analysis

Credibility

Are the Network’s objectives still relevant? To what extent has the Network aligned

with GEF goals on gender mainstreaming and indigenous peoples’ inclusion?

Has the CSO Network contributed to shaping the GEF agenda (getting new issues on the GEF agenda, policies incorporated by the Council)?

- Council and GEF SEC Documents

- Network Documents- Non-GEF CSO Networks

- Document review - Interviews and Focus


- Surveys- (Online) Self-Assessment- Comparative analysis with

other networks

Capacity

How are Network members adding value to one another’s work, i.e. achieving more together than they could alone?

Are there clear signals of development of CSO/member capacity?

- Data / Results from Surveys, Interviews, and other primary sources

- Interviews and Focus Groups/Focused meetings with key stakeholders

- Surveys- (Online) Self-Assessment

Progress

towards

Results

Are there clear signals of influence on GEF policy and program implementation? Can a case be made as to Network contribution?

Has the Network membership monitored the implementation of GEF portfolios and policies at the country level?


- Network Documents

- Document review - Interviews and Focus Groups

with key stakeholders- Surveys- (Online) Self-Assessment

Key Questions (2)Key Evaluation Questions Network

Elements Example Evaluation Questions Information Sources Possible Approaches

How are the CSO Network’s features (governance, structure, membership, connectivity, etc.) contributing to its ability to meet its objectives

Connectivity

How effective and efficient are the connections the network makes?

Are all members contributing, individually or through joint efforts, to network goals?

- Network Documents- Data / Results from

Surveys, Interviews, and other primary sources (e.g. workshops)

- (Online) Self-Assessment- Interviews and Focus


- Cost / Level of Effort Overview Assessment

- Social Network Analysis

Membership

Who participates in the Network and why? Are women’s, indigenous peoples’ and youth organizations represented? Has the Network assembled member organizations with the capacities needed to meet Network goals (experience, skills, and connections)?

Is the process for Network membership transparent, effective, and efficient? Has it changed over time?

What is the geographic distribution of membership in relation to GEF operations?

What have been the trends in membership?



Surveys, Interviews, Focused meetings with key stakeholders and other primary sources

- Document review - Social Network Analysis- Surveys- Meta-Evaluations- Comparison to other

Networks- Visual Timeline

(infographic Representation)

Structure

Has the Network and GEF Partnership adjusted to meet changing GEF needs and priorities?

What infrastructure is in place for Network coordination and communications?

Are these coordination and communication structures efficient and effective?

Are lessons from similar networks (Adaptation Fund, GCF, CIF, etc.) used to inform the workings of the GEF CSO Network?

- Membership Databases- Council and GEF SEC

Documents- Network Documents

- (Online) Self-Assessment- Social Network Analysis- Visual Timeline

(infographic Representation)

- Document review

Key Questions (3)Key Evaluation

QuestionsNetwork Elements Example Evaluation Questions Information Sources Possible Approaches

How are the CSO Network’s features (governance, structure, membership, connectivity, etc.) contributing to its ability to meet its objectives

Governance

Are the Network’s governance rules applied in a transparent manner?

Is there a transparent conflict resolution process? Do Network members actively participate in Network

elections? Do decision-making processes encourage members to

contribute and collaborate? How dependent is the Network on a small number of

individuals? (male/female disaggregated) Do governance structures take into consideration gender

mainstreaming?



Surveys, Interviews, and other primary sources



- Surveys- (Online) Self-Assessment- Meta-Evaluations- Comparative analysis

with other networks

Resources

What is the level of financial and technical resources provided to the Network?

Has the Network secured needed material resources? Is the Network adapting its business plan over time? How has the GEF partnership [GEF SEC, Agencies, OFPs, IEO,

etc…] supported the work of the CSO network?


Surveys, Interviews, and other primary sources


Groups with key stakeholders

Capacity

Does the Network have the needed capacities to advance members’ skills & Network goals?

- Data / Results from Surveys, Interviews, and other primary sources

- Network Documents

- Interviews and Focus Groups/Focused meetings with key stakeholders

- Surveys- (Online) Self-Assessment

• Decisions on methods and tools will be made on the basis of issues identified.

• Evaluation activities will involve: Further document review Comparative analysis with other networks Meta-evaluation Select individual interviews with key stakeholders Focus group meetings with key stakeholders Survey (CSO Network members/non-members/GEFSEC/Agencies) Self-Assessment (Coordination Committee)

Possible Approaches

Additional Stakeholder Involvement

• Reference Group: • Includes representatives from the GEF Secretariat, GEF Agencies, the CSO

Network, and SGP. The Reference Group will: • Share comment on the Approach Paper and drafts of the Report; identify &

establish contact with individuals for interviews/focus groups and identify & facilitate access to information.

• 10 individuals

• Expert Peer Review Group:– Consists of relevant evaluation specialists from GEF Agency Evaluation

Office and the broader evaluation community. – 3 individuals.

Limitations

• Networks are inherently complex and dynamic systems which makes them difficult to evaluate.

• Main limitations of this evaluation will be: Access to entirety of CSO Network (over 460

members, different regions/access/languages) Access to older and archived documents of the CSO

Network Relatively short timeframe

Knowledge Management & Communications

• The CSO Network Evaluation will add to the growing body of literature on network evaluations and thus will be of broad interest to the evaluation community.

• The IEO will disseminate information gathered at various points in the course of the evaluation with stakeholders.

• Information sharing could take place through various platforms such as:– Online stakeholder groups– Webinars– Published knowledge products

Management of the Evaluation

• The evaluation will be task managed by Ms. Baljit Wadhwa, Senior Evaluation Officer, leading a team comprising of GEF IEO staff and consultants.

• Consultants are contracted to undertake specific elements which include peer review and analysis of data on the CSO membership, connectivity and network health though self assessments and/or social network analysis, etc.

Time Frame

Phase Evaluation Phase Time Frame

1 Pre-evaluation desk review, upstream consultations & Approach Paper End of June 2015

2 Further desk review; identification of data gaps; further methods selection; survey instrument July – August 2015

3Application of appropriate methods/tools for additional data gathering and analysisPeer Review & Reference Group Consultation

September 2015 – January 2016September 2015

4

Triangulation, verification, gap analysis and preparation of Evaluation ReportDraft Evaluation shared and discussed with Reference Group/ and stakeholders and edits finalized

January - April 15, 2016

5

Final Evaluation shared with GEF CouncilEvaluation Conclusions & Recommendation presented at GEF Council meetingKnowledge products and dissemination activities

May 2016June 2016May - September 2016

Questions and Feedback

We are looking for feedback on:• Survey Questions• Key Questions• Your Experience with the GEF CSO Network

Thank you

[email protected]