More Enabling and Less Controlling!
Rio De Janeiro, 6-7 November 2008
Lichia Saner-YiuCSEND, Geneva, www.csend.org
UNFCCC Meeting on Experiences with Performance Indicators for Monitoring and Evaluation of Capacity-Building in Developing Countries
Objectives of Presentation
To urgently draw attention to the need to review CB performance indicators in the context of an institutionalised M&E processes in the partner countries
To clarify that monitoring should not be “mini-evaluation” but rather a management tool!
To propose a monitoring tool which will support greater effectiveness of the CB projects/programme!
Field Experience
Capacity Building for the Public Administrative Reforms – China (1988-1997), Slovenia (1994-1996) etc.
Building Business Training Centres in Russia (1995-1997)
Building Negotiation Capacities in the International Arenas: UNFCCC, WTO, FTAA.. (2000-2003)
Strengthening management capacities of UN Agencies & INGOs (ILO, UNICEF, WHO, UNHCR, Action Aid …)
Thematic Training and short term consulting inputs for 20+ years
Continue to teach in different universities on topics related to institutional performance management and transformation
Contexts
Decision 2/CP.7 The Paris Declaration for Aid Effectiveness
(2005) The ACCRA Agenda for Action (2008) calling
for accelerated progress in areas of country ownership, inclusive partnerships and achieving development results. Specifically, “Developing countries will strengthen their capacity to lead and manage development”
Framework | Scope of priority areas (UNFCCC, 2007)
9. Research and systematic observation
10. Development and transfer of technology
11. Decision-making and participation in international negotiations
12. CDM13. Article 4.8 and 4.9 (LDC
Work Programme, NAPA)14. Education, training and
public awareness15. Information and networking
1. Institutional CB (National Focal Point)
2. Enabling environment3. National Communications4. National climate change
programmes5. GHG inventories6. Vulnerability and adaptation
assessment7. CB for implementation of
adaptation measures8. Assessment for
implementation of mitigation options
The Challenge of Capacity Development (CD)
“Capacity Development is a fundamental component of development and aid effectiveness and a key element in achieving the Millennium Development Goals” (OECD/DAC 2006).
In particular CD is critical to achieving objectives of ownership, aid alignment and mutual accountability, as set in the Paris Declaration on Aid Effectiveness (March, 2005).
However, the traditional approach to CD (as a technical process or transfer of knowledge or institutions from North to South) rarely leads to the expected results i.e. sustainable capacity within partner countries.
Increase in funding is not sufficient to address the challenge of capacity development. We need a fundamental change in development practice and the way that we look at the role and function of M&E.
The Challenge of Capacity Development (CD)
The need for Adequate CD
Focusing on capacity as an endogenous process (led but not exclusively by country actors) including M&E components.
Agreeing at country level on clear (institutional) capacity objectives.
Making effective use of existing capacities and harmonising external support within the MEA framework.
Monitoring outcomes from the perspective of beneficiaries (OECD/DAC Recommendations).
Enhancing institutional learning and knowledge creation through M&E processes for all development partners.
Parameters
Shortcomings of Current Monitoring Practices
It is more of controling than for enabling.
It relies more on self-reporting, than evidence and documented records.
It remains short in providing opportunities to the partners and external (non-national) stakeholders in taking charge of the CD process and in being held accountable for what has been achieved and what has not.
Shortcomings of Current Monitoring Practices -2
The process for the QA of TBCD remains exogenous rather than endogenous.
The learning remains at best temporal, at worst short-lived on the account of the partners since there is no mechanism to ensure institutionalisation of learning.
Monitoring remains as activities to be checked off, rather than an integral part of the day-to-day process and mangement system
Confusing Use of Terminology
Mixing terms of “evaluation”, “inspection”, “supervision” and “monitoring”
Monitoring in the parlance of the QM is about “ensuring the CB process is being managed and implemented as required so as to provide objective evidence that the CB process is effective in meeting requirements.”
Monitoring, therefore, should involve reviewing the entire CB process at each stage of the project cycle (from design to implementation) and not just mid-term review or end-of the pipe evaluation.
Evaluation About the past. Who remembers the W’s after 3
or 4 years of project duration? Lessons learnt for whom? Project partners
(beneficiaries, donors, experts) move on to new jobs
Very costly and very heavy for beneficiaries (evaluation tourism)
Against Paris Declaration (e.g.EIF: multiple evaluation methods, LDC overwhelmed with compliance duties)
OECD Survey on Monitoring and Paris Declaration (2008).
Les than 10% of countries have sound frameworks to monitor and assess development results. While some progress has been made since 2005, an enormous effort will be required to meet the target of 35% by 2010
Urgent Need to Invest Less evaluation, more monitoring! Change our mindset and approach
to monitoring Invest in a management information
system that monitors and enables
In order to accelerate the CB and the attainment of UNFCCC on "preventing dangerous anthropogenic
interference with Earth's climate system“ and MDG 7 –
Environmental Sustainability
Proposal by Engel, Land, Keijzer (2006)1
A balanced framework to define the term “Capacity Building” in clear terms (capability to produce development results, to self-organise and act, to adapt and self-renew, relate& create adequate operating space, and to achieve coherence/vision/ strategy)
An excellent framework to develop indicators for baseline studies, progress assessment and benchmark
Yet, there remains a missing link – an institutionalised management system and database to support the emergence of these essential capabilities through practice and to facilitate effective management decision making and self regulation through evidence and data
1. “A balanced approach to monitoring and evaluating capacity and performance”
CSEND Proposal
Capacity Building model
of Monitoring
The Three-Tiered Monitoring Framework at the global level
Tracking Aid for CB Flows
Donor Self-Assessment
National Self-Assessment
Quantitative Information
Qualitative Information
UNFCCC CBReview
CSEND’s Non Fiduciary AfT Monitoring Framework (within borders)
Tracking CB Implementation Process
Verification of Participating Donor In-Country Inputs
Verification of Executing Agency In-Country Inputs
Documentation of Input-Throughput- Output & Outcome data
Validation of AfT Processes
Knowledge Creation
Continuous Improve-ment
Based on Paris Decla-ration Principles
Verification of National Inputs
Project Cycle
2. Initiating & formulating
project proposal
3. Appraising &
approving project(at country and global levels)
4. Implementing
project
5. Evaluating project
outcome
Monitor
1. Defining strategic interests & needs
of a DC
CB programming
process(framing the
scope)
Global
National
What Is Needed for An Effective Monitoring System -
M&E protocol Monitoring methodology (focusing on
organisation, management and institutional collaboration)
Third party verification according to the agreed CB objectives and measure
A matrix structure and delineating levels of responsibility for monitoring
Institutional actors and roles Level of responsibilities regarding monitoring Periodicity of monitoring actions Inputs needed and from Outputs produced Monitoring criteria & indicators Records
Example of M&E System (EIF)
EIFActors
Level of Responsibilities regarding Monitoring
Periodicity
Inputs Outputs Monitoring Criteria & Indicators
Records
NSC Oversight on the Use of EIF resources for the country
Setting national policy regarding trade development and performance targets
Supervise the performance of National Implementation Organs and mechanisms
6 months National Trade Development Plan and Priorities
PRSP Country Strategy and Programme Priorities if existing
DTIS and Action Matrix
NIU quarterly implementation reports (both technical and financial)
NIU process audit reports
Country EIF programme, strategies, workplans
Semester Country EIF Implementation Review (Tier 1 & 2)
Annual Country EIF Programme Performance Report
Monitoring Criteria Compliance to
National EIF programme formulation procedure
Compliance to National EIF project approval procedure
Decisions on Tier 1 workplan and Tier 2 programme priorities
EIF Country Programme Monitoring Records
NSC audit records
Request for Corrective or Preventive actions
Corrective action report
It aligns CB with DC/LDC country development and ministerial performance objectives within the context of national strategy
It provides a process management tool for learning transfer, continuous improvement of capacity building processes and outcome (higher pay-off)
It gives ownership and accountability to the beneficiary countries and counterparts
It supports intermittent programme evaluation (monitoring means collecting in-depth data and rich information source)
It facilitates multilevel knowledge creation and knowledge management amongst donors, beneficiaries and experts.
Added Value of a CB based monitoring management system Safeguards against sub-optimal use of resources Ensures result-based accountability of the
capacity building function Documents quantifiable data for process
improvement Anchors CC agenda within recipient country’s
development agenda, planning and implementation processes
Knowledge management in the hands of ALL partners
Vision to Achieve
Capacity building should assist developing countries to build, develop, strengthen, enhance, and improve their capabilities to achieve the objective of the Convention through the implementation of the provisions of the Convention and the preparation for their effective participation in the Kyoto Protocol process (italic added).
Thank You !!