livestock and fish monitoring, evaluation and learning framework
DESCRIPTION
Presented by Keith Child at the Livestock and Fish Monitoring, Evaluation and Learning planning meeting, Nairobi, 27-28 November 2013TRANSCRIPT
Livestock and Fish monitoring, evaluation and learning framework
Keith Child
Livestock and Fish Monitoring, Evaluation and Learning planning meeting, Nairobi, 27-28 November 2013
Agenda
1. MEL Process and Framework Goals
2. Parts of the Framework
Purpose of the MEL Framework
A comprehensive, yet concise narrative of why the M&E system is important, how it operates, what kinds of data it will collect and by whom
• Provides a common vision of what an M&E system will look like
• Provides a benchmark for measuring progress
• Provides a mandate and set of responsibilities
What makes a good framework?
Clarity of Purpose: a ‘framework’ not an instruction manual;A Compelling Vision: how will we achieve success;Relevant: Appropriate to the CRPEconomical: Achievable at an affordable costConcision: as long as necessary, as short as possible;Professional: must look and read like a formal document.
Background
• November 2012: Incomplete rough draft
• June 2013: Senior Scientist (Impact and Learning)
• July-August: MEL Framework consultations
– 4 formal meetings, many informal consultations• Impasse: RBM vs. Hybrid visions of the MEL Framework
• September 2013: Plan to finalize Framework presented to PPMC
• November 2013: Presentation of draft Framework to MEL Meeting
Forward
• Post MEL Meeting revisions to transform draft into Working Paper
• December 2013: Presentation to PPMC in Tanzania
• Post PPMC Meeting revisions to transform Working Paper into Finalized Framework
• April 2014: Presentation to PPMC in Penang for Finalization
Proposed Content of MEL Framework
MEL Framework Introduction
• Introduction to Framework
• Background: CGIAR and Livestock and Fish
• Challenges
• Structure of MEL Framework
• Integrated Phases
• Attribution versus Contribution
Integrated Phases
Implementation Phases and Corresponding MEL Research Questions
Dev
elop
men
t Ph
ase
(100
,000
s be
nefic
iari
es)
Scal
ing-
Out
Ph
ase
(mill
ions
of
bene
ficia
ries
)
Rese
arch
Pha
se
(10,
000s
be
nefic
iari
es)
Ong
oing
Fu
nctio
ns
Years 12-20 (SLOs)Years 5-8 (Progress towards IDOS) Years 8-12 (IDO data collection)
What Works? Should it work?
How and Why did it work?For whom will it work?
What level of attribution can be claimed?
How to Scale-Up?Will it continue to work?
Will it work somewhere else?How has implementation contributed to the results?
Are Program benefits sustainable?
How to Scale-Out?Is the rationale for why it worked still sound?
Can outputs be transferred/generalized to different settings?
How have innovations been adapted to local contexts?
International Public Goods
Program Monitoring
Attribution versus Contribution
• Attribution: whether or not and how much of a particular change can be attributed to an intervention
• Contribution: whether or not and how an intervention contributes to the change
Body: 5 Components
1. Organizational Perfomance
2. Learning and Reflection
3. Outcome and Impact Monitoring
4. Knowledge Management
5. Research Agenda
Component One: Organizational Performance
• Evaluation
– CRP Independent External Evaluations– CRP Commissioned External Evaluations– Subtheme and Project Evaluations
• Routine Portfolio Monitoring
• Performance Indicator Matrix
Evaluation Strategy• Nested approach in which lower-level evaluations feed
into and inform higher level evaluations
Project Evaluations
• Guidelines for type and frequency determined by total value of project over lifetime
Component Two: Learning and Reflection
• Theory of Change and Impact Pathways
– Evidence Base
• Best Bet Selection Criteria
• Implementation Theory
• Ex ante Impact Assessments
• Ex post Impact Assessments
Theory of Change
• Calls for theory based approach that relies on development of explicit ToC for each VC, by CRP Phase:
– Identify critical linkages between program inputs and impacts;
– Identify critical conditions for success (e.g., contextual factors such as implementation theory, policy and economic conditions, etc.);
– Identify alternative explanations of change;
– Facilitate the identification of research questions that need to be tested in order to confirm the original program ToC.
• Step from ‘does it work’ to understanding what it is about the program that makes it work
Component Three: Outcomes and Impact Monitoring
• Development Indicators
– Harmonizing within the CRP and CGIAR• Baselines and Benchmarking
• Targeting
• International Public Goods
Baselines and Benchmarking
• Baselines will be conducted for specific donor-funded projects, including projects that are to be piloted during the Research Phase.
• IDO data will be collected through ‘Benchmarking’
– benchmarking exercises in which the best possible data will be used in order to establish estimated values for IDO indicators; in some cases, this may involve statistical modelling
Component Four: Knowledge Management
• Information Management
– CRP Monitoring Information System
– Performance Indicator Matrix
– Development Indicator Bank
– Evidence Base
• Communications
Component Five: Research Agenda
• Project Research (e.g., epIAs, project evaluations, project ToCs, etc.)
– Responsibility of Project Managers
– Funded by projects
• Program Research (e.g., epIEs, IEEs, CCEEs, VC ToCs, etc.)
– Responsibility of MEL Steering Committee and CRP Management
– Funded by CRP
• Research Quality and Ethics
Conclusion
• The MEL Team
– MELCoP
• Appendix One: Terminology
• References
CGIAR is a global partnership that unites organizations engaged in research for a food secure future. The CGIAR Research Program on Livestock and Fish aims to increase the productivity of small-scale livestock and fish systems in sustainable ways, making meat, milk and fish more available and affordable across the developing world.
CGIAR Research Program on Livestock and Fish
livestockfish.cgiar.org