m&e of africa rising
DESCRIPTION
M&E of Africa RISING. Carlo Azzarri, Beliyou Haile, Cleophelia Roberts, Melanie Bacou IFPRI/ HarvestChoice London, July 17, 2014. Agenda. Africa RISING M&E Objectives and Challenges ARBES Project Mapping and Monitoring Tool (PMMT) Purpose and scope Major components Next Steps. - PowerPoint PPT PresentationTRANSCRIPT
Carlo Azzarri, Beliyou Haile, Cleophelia Roberts, Melanie BacouIFPRI/HarvestChoice
London, July 17, 2014
M&E of Africa RISING
Africa RISING M&E Objectives and Challenges ARBES Project Mapping and Monitoring Tool (PMMT)
− Purpose and scope− Major components
Next Steps
Agenda
2
Africa RISING M&E Objectives and Challenges
3
Support effective project management
Fulfill FtF reporting requirements Generate knowledge on what
works and what doesn’t work Data/analytical components of
the M&E system:− Delineation and characterization
of target farming systems− Inventory of technologies− Ex-ante evaluation* − Attribution assessment*
M&E of Africa RISINGLearning about climbing beans in Linthipe, Dedza 2014.
5
Mega-site stratification by drivers of SI and creation of “development domains”
Identification of representative action research and control sites
Open access M&E data management platform (PMMT)
FtF indicators reporting Household and community
surveys (ARBES) Site visits and focus groups Meetings (survey design, project
review & planning, annual M&E)
M&E ActivitiesClosely watching the fava beans, Sinana, Oct. 2013.
Compiling and verifying data on FtF indicators (suitable level of detail?)
Compiling and verifying data on AR beneficiaries
Sequencing: selection of action sites before setting up specific research activities
Engagement and communication (e.g. lengthy recruitment of local M&E coordinators)
Delays in baseline surveys Sustainability and its
dimensions: indicators and benchmarks?
A few M&E Challenges…
6
Evaluation questions:• Did the program contribute to program goal and objectives? • Did it increase agricultural productivity?• Are the program’s approaches replicable/scalable?• Did farmers adopt new/improved technology(s) and practices?• What are the tradeoffs among different impacts?• How does impact vary across different domains, geographies,
household types, and gender?• Are the program-induced improvements sustainable?• Did the program demonstrate implementation methods that other
projects can use?
Impact Evaluation (IE)
7
Practicality of / approaches to attribution of cause and effect for an AR4D program
(Random) selection of action sites and/or subjects
Conflicting incentives among actors
Attribution with small N (action sites, beneficiary households) – generalizability of results
Impact of a “bundle of interventions”
Confounding factors Moving targets
A few E Challenges…
8
Africa RISING Baseline Evaluation Surveys (ARBES)
9
Malawi 2013; Tanzania, Ghana, Mali, and Ethiopia 2014 ARBES. In 2013 we started an ambitious workplan on conducting household and community surveys in five of the six AR countries in less than a year time (mega-site representative; N=700-1300; 50-page Qx).
AR Evaluation. We would like to assess the overall impact of the program on different outcomes (poverty, nutrition, health,…) using this approach.
Babati (Tanzania) impact evaluation. The team will continue and refine the impact assessment research project in Babati. Comments welcome!
Data and Analysis
Feb 2013
July 2013
Aug 2013 Feb 2014 Feb
2016
Initial planting at demonstration
plots
Follow-up field day: farmers rank preferred seeds
Randomization 996 farmers in 25 villages
205 receive fertilizers and
improved seeds
210 receive AR intervention
End-line survey: measure impacts
105 farmers non-beneficiaries
elsewhere
270 controls
206 Babati farmers non-beneficiaries
Tanzania IE timeline and design
Project Mapping and Monitoring Tool
12
Scope and PurposeMajor Components
http://dev.harvestchoice.org/africarising/
Need for a simple, flexible and engaging tool to report on program performance and capture decisions/adjustments made over 5-year program lifecycle:
− Baseline site characteristics− Site stratification and selection− Intervention details− Bi-annual FtF (and other project-
specific) indicators− Surveys and evaluation results− Links to intermediary and final output
(incl. primary data)13
Evaluation is hard.Monitoring is painful…
Aligning project goals, 2012.
14
MSExcel revisions no more! Grow organically based on
evolving M&E needs and partners’ feedback and capacity – do not overbuild
Flexible, adapted to most M&E designs
Simple to deploy in the field, no strong dependence on high-speed Internet connectivity
Leverage existing CGIAR tools and repositories
Openness (to and from 3rd party applications and databases)
PMMT Design PrinciplesInterviewing a farmer, 2013.
M&E site stratification & selection. Powerful spatial visualization features to provide rich contextual information, and overlays of local biophysical characteristics with socio-economic data in support of action/control site selection process.
15
Powerful Spatial Visualization
Streamlined indicator data entry and reporting. Familiar, secure interface for data entry with pre-set list of indicators help clarify reporting requirements. Data import/export tools for multiple end-uses.
16
Simplified Project Performance Monitoring
17
Series of in-country M&E and PMMT training planned through October 2014.
On-line PMMT User Guide Video tutorial
Additional Training Resources
Interviewing a farmer, 2013.
Next Steps
18
Analysis of ARBES and support to research teams
Add new spatial layers using results from recent LSMS-ISA and ARBES surveys
Provide vertical roll-ups of indicators across megasites, and mini program dashboard
Tighter integration between PMMT and CGIAR report and data catalogs (CGSpace, AgTrials, and ILRI CKAN)
Allow 3rd-party spatial data
sources to be read into and visualized alongside Africa RISING layers
Build support for off-line data entry (possibly using mobile devices)
Embed HarvestChoice MAPPR tools (point/polygon/domain summaries) for advanced spatial analyses
New hiring (Data Manager, ARF in ESA, SRA in WA and ETH?)
Plan for AREES/ARMES in 2016
M&E Roadmap
19
The End
Thanks!
20
ARBES Qx household summary contents
Location info, GPS Roster Education Labor Health Women and child anthropometrics Agriculture -general- Crop inputs (Conservation Agr.) Crop production Crop inputs (costs) Crop inputs (labor time use) Crop inputs (seeds) Crop sales Crop storage Livestock ownership and income Livestock feed
SECOND VISIT Problems and coping strategies Agricultural extension and AR program Other income Credit Housing, utilities, assets, distance to
services Subject welfare and food security Food Expenditures / Consumption Non-Food Expenditures Shocks Re-contact info
FIRST VISITHead Individual Best Informed
ARBES Qx community summary contents (5 to 8 informants)
Location info, GPS
Informants’ roster
Access to basic services
Agricultural labor, extension services, agricultural problems
Land use
Demographics, cooperatives, migration Water access, shocks, food consumption Market prices Conversion of non-standard units
IE Design
Beneficiary HHs Non-beneficiary HHs Control HHs
Action Sites Control Sites
Spillover effects
Program impact