building country m&e capacity to manage for and report on results - nigel brett
TRANSCRIPT
Annual performance reviewWORKSHOP
1-4 March 2009Bangkok, Thailand
Building country M&E capacity to manage for and report on results
The problem: Reporting results
• Projects generally report on activities and outputs, BUT did not generate evidence of achievement of objectives - RESULTS.
• Without this evidence IFAD and Government have to assume that outputs have resulted in objectives, supported by anecdotes from individual interviews.
• Development partners said they knew little of what IFAD is doing and what its projects achieve.
The Bangladesh response
IFAD’s programme in Bangladesh has generated project outcomes and impact. This has been measured by:
• Sample surveys on project impact: three final surveys, two mid-term surveys, three baseline reports.
• Case studies to illustrate impact on individual participants
• Project reporting of data on micro-credit, fisheries, etc.
• Knowledge Attitude Practice surveys on effective of training
• Dissemination of results via country newsletters.
Example of impact reporting in COSOP MTR
Change in poverty – Aquaculture Dev Project (2000-2006)
Survey group Percentage in each poverty class Hard core poor Moderate poor All poor Non-poor Total Before
Project* 42% 10% 52% 48% 100%
Control 37% 13% 50% 50% 100%
Now
Project* 32% 11% 42% 58% 100%
Control 38% 10% 48% 52% 100%
Change
Project* -10% 0% -10% 10% 0%
Contol 1% -3% -2% 2% 0%
* total for all project groups weighted according to proportion of the population in each group source: BIDS impact survey, 2006
Example of impact reporting – COSOP MTR
Increase in asset ownership
Value of assets (Taka) Area of land (decimal)
Productive Household Owned land Operated land
Before
Project* 20396 11455 72 72 Control 19835 14272 108 75 Now Project* 36925 18140 75 83 Control 33057 18441 112 85 Change % Project* 81% 58% 4% 16% Control 67% 29% 4% 13%
* total for all groups weighted according to proportion of the population in each group source: BIDS impact survey, 2006
Example of impact reporting – COSOP MTR
Increase in land, livestock and farm equipment
Project Control Crop
before after change Before after Change
Own land 77.3 84.5 9% 87.6 87.8 0% Avg. cultivated land per h’hold (decimal)
Lease or share 12.9 33.0 156% 10.5 22.5 115%
Cattle/buffalo 0.9 1.8 101% 1.3 1.6 19% Sheep/goat 0.8 1.5 96% 1.0 1.2 15%
Livestock: average number per household Poultry 7.3 19.0 159% 7.8 11.1 43%
Thresher 0.8 5.6 626% 0.8 2.3 201% Weeder 9.4 27.6 192% 15.3 26.0 70%
Farm equipment: Percent of households owning
Shallow tubewell 8.1 18.7 131% 10.7 17.6 64%
Source: Impact study sample survey
What is needed to enable effective project M&E? Part A - Planning
Planning the project:
• M&E system design as an annex in IFAD Design Document
• M&E unit: independent unit within or attached to PMU with own staff down to field level, transport, computers etc.
What is needed to enable effective project M&E? Part B – Implementation Support
Implementation support: M&E specialists give hands-on support: •Plan and advise on M&E at project start-up or at MTR, •ToR for surveys and studies •Train M&E and other field data collection staff, •KAP training surveys •Writeshop (through ENRAP) to build capacity for case studies.•Review draft survey reports and interpret survey results.
What is needed to enable effective project M&E? Part C – IFAD missions
Supervision missions: • Pay specific attention to
performance of project M&E. • Prior to mission, project sent
blank tables for data on specific performance areas, such as micro-credit.
Design and completion missions:• Commission studies using
participatory tools which require expert facilitation.
Examples of blank tables
Credit and Savings (latest available data)
Sl.no Name of the Organization Loan from PKSF to PO (lakh) Loans from PO to members (Tk lakh) No. of running loans Net
Disburse Recover Outstanding Disbursed Recovered Outstanding Overdue male female Total Savings
(Lk)
1 TMSS
2 SDC
3 SSS
4 PPSS
5 Annesha Found.
6 POPI
7 Dak Diye Jai
8 Uddipan
9 NUS
10 HFSKS
11 CCDA
12 PBK
13 SDS
14 EWF
15 GUP
16 Sangkalpa Trust
17 PMUK
18 BEES
19 VARD
20 BDS
21 COAST-Trust
22 ICDA
23 GJUS
24 CARSA Found.
25 CDIP
Total
Examples of blank tables
Use of loans
During 2007/08 Average size of loans (Taka) Name of IGAs
Hardcore Moderate Total
Cumulative up to June 2008
During 2007/08 Cumulative Amount (Tk) Persons Amount (Tk) Persons Amount (Tk) Persons Amount (Tk) Persons hardcore moderate all all General livestock Chick rearer Parent breeder Poultry keeper Duck Keeper Feed seller Goat keeper Buck Station Cow rearing Beef fattening Broiler Layer Mini hatcherery Homested Total (General) Specialized Livestock
Poultry worker Dairy production Total (apecialised) Total(gen+spec.) Others Trade & shop Food processing Agric & veg Transport Other Grand total
Examples of blank tables
Training of Beneficiaries from POs Training provided during 2007/8 Training provided (cumulative from start of project to most recent date)
General Livestock Training Specialized Livestock Training General Livestock Training Specialized Livestock Training Sl no.
POs name
Hardcore Moderate Total Hardcore Moderate Total
Social Dev.
Training Hardcore Moderate Total Hardcore Moderate Total
Social Dev. Training
1 Annesha Found.
2 BDS
3 BEES
4 Carsa Foundation
5 CCDA
6 CDIP
6 Coast Trust
7 DDJ
8 EWF
9 GJUS
10 GUP
11 HFSKS
12 ICDA
13 NUSA
14 PBK
15 PMUK
16 POPI
17 PPSS
18 Sangkalpa Trust
19 SDC
20 SDS
21 SSS
22 TMSS
23 Uddipan
24 VARD
Total
What is needed to enable effective project M&E? Part D - Dissemination
Country Newsletters in English (two per year for last 3 years): •Target audience: donors, government, NGOs, consultants, academics. •Primarily distributed in electronic format, also printed
Country Newsletters in Bangla – planned to produce two per year • Target audience: field staff of project, partner agencies, local
authorities, communities and other stakeholders. • Contents – more news focused than English edition• Distributed as a printed version
Key messages to take home
• M&E is difficult and complicated, but it is not impossible.• Pick one approach and stick to it - same M&E design, costs and
approach for all projects in the country programme;• Stick to it for the long term - application of exactly the same approach
for many years;• Use the same M&E support team for all projects for the long term
(consistent messages, and relationship building);• Apply sufficient and regular annual IFAD M&E implementation support
budget for each project (40 consultant days per project; total budget of approx USD 36,000 per project or USD 6000/year);
• Invest in the M&E field officers.. Make sure they have the required equipment, transport and training..
• Ensure you send the same empty data tables to all projects prior to supervision missions for compilation.
• Ensure one of the M&E support team on each supervision mission.