monitoring and evaluation in mch programs and projects mch in developing countries feb 10, 2011
TRANSCRIPT
Monitoring and Evaluation
in
MCH Programs and Projects
MCH in Developing Countries Feb 10, 2011
• Session objectives:– Explain similarities and differences between
monitoring and evaluation – Describe the major purposes, types and tools
for MCH program evaluation– Contrast traditional evaluation approaches with
participatory evaluation – Develop a monitoring plan for one program
activity
Objectives of both monitoring & evaluation:
• Improve program planning process
• Improve program management
• Improve program performance
• Assess program effects on beneficiaries
• Strengthen links with participant groups
What is monitoring?
Why do we do it?
Monitoring:• The periodic, regular collection and
analysis of selected indicators
• Conducted to determine whether key activities are being carried out as planned, and identify corrections needed
• The MIS (management information system) provides early indication of progress, or a lack of progress
Monitoring:• Can be carried out by any staff, not a specific
evaluation staff person
• Informs a project about strengths & weaknesses of the strategies: do we need to refine them?
• Important: use it to provide regular feedback to staff
Is Monitoring Supervision?
Is Monitoring Supervision?
• We monitor implementation of an intervention – typically output and process indicators
• We supervise an individual
Monitoring and Evaluation:how are they related?
• Monitoring:– the regular collection
and analysis of selected indicators
– conducted to determine whether key activities are being carried out as planned, and identify corrections needed
• Evaluation:– done at longer intervals
than monitoring
– conducted to provide an indication of success of a program, and identify problem areas
Characteristics of Evaluation• Assesses the value of something• Systematic and ‘objective’ review of evidence
related to specific questions about a program• Carried out selectively – costly and time
consuming • Assess performance and effects of program
efforts -- requires analysis Process: outputs, organization, management Outcome or impact: effects on beneficiaries
Evaluation ProvidesInformation on:
• Strategy –Are the right things being done?
• Operations –Are things being done right?
• Learning –Is it having the desired effects? Are there better ways?
Types of evaluation:
• Formative
• Summative
• Of relative need for a program
• Of the feasibility of a program design
• Of program performance or process
• Of direct effects or outcomes of a program
• Of long-term impact
• Internal or external
Internal Evaluations• Staff know more history, organization,
culture, problems, successes• Can interpret evaluation findings more
accurately • May be too close, hard to be ‘objective’• Part of power structure, may have
personal agendas• Likely not highly trained in evaluation• Known to the program so less likely to
be threatening, disruptive
External evaluators• Can take a ‘fresh’ look at the program
• Not personally involved; more ‘objective’
• Not a part of the normal power structure
• Less chance of personal gain
• Well trained in evaluation methods, seen as an ‘expert’ by program
• Outsiders may not understand the program or people involved
• May cause anxiety if not known and trusted
The best of both: Participatory Evaluation
• Participation by a range of stakeholders
• Focus on program participants, not donors
• Focus is on learning, not just accountability
• Flexible design, not predetermined
• Use rapid appraisal methods, less formal
• Consultants are facilitators, not the actual ‘evaluator’
• Evaluation team is key to approach
What does an evaluation team do?
Preparation:
• Establish purpose, methods (‘scope of work’) including the evaluation questions
• Develop data gathering instruments
• Develop team plan and itinerary
Main activities:
• Visit main offices, field sites
• Gather data (document review; observations, group discussions, interviews; review surveys)
Report preparation:
• Team discussion of findings, come to tentative conclusions and recommendations
• Draft main findings of the evaluation report
• Discuss draft of findings with program staff, other stakeholders
• Revise findings, recommendations as needed
• Finalize and submit report
• Share with staff, community, MOH, others
When in doubt, observe and ask questions.
When certain, observe at length and ask many more questions!
From Halcolm’s Evaluation Laws
Monitoring exercise:
• Choose one objective for your group project
• For that objective, select one important activity that the project will monitor
• Discuss and fill in the information for the six exercise questions