Analytical Training in the Context of M&E Capacity Building

Siân Curtis and Gustavo Angeles

July 10-11, 2007

MEASURE Evaluation’s Training and CB Program

Components:

I. Training program

II. Partnership with five universities and training centers for building M&E training and TA capacity (and implementation of component I)

III. Learning materials

IV. CB through technical assistance

Partnership with Universities and Training Institutions

U. Pretoria/South Africa

Mahidol U./Thailand CESAG/

SenegalMexico/

INSP

Addis Ababa U./Ethiopia

MEASURE Evaluation’s Training Portfolio

Regional Workshops on:

- M&E of PHN programs

- Program impact evaluation

- M&E of HIV/AIDS programs

- Special: TB, RHIS

Country workshops:

- Use of household survey data and presentation of results to policy makers – Ecuador (with CDC)

- Other country M&E workshops (e.g. Kenya, Nigeria, South Africa)

Master’s Programs:

- M&E tracks in MPH (18 months) and MHSc (2 yrs), INSP/Mexico

- M&E track in MPH (18 months), SPH at Univ. of Pretoria/South Africa

- M&E concentration in Master in Population and RH Research, Mahidol Univ./Thailand

Workshops and Masters programs (Since 1998)

Regional Short-term Workshops:

Number of Number of Number of Number FundedWorkshops Applicants Participants by MEASURE

Evaluation

M&E of PHN 28 1,368 771 124

Impact Evaluation 9 523 145 53

M&E of HIV/AIDS 6 439 170 27

Total 43 2,330 1,086 204

Masters Programs:

Number of Number of Number of Number Funded Cohorts Applicants Students by MEASURE

Evaluation

Thailand 9 446 119 41

South Africa 7 857 124 29

Mexico 2 107 19 3

Costa Rica* 3 102 44 11

Total 21 1,512 306 84

Workshop on M&E of PHN programs

Duration: 2 or 3 weeksAudience:mid-level professionals working in M&EContent:

- Fundamentals of M&E: strategic information, frameworks, data systems and sources, indicators, evaluation designs, use of results

- Designing M&E plans: hands-on preparation of an M&E plan

- M&E of programmatic areas: FP, MCH, HIV/AIDS, Nutrition, others

- Calculation, interpretation and use of population-level PHN indicators (1 day)

- Calculation of coverage indicators using service statistics (1 day)

- SPSS/Stata for calculating indicators and basic descriptive analysis (1.5 days)

- Other: EpiInfo, basic survey methodology, sampling

Format:

Lectures, exercises, case studies, group work during 2 weeks with final presentation of M&E plan.

Workshop on Program Impact Evaluation

Duration: 3 or 4 weeksAudience:mid-level professionals working in M&EContent:

- Fundamentals of M&E: strategic information, frameworks, data systems and sources, indicators, evaluation designs, use of results, M&E plans

- Concepts and estimation strategies for evaluating program impact: evaluation designs, experimental and observational methods, selection of estimation strategy based on type of available data

- Hands-on practice using real household and facility survey data

- Household survey content and sampling design (1 day)

- Calculation, interpretation and use of population-level PHN indicators (1 day)

- Stata for data management and analysis

Format:

Lectures, exercises, case studies, group work during 3 weeks with final presentation of estimating a program impact

Workshop on M&E of HIV/AIDS programs

Duration: 2 weeksAudience:mid-level professionals working in M&EContent:

- Fundamentals of M&E: strategic information, frameworks, data systems and sources, indicators, evaluation designs, use of results

- Designing M&E plans for HIV/AIDS programs: hands-on preparation of a M&E plan

- M&E of HIV/AIDS programmatic sub-areas: prevention, care, support, amd treatment, PMTCT

- Calculation, interpretation and use of indicators

- Use of recently developed tools: systems strengthening, data quality assessment

Format:

Lectures, exercises, case studies, group work during 2 weeks with final presentation of M&E plan.

Workshop on Use of Household Survey Data and Presentation of Results

Duration: 2 weeks; in Ecuador (with CDC/DRH)

Audience:Pairs (technical and boss)

Content:

- Use of information for decision making

- Objectives, organization, content of HH survey

- Selection of policy-relevant topic

- Organization of data files and data management

- Calculation and interpretation of indicators

- Preparation of presentation to policy makers

- Presentation at forum-type public meeting with stakeholders

Format:

Lectures, exercises, group work to prepare analysis and presentation at forum

Masters Programs

Duration: 12 months, 18 months, 2 years

Audience:mid-level professionals

Content (through courses):

- Program M&E

- Quantitative methods for evaluating program impact

- Biostatistics, intermediate and advanced

- Epidemiology

- Research methodology

- Survey design and sampling

- Demography and RH

- Health measurement

- Statistical methods

- Thesis on M&E topic, usually using HH survey data

Format:

Lectures, exercises, thesis work

What we have learned on training:

- Analytical skills are not easy to transfer; it needs time and structured training activities

- Need to define which skills will be transferred

- Learning is done by doing: less lecture, more hands-on practice with real data; internships for data analysis

- More complete in-depth formation through Master of Public Health programs (with thesis using real data)

- Follow-up of participants:

- Ongoing support

- Opportunities to apply skills

- Opportunities for more advanced training

What we have learned on building training capacity:

- Training institution needs to be genuinely interested in strengthening its capacity in this area

- Better if it already identified M&E or analytical capacity in health data as areas for institutional expansion

- Engage leadership

- Identify a “Champion”, with authority, to implement CB process

- Identify a “team” of interested faculty; train, equip and support them

- How to get all of above?

- Provide financial incentives to institution, but it will be sustainable only if there is a market for analytical capacity (both in training of and its application)

- Financial incentives need to reach the implementers

- Key: Long-term support to institution

Periodic replenishment of leader and implementers as people move on.

MEASURE Evaluation is funded by the U.S. Agency for International

Development (USAID) through Cooperative Agreement GPO-A-00-03-00003-00

and is implemented by the Carolina Population Center at the University of North

Carolina in partnership with Constella Futures International, John Snow, Inc.,

ORC Macro International, and Tulane University.

Visit us online at http://www.cpc.unc.edu/measure.