multi-dimensions of nutrition-related research to strengthen programming: using process evaluation...
Post on 22-Jan-2018
121 Views
Preview:
TRANSCRIPT
USING PROCESS EVALUATION TO STRENGTHEN PROGRAM DELIVERY
GHPC May 17, 2016 – Jennifer Nielsen, PhD, Senior Nutrition Advisor
WHY PROCESS EVALUATION?
• Randomized controlled trials are considered the gold standard in medicine for evidence of impact of a treatment; increasingly used for effectiveness of broader public health interventions
• With a highly complex intervention there is need for more information on the dose and fidelity of delivery to add plausibility to the evidence (intervention - - - > impact)
• HKI’s Enhanced Homestead Food Production (EHFP) includes • Training to improve horticultural and husbandry practices • Behavior change communication to influence nutrition, health, hygiene practices • Multiple partnerships • Cascade training structures with three levels • Gender empowerment/transformation strategies
In short, many opportunities for losses in both dose and fidelity!
Rationale
BCC training on ENA for beneficiaries
Outcomes Impact Inputs Process Outputs
HKI, APRG and governmental
structures (Ministries of
Health, Agriculture,
Animals, Environment, and the Promotion of
Women, local authorities and officials) work
together
Training in plant and
animal production
techniques for master trainers
BCC training on ENA for community
level nutrition trainers
Develop a training strategy in animal; and
plant production techniques
BCC training on ENA for
master trainers
Develop a behavior change communication
(BCC) strategy to promote Essential Nutrition Actions
(ENA)
Training in plant and
animal production
techniques for Village Farm
Leaders (VFL)
Village Model Farms (VMF)
established
Training in plant and
animal production techniques
for beneficiaries
Establishment of individual
farms (40 women per
village)
Improved maternal and child health
and nutrition outcomes
Agriculture and zoological inputs
distributed
Improvements in small
ruminant and poultry
production
Improvements in fruit and vegetable
production
Improvements in household
consumption
Increased Income
Beneficiaries received and understood
BCC training on ENA
Improvements in nutrition and
feeding practices for children,
pregnant women and breastfeeding
mothers
Adoption of agriculture practices
Women’s empowerment
improved
Women’s assets increased
Increased availability of micronutrient rich fruits and
vegetables
Adoption of ENA
practices by beneficiaries
Improvements in care & hygiene
practices for children,
pregnant women and breastfeeding
mothers
Beneficiaries received and understood agriculture
training
Increased availability of
food from animal origin
Nutrition Pathway Production
Pathway Income Pathway
Program Theory - EHFP Burkina
AKA…..
5
• Program funded by USAID – OFDA (2009-2012)
• IFPRI research lead; HKI implementation lead
• First attempt to adapt EHFP model to Africa
• Implemented as RCT with longitudinal design following HH with children 3-12 months at baseline1
• Process evaluation conducted ~ midterm April-May 2011
1Olney et al (2015, 2016) J. Nutr.
PROCESS EVALUATION IN BURKINA
• Mixed methods – SSI both closed and open ended with
random sample of beneficiaries (n=145) and controls (n=75)
– KII with purposively selected master trainers in ag (n=13) and nutrition (n=24)
– KII with community nutrition volunteers: • older women leaders (n=30) • health committee members (n=28)
– KII with village farm leaders (n=58)
• Guided by PIP (supplies delivered; training attended; contents understood; knowledge gained; motivation and activities at all levels)
• Explored informants perceptions of program quality and areas for improvement
METHODOLOGY
• Enumerator training, data collection and analysis led by IFPRI with significant involvement of HKI implementing team
• Findings vetted and debated with implementation team
• Findings also shared with advisory committees at local and national levels
• Recommendations incorporated, as feasible, in subsequent work plan
ROLL OUT
• Water challenges: demand exceeding supply – Repaired and built boreholes – Foot pumps – Drip irrigation for VMF and HH – Watering cans and wheelbarrows – Relocated some VMF – More drought-resistant crops
• High loss of poultry, low production & consumption
– Mixed poultry breeds and more local procurement – Assured vaccine supply
• Low fruit production and consumption – Mangoes not yet bearing fruit (3-4 years from) – Papayas bear for 8 months but ~50% died
FINDINGS: PRODUCTION & INCOME PATHWAYS
• Leakage from Master Trainers to Volunteers to Beneficiaries
– Some topics in particular (rich complementary foods, feeding sick child, anemia control, women’s nutrition)
– Refresher training on key topics – Focus messages (egg consumption) – Training in facilitation skills – Supportive supervision
• Motivation of volunteers mixed – Public recognition – Small gifts
• Home visits less than expected
– Emphasize group discussions
FINDINGS: KNOWLEDGE - CONSUMPTION PATHWAY
• Lag times at multiple stages reduced exposure
– From award to beginning of implementation (6 months)
• Training of Master Trainers and delivery of inputs
• Planning of research – Between baseline and VMF production +
nutrition BCC (6 months) • Training of VMF then HH • Staple (rainy season) production • ENA training • HH gardens year 2
– Horticulture production on off (dry) season
FINDINGS: LOGISTICS
• Seasonality constraint (endline = baseline) allowed only 2 years of implementation
• Tremendous enthusiasm of communities and local partners
• Within intervention communities
– “My husband thinks that my work at the VMF is contributing to improved health in our household”
– “I have a family garden to cultivate vegetables and meet the nutritional needs of my household”
• Spillover doubled population reach; demand from control communities
TESTIMONIALS
• Research findings valuable but come at cost to implementation
– Staff time to support logistics – Time burden on beneficiaries and controls
• Not all problems can be solved – Water and soil constraints – Weaknesses in larger input delivery systems – Intrinsic motivation not universal (also VMF vs. ENA) – Low education level of community volunteers
• Complexity of model – Multisectorality of contents and partners – ENA (and now EHA) dense with messages and behaviors to
change
CHALLENGES IN PROCESS EVALUATION
• Findings informed strengthened program delivery
• Process gave voice to staff and beneficiaries
• Informed changes to model for subsequent phase
– Timing/design of research – Water management plans – Incentive strategies – Reinforced WASH and malaria
control components – Test LNS to fill nutrient gaps
• Effort is high; returns are higher
RETURNS
MERCI
15
“We could never learn to be brave and patient if there were only joy in the world.” – Helen Keller
top related