1

Patrick Webb USAID August 2013

Repaving the pathways:

What do we need to know about processes that support program impacts?

2

Nutrition Innovation Lab over-arching questions In what ways do investments in agriculture achieve

significant measurable impacts in nutrition? Can impact pathways be empirically demonstrated?

How can large-scale programs best incorporate such knowledge into cost-effective multi-sectoral interventions to improve nutrition?

How can policy and program implementation processes be

enhanced to support both nutrition-specific and nutrition-sensitive actions?

3 Source: Gillespie et al. (2012)

4

5

Bardasi et al. (2010) show that who reports can matter even for seemingly objective information. Who provides the information matters, and it matters more in some cases than others. People in the household might all respond the same way when reporting on simple measures of household conditions, e.g. the material used for the roof, but for much of what is captured in surveys different people in the household will have different responses.

Source: Bardasi et al. (2010) Do Labor Statistics Depend on How and to Whom the Questions Are Asked? Results from Tanzania (World Bank)

6 Source: Coates et al. (2010) He Said, She Said. Food Security

7 Source: Coates et al. (2010) He Said, She Said. Food Security

8

Is the concept of household food insecurity useful if the situation of individual family members is different, and if few food insecurity experiences are collectively and similarly shared? Questions should be excluded that are likely to be answered inaccurately by men or by women due to their separate spheres of food-related responsibility. On the other hand, certain questions were shown to be relevant to both genders, and discordance appeared to be explained by different experiences of food insecurity within the same household.

Coates et al. (2010) He Said, She Said. Food Security

9

10

World Bank (2010) What can we learn from nutrition impact evaluations? Washington, D.C.

11

There has been limited empirically-based research on the sociopolitical factors that influence national policy formulation, including the ability to generate effective policy [and program] traction and resources. This may in part be because policy decisions are complex, ambiguous and involve many actors with varying aims, perspectives and power.

Source: Lapping et al. (2012) Development of Nutrition Agenda in Vietnam (Health Policy and Planning)

12

Knowledge, Attitudes and Practices (KAP) Individuals Policymakers

Key practices Hand-washing Washing ones hands

Health-seeking Seeking help (or not)

Key attitudes to Child weight-loss Chronic undernutrition Latrine use Open defecation

Key behaviors Birth spacing Staff retention/rotation Meds. compliance Policy instrument fidelity

13 Source: Pelletier et al. (2010) A2Z-AED

14 Source: Bhutta et al. (2013) Pakistan

15 Source: Swart et al. (2008) Nutrition: Primary Health Care Perspective (Durban)

Defined Goal: Health and Well-being of Nepalis Improved and Sustained

Strategic Objective: To Improve the Nutritional Status of Women and Children Under Two Years of Age

Intermediate Result 1: Household (HH) health and nutrition behaviors are improved.

Intermediate Result 2: Women and children increase use of quality nutrition and health services.

Intermediate Result 3: Women and their families increase consumption of diverse and nutritious foods.

Intermediate Result 4: Coordination on nutrition between government and other actors is strengthened.

PresenterPresentation NotesFinally, the N/CRSP will aim to understand if effectiveness of an integrated program such as Suaahara and that its key goals have been achieved ie a. behaviors are improved 2. access to health services is improved 3. there is increased consumption of diverse and nutritious foods and 4. there is improved coordination.

Legend

Terai

Hill Districts Mountain Districts

Sentinel Sites 17

Service Delivery

Sector Coordination

Cross-ministry coordination

Cross-Policy and donor coordination

18

Water Quality

Shared pathogens

Mycotoxins

Nutrient-rich foods

Animal proteins

19

Lapping et al. (2012) Vietnam, 22 policy makers

Pelletier et al. (2012) Bolivia, Peru, Guatemala: We employed semi-structured interviews with selected stakeholders and key informants; we engaged several staff members in discussions. Pelletier et al. (2011) Ethiopia, Senegal, Uganda, the

Philippines, Thailand: Accounts of the nutrition policy process were elicited [from] 18 respondents from 12 countries [and ] 6 respondents from donors or NGOs were asked to comment.

Sample size for process interviews?

20

21

1. Vertical coherence. Do individuals in chain of authority share common understanding of policy agendas, problems, technical issues, capacity needs and constraints.

2. Horizontal coherence. Do individuals with similar levels of authority share a common understanding.

3. Collaboration dynamics. What are determines effective collaboration? Does KAP change over time? Does this enhance effectiveness?

4. Policy/program fidelity as an outcome of vertical and horizontal coherence. Is effective roll out and scale up determined by coherence?

5. Policy and Program Fidelity as a determinant of nutrition outcomes. Do measures of coherence serve as explanatory variables for nutrition outcomes?

22

Program/policy fidelity (did intervention match design?) Reasons for uptake of programmed resources at household

level (exposure to ideas, services and assets; decisions to adopt or not; constraints to adoption)

Effectiveness of program uptake (process tracing to determine impact of adoption had on nutrition outcomes)

Sustainability of program intent (duration, intensity of adoption)

Equity issues (socioeconomic and geographic distribution of coverage, exposure, adoption/acceptance, and impacts)

Program and policy fidelity (technical issues)

23 En

viro

nmen

tal c

ondi

tions

(myc

otox

ins,

wat

er q

ualit

y, gu

t m

icro

biom

e)

Clim

ate

and

geog

raph

y (s

easo

ns, e

colo

gy, r

ainf

all,

mar

ket

acce

ss, p

rices

)

Individual level (diet quality, nutrition & health)

National Level (policy context, political framing )

Meso level (policy process, resource management, multisectoral collaboration)

Field level (effective training, resourcing, collaboration among field service providers)

Household level (assets, behaviors, consumption, etc.)

24

Take-home messages: 1. Process research in its infancy despite all the calls for

empirically-based delivery science (attention needed to appropriate methods -- not just RCT/non-RCT, but who to ask and what to ask of whom?)

2. Role of individuals matters in institutions, as in households.

3. Delivery science is no luxury. Understanding how impact achieved (not just if achieved), is critical to USAID going forward at scale.

25

27

Limited evidence with inconsistent findings across varied interventions in different settings means no general conclusions can be drawn on the benefits or disadvantages of integrated service delivery.

Systematic review of 8, 274 citations = 12 studies

Atun et al. (2010) A systematic review of the evidence on integration of targeted health interventions into health systems (Health Policy and Planning)

28

Levels of enquiry for CRSP process research 1. Central Government (policy process) 2. Central Program Management (Suuahara, USAID Mission) 3. Suaahara Program Partners (NGOs, other donor-led programs) 4. District Level (health and agric. personnel; others (e.g MOLD) 5. Health facility staff/Agricultural outreach staff 6. FCHVs 7. Poshan Amaa (PA)/Model Farm managers 8. Households (mothers, fathers, others)

Tufts

JHU

30

Agency Annual Cost* Inputs and services provided

World Bank $30 per child Vitamin supplements, deworming, iron fortification of staples, salt iodization, CMAM

REACH $36 per child Soap, bednets, malaria treatment, home gardens, clean water.

SNRP (EU) $61 per person Nutrition education, water, hygiene, seeds, village savings banks, extension services

WALA (USAID)

$61 per person Seeds, irrigation, nutrition and health education, health services, microfinance

Millennium Villages

$60 per person Village storage, seeds, clinics and schools, seeds, internet access, phones,

IFSP Mulanje $11 - $46 per person

Seeds, irrigation, food-for-work (trees, roads), livelihoods (training, inputs), food technology

Source: Black et al. (2008) Lancet Paper 1

Reduced

Source: Masset et al. 2011

Knowledge, income control (gender roles, BCC) Animal source

foods, nutrient- dense foods

Food safety (mycotoxins)

Environmental enteropathy (clean water, open defecation) Empty calories;

opportunity costs Do no harm (malaria, bird flu, animal waste)

Logical framework for assessing impact of agricultural interventions on nutrition (Masset et al. 2011)

Mountain sites

Hill sites

Valley (Terai) sites

35 Source: Benson (2007) Cross-Sectoral Coordination (FNB)

1

2 2

3

ENA/EHA Agriculture

Health Delivery Training Activities

Model Farms Project Management

Income growth

Inputs and Activities

Seeds, fertilizer, model farms (IR4)

BCC, health service delivery (IR4)

Service usage (IR2)

Outputs

Impacts

Outcomes

Crop diversity

Changed behaviors (IR1)

Diet diversity (IR3) Better birth outcomes, health status, micronutrient status

Child Stunting (SO)

Mothers Nutrition (SO)

Data collection foci on Integrated Programming

PresenterPresentation NotesN/CRSP research activities are being implemented by Tufts and JHU. To address the three focus areas, complex data collection needs to be undertaken in collaboration with different entities/partners. The data collection will be complementary to the activities of IFPRI who is conducting the baseline and maybe the endline of the Suaahara evaluation. Data collection will include both qualitative and quantitative methodologies with household surveys,, key informant interviews, surveys and semi-structured interviews at program, district, facility level. Data on integrated program management will also be obtained from Suaahara team which Tufts will obtain through semi-structured interviews

ENA/EHA Agriculture

Health Delivery Model Farms Project Management

Income growth

Seeds, fertilizer, model farms (IR4)

BCC, health service delivery (IR4)

Service usage (IR2)

Crop diversity Changed behaviors (IR1)

Diet diversity (IR3)

Child Stunting (SO)

Mothers Nutrition (SO)

Service Delivery

Sector Coordination

Cross-ministry coordination

Cross-Policy and donor coordination

PresenterPresentation NotesN/CRSP research activities are being implemented by Tufts and JHU. To address the three focus areas, complex data collection needs to be undertaken in collaboration with different entities/partners. The data collection will be complementary to the activities of IFPRI who is conducting the baseline and maybe the endline of the Suaahara evaluation. Data collection will include both qualitative and quantitative methodologies with household surveys,, key informant interviews, surveys and semi-structured interviews at program, district, facility level. Data on integrated program management will also be obtained from Suaahara team which Tufts will obtain through semi-structured interviews

39

Nutrition Outcomes

Mycotoxins

Water Quality

Shared pathogens

Environmental Enteropathy

Agriculture investments

Slide Number 1Slide Number 2Slide Number 3Slide Number 4Slide Number 5Slide Number 6Slide Number 7Slide Number 8Slide Number 9Slide Number 10Slide Number 11Slide Number 12Slide Number 13Slide Number 14Slide Number 15Slide Number 16Slide Number 17Slide Number 18Slide Number 19Slide Number 20Slide Number 21Slide Number 22Slide Number 23Slide Number 24Slide Number 25Slide Number 26Slide Number 27Slide Number 28Slide Number 29Slide Number 30Slide Number 31Slide Number 32Slide Number 33Slide Number 34Slide Number 35Slide Number 36Slide Number 37Slide Number 38Slide Number 39