unc water and health conference 2011: heterogeneity in trial data: learning from difference,...
DESCRIPTION
TRANSCRIPT
![Page 1: UNC Water and Health Conference 2011: Heterogeneity in trial data: learning from difference, Professor Rick Rheingans, University of Florida and SHARE](https://reader033.vdocuments.us/reader033/viewer/2022051817/547d44e7b47959c5508b48b6/html5/thumbnails/1.jpg)
Heterogeneity in Trial Data: Learning from Difference
Rick Rheingans, PhDUniversity of Florida
SHARE Research Consortium
![Page 2: UNC Water and Health Conference 2011: Heterogeneity in trial data: learning from difference, Professor Rick Rheingans, University of Florida and SHARE](https://reader033.vdocuments.us/reader033/viewer/2022051817/547d44e7b47959c5508b48b6/html5/thumbnails/2.jpg)
What Works Best? It Depends
• Sector debate at the interface of science and policy• Based on reasonable questions: what will work
best here?
• Differences between studies– Meta-analyses
• Differences within studies– Analytical focus on main effects
• Differences outside of studies
![Page 3: UNC Water and Health Conference 2011: Heterogeneity in trial data: learning from difference, Professor Rick Rheingans, University of Florida and SHARE](https://reader033.vdocuments.us/reader033/viewer/2022051817/547d44e7b47959c5508b48b6/html5/thumbnails/3.jpg)
Meta-analyses and Heterogeneity
![Page 4: UNC Water and Health Conference 2011: Heterogeneity in trial data: learning from difference, Professor Rick Rheingans, University of Florida and SHARE](https://reader033.vdocuments.us/reader033/viewer/2022051817/547d44e7b47959c5508b48b6/html5/thumbnails/4.jpg)
• Variability across and within studies• Depends on behaviors• Depends on who – higher protection among the most
vulnerable• Depends on initial water quality and other exposures
![Page 5: UNC Water and Health Conference 2011: Heterogeneity in trial data: learning from difference, Professor Rick Rheingans, University of Florida and SHARE](https://reader033.vdocuments.us/reader033/viewer/2022051817/547d44e7b47959c5508b48b6/html5/thumbnails/5.jpg)
Sources of Variability within Trials
• Different effect levels in different sub-populations due to behavior or vulnerability– Opportunity for targeting
• Spatial differences environmental conditions affecting exposure– Opportunity for geographic targeting
• Differences between settings based on implementation– Opportunity to adjust adjust the intervention
![Page 6: UNC Water and Health Conference 2011: Heterogeneity in trial data: learning from difference, Professor Rick Rheingans, University of Florida and SHARE](https://reader033.vdocuments.us/reader033/viewer/2022051817/547d44e7b47959c5508b48b6/html5/thumbnails/6.jpg)
Analytical Tools for Teasing Out Difference
• Random effects models– Did the intervention work differently in different
communities – especially for cluster randomized trials• Effect modification– Are there characteristics of individuals or
communities that change the impact of the intervention
– Stratification to look at discrete populations
• Focus is usually on the main effect
![Page 7: UNC Water and Health Conference 2011: Heterogeneity in trial data: learning from difference, Professor Rick Rheingans, University of Florida and SHARE](https://reader033.vdocuments.us/reader033/viewer/2022051817/547d44e7b47959c5508b48b6/html5/thumbnails/7.jpg)
Grappling with Differences: School Water, Sanitation and Hygiene Impacts
• SWASH+ – Collaborative applied research and advocacy project led by
CARE in western Kenya• Cluster-randomized trial in 185 schools– Included hygiene promotion, water treatment, sanitation
infrastructure, and water supply• Objective:– Estimate the impact of school WASH interventions on
health (helminthes and diarrhea), educational outcomes (absenteeism and performance), and behaviors (e.g., diffusion to homes)
![Page 8: UNC Water and Health Conference 2011: Heterogeneity in trial data: learning from difference, Professor Rick Rheingans, University of Florida and SHARE](https://reader033.vdocuments.us/reader033/viewer/2022051817/547d44e7b47959c5508b48b6/html5/thumbnails/8.jpg)
What’s the Question?
• National and global policy and advocacy interest in estimating the main effects– Days of absence avoided– Percent reduction in diarrhea
• Compare it to other school investments• Compare it to other WASH investments
• What if the most important answer is - it depends?
![Page 9: UNC Water and Health Conference 2011: Heterogeneity in trial data: learning from difference, Professor Rick Rheingans, University of Florida and SHARE](https://reader033.vdocuments.us/reader033/viewer/2022051817/547d44e7b47959c5508b48b6/html5/thumbnails/9.jpg)
Differential Impacts of School Water, Sanitation and Hygiene
• Absenteeism (Freeman et al, 2011)– Strong impact for girls (Odds Ratio 0.4), no measureable impact for boys
• Helminths reinfections - – Differences by gender
• Ascaris for girls; especially poorest• Hookworm for boys; especially poorest
– Differences by behavior• Reduced hookworm reinfection among boys without shoes
• Diffusion of behavior change (water treatment) to homes– Strongest effect among the poorest households
• Differences between schools and regions
![Page 10: UNC Water and Health Conference 2011: Heterogeneity in trial data: learning from difference, Professor Rick Rheingans, University of Florida and SHARE](https://reader033.vdocuments.us/reader033/viewer/2022051817/547d44e7b47959c5508b48b6/html5/thumbnails/10.jpg)
• Conduct across 3 Districts in western Kenya
• Differing socio-economic and exposure conditions
![Page 11: UNC Water and Health Conference 2011: Heterogeneity in trial data: learning from difference, Professor Rick Rheingans, University of Florida and SHARE](https://reader033.vdocuments.us/reader033/viewer/2022051817/547d44e7b47959c5508b48b6/html5/thumbnails/11.jpg)
Trying to Explain Differences in School-Cluster Performance
• Reveals challenges in sustaining hand washing facilities and water treatment
• In compliance adjusted analysis, both having HW facilities and treated water are associated with reduced absence
![Page 12: UNC Water and Health Conference 2011: Heterogeneity in trial data: learning from difference, Professor Rick Rheingans, University of Florida and SHARE](https://reader033.vdocuments.us/reader033/viewer/2022051817/547d44e7b47959c5508b48b6/html5/thumbnails/12.jpg)
Trying to Explain Differences: New Pathways
• In schools receiving new latrines, children had increases in fecal hand contamination
• Suggests – Importance of latrine cleanliness– Interdependence of hand-washing and sanitation– Need for anal cleansing materials
![Page 13: UNC Water and Health Conference 2011: Heterogeneity in trial data: learning from difference, Professor Rick Rheingans, University of Florida and SHARE](https://reader033.vdocuments.us/reader033/viewer/2022051817/547d44e7b47959c5508b48b6/html5/thumbnails/13.jpg)
Implications?
• What to invest in: – De-worming? – School uniforms? – More teachers? – School WASH?
![Page 14: UNC Water and Health Conference 2011: Heterogeneity in trial data: learning from difference, Professor Rick Rheingans, University of Florida and SHARE](https://reader033.vdocuments.us/reader033/viewer/2022051817/547d44e7b47959c5508b48b6/html5/thumbnails/14.jpg)
Different Conditions and Impact Variability: A Hypothetical Exercise
• Overall impact estimates provide us with the ‘average’ setting, but what will it be in a particular setting?
• Assume a setting where on average 35% of under-5 diarrhea preventable through improved sanitation
• Part of diarrhea burden is due to non-sanitation related exposures
• Some due to whether the household has sanitation
• Some due to whether they have to share that facility
• Some due depending how community’s coverage
![Page 15: UNC Water and Health Conference 2011: Heterogeneity in trial data: learning from difference, Professor Rick Rheingans, University of Florida and SHARE](https://reader033.vdocuments.us/reader033/viewer/2022051817/547d44e7b47959c5508b48b6/html5/thumbnails/15.jpg)
Different Conditions and Impact Variability: A Hypothetical Exercise
• Overall impact estimates provide us with the ‘average’ setting, but what will it be in a particular setting?
• Assume a setting where on average 35% of under-5 diarrhea preventable through improved sanitation
• Part of diarrhea burden is due to non-sanitation related exposures
• Some due to whether the household has sanitation
• Some due to whether they have to share that facility
• Some due depending how community’s coverage
Household
Other
Comm35% Preventable
![Page 16: UNC Water and Health Conference 2011: Heterogeneity in trial data: learning from difference, Professor Rick Rheingans, University of Florida and SHARE](https://reader033.vdocuments.us/reader033/viewer/2022051817/547d44e7b47959c5508b48b6/html5/thumbnails/16.jpg)
What Happens with Greater Community Exposures
• If there is heterogeneity in level of community exposure –– Would severe
diarrhea rates go up?
– Would the preventable fraction with sanitation go up?
Household
Other
25% Preventable?
Household
Other
Comm
Household
Other
Comm
Household
Other
Comm35% Preventable
50% Preventable?
65% Preventable?
![Page 17: UNC Water and Health Conference 2011: Heterogeneity in trial data: learning from difference, Professor Rick Rheingans, University of Florida and SHARE](https://reader033.vdocuments.us/reader033/viewer/2022051817/547d44e7b47959c5508b48b6/html5/thumbnails/17.jpg)
Variability in Community Level Exposure
• One measure may be population density of people without sanitation
• Based on cluster-level coverage and population density
• Varies widely within countries and provinces
![Page 18: UNC Water and Health Conference 2011: Heterogeneity in trial data: learning from difference, Professor Rick Rheingans, University of Florida and SHARE](https://reader033.vdocuments.us/reader033/viewer/2022051817/547d44e7b47959c5508b48b6/html5/thumbnails/18.jpg)
Variability in Community Level Exposure
• One measure may be population density of people without sanitation
• Based on cluster-level coverage and population density
• Varies widely within countries and provinces
Does sanitations impact change?
![Page 19: UNC Water and Health Conference 2011: Heterogeneity in trial data: learning from difference, Professor Rick Rheingans, University of Florida and SHARE](https://reader033.vdocuments.us/reader033/viewer/2022051817/547d44e7b47959c5508b48b6/html5/thumbnails/19.jpg)
Other Sources of Differences
• Could also consider heterogeneity in vulnerability (e.g., nutritional status)– Increased odds diarrhea mortality with low
weight for age (Caulfield et al, 2004)– Increased risk of illness, for a given exposure– Increased risk of mortality, given illness
• May not affect the fraction preventable through sanitation, but would increase the number of severe cases preventable
![Page 20: UNC Water and Health Conference 2011: Heterogeneity in trial data: learning from difference, Professor Rick Rheingans, University of Florida and SHARE](https://reader033.vdocuments.us/reader033/viewer/2022051817/547d44e7b47959c5508b48b6/html5/thumbnails/20.jpg)
Differences in Impact?
Median 16.7%
Median
![Page 21: UNC Water and Health Conference 2011: Heterogeneity in trial data: learning from difference, Professor Rick Rheingans, University of Florida and SHARE](https://reader033.vdocuments.us/reader033/viewer/2022051817/547d44e7b47959c5508b48b6/html5/thumbnails/21.jpg)
Differences in Impact?• How does
sanitation impact vary across the space?
• Could heterogeneity in impact trial data help us understand how much?
• Same is likely true for other WASH interventions
![Page 22: UNC Water and Health Conference 2011: Heterogeneity in trial data: learning from difference, Professor Rick Rheingans, University of Florida and SHARE](https://reader033.vdocuments.us/reader033/viewer/2022051817/547d44e7b47959c5508b48b6/html5/thumbnails/22.jpg)
Salvador, Brazil Sanitation Trial
• Not a randomized trial – repeated cross-sectional study before and after city-wide sanitation project (Barreto et al, 2010)
• Took advantage of different levels of household and neighborhood change to estimate impact on childhood diarrhea and helminth infections
• Lessons from heterogeneity within the trial– Changes in community sanitation coverage were more important
than whether households received a connection– Impacts on helminth reduction were strongest among the poorest– Showed that intervention reduced the impact of SES on diarrhea
disparities
![Page 23: UNC Water and Health Conference 2011: Heterogeneity in trial data: learning from difference, Professor Rick Rheingans, University of Florida and SHARE](https://reader033.vdocuments.us/reader033/viewer/2022051817/547d44e7b47959c5508b48b6/html5/thumbnails/23.jpg)
Trial Heterogeneity and Generalizability
• Trials often focus on settings with high levels of burden and homogeneity– Increase the measureable impact– Reduce the size of the intervention needed
• However lack of heterogeneity within the trial can make it hard to generalize to a broader setting– External validity
![Page 24: UNC Water and Health Conference 2011: Heterogeneity in trial data: learning from difference, Professor Rick Rheingans, University of Florida and SHARE](https://reader033.vdocuments.us/reader033/viewer/2022051817/547d44e7b47959c5508b48b6/html5/thumbnails/24.jpg)
Example Deworming and Soil Transmitted Helminths
• Miguel and Kremer tested the impact of deworming for STH on educational outcomes in western Kenya
• Found that deworming can significantly reduce absenteeism (Miguel and Kremer, 2004); spillover effects; and increased long-term earnings (Baird et al, 2011)
• However the prevalence of STH was uniformly high within the study, compared to the rest of the country
![Page 25: UNC Water and Health Conference 2011: Heterogeneity in trial data: learning from difference, Professor Rick Rheingans, University of Florida and SHARE](https://reader033.vdocuments.us/reader033/viewer/2022051817/547d44e7b47959c5508b48b6/html5/thumbnails/25.jpg)
Translating to Heterogeneous Settings• Pullan and colleagues developed spatial estimates
of national burden to identify where mass treatment would be most appropriate
![Page 26: UNC Water and Health Conference 2011: Heterogeneity in trial data: learning from difference, Professor Rick Rheingans, University of Florida and SHARE](https://reader033.vdocuments.us/reader033/viewer/2022051817/547d44e7b47959c5508b48b6/html5/thumbnails/26.jpg)
Schistosomiasis Control in China
• Liang et al examined the impact environmental and chemotherapy interventions for Schisto control
• Developed mathematical models of transmission
• Used data from intervention trials to calibrate the models in different settings
• Identified patterns for generalizing
![Page 27: UNC Water and Health Conference 2011: Heterogeneity in trial data: learning from difference, Professor Rick Rheingans, University of Florida and SHARE](https://reader033.vdocuments.us/reader033/viewer/2022051817/547d44e7b47959c5508b48b6/html5/thumbnails/27.jpg)
Using Variability to Make a Difference
• Getting more out of trials– Analyzing factors modifying intervention effect
– Better characterizing mechanisms – connecting interventions to outcomes• Exposure and environmental studies
• Modeling and new analytical techniques
– Deliberate attention to external validity and generalizability in trial design
• Better translation– Using non-trial outcome data to better understand what happens in
more diverse settings
– Better characterizing contexts for which we would like to know the effect of interventions
– Better policy signals - to encourage more effective intervention selection