measuring & improving the lasting benefits of wash services
TRANSCRIPT
Deep Dive: Measuring & Improving the Lasting Benefit of WASH Services Jonathan Mitchell, Concentric Development Jonathan Wiles, Living Water International
1 WHAT to Measure?
1
Survey of Past Project Functionality
Living Water International
Program Excellence Group
June 6, 2013
» “How many Living Water-implemented water systems are still working today?”
All with GPS = 10,466
With GPS in Current Operational Areas = 9,435
System installations with GPS in Current Operational Areas = 5,717
System installations with GPS in Current Operational Areas not experiencing violent conflict or pandemic = 5,213
Table Discussion 1
Question: » What is YOUR “learning lab,” and what
question do YOU want to answer about the lasting benefits of WASH services?
10 minutes
2 HOW to Measure?
System installations with GPS in Current Operational Areas not experiencing violent conflict or pandemic = 5,213
Calculating Samples Country Population Sample
India 1332 100
Ethiopia 140 62
Ghana 54 37
Kenya 81 91
Rwanda 376 84
Uganda 290 69
Zambia 109 42
El Salvador 475 81
Guatemala 290 79
Haiti 109 58
Mexico 125 57
Nicaragua 288 77
837
» Attrition: Burkina Faso, Peru, Honduras, Zimbabwe.
» Random samples were calculated at home office & assigned to country operations.
» Samples were calculated for a 10% margin of error & 95% confidence interval at country level (yielding 2% global MoE).
Designing the Survey Logic
Begin Choose the site from your drop-
down menu
Do you observe any problems with the WP? q Limited flow q Poor siting q Potential contamination q Objectionable taste/smell q Other: ___________
YES
What is the water source? q Protected spring/gravity q Borehole/gravity q Other (specify):___
What type of hand pump? q AfriDev q India Mark II q Other (specify):___
What type of water point is it?
Tap/standpipe
Hand pump NO
Click here to capture geo-tagged photo of water point
being operated (flowing water visible if functional)
Is any water
currently flowing?
No
water
Water How much water does the WP produce? q Enough q Insufficient q Inconsistent/seasonal
Do local users observe any problems with the WP? q Limited flow q Poor siting q Potential contamination q Objectionable taste/smell q Other: ___________
Is this WP a primary drinking water source for local users?
When was the WP last functional?
What problems contribute to its non-functionality? q No/poor supply chain q No technical support q Insufficient management q No user demand/motivation q Poor siting q Other:___________
If/when this WP is functioning, would it be a
primary drinking water source for local users?
Who is responsible for managing the WP? q Village Water Committee q Institution (school, clinic…) q Local government q District/county government q Other: ___________
Do local users pay for the use/operation/maintenance of the WP? q Per container q Monthly fee q Annual fee q When system breaks down q Never pay q Other:__________
Do you have any final observations or comments?
Thank you!
Choosing Field Methods » Balancing Credibility & Cost (Internal vs. external data collectors) » Electronic vs. paper
» GPS validation » Multi-language support (English, Creole, Amharic, Spanish, Swahili)
Managing Field Costs Country Sample “Grant”
India 100 $4,500
Ethiopia 62 $2,000
Ghana 37 $1,500
Kenya 91 $2,500
Rwanda 84 $2,500
Uganda 69 $2,000
Zambia 42 $1,500
El Salvador 81 $2,000
Guatemala 79 $2,000
Haiti 58 $2,000
Mexico 57 $2,000
Nicaragua 77 $2,000
837 $26,500
» Internal “Grant” logic » 31-50: $1,500 » 51-70: $2,000 » 70+: $2,500
Collecting the Data Country Population Sample Collected
India 1332 100 94
Ethiopia 140 62 62
Ghana 54 37 32
Kenya 81 91 84
Rwanda 376 84 84
Uganda 290 69 66
Zambia 109 42 39
El Salvador 475 81 80
Guatemala 290 79 79
Haiti 109 58 53
Mexico 125 57 49
Nicaragua 288 77 70
837 792 (95.7%)
Table Discussion 2
Question: » What kind of survey or study would help YOU
answer YOUR question about the lasting benefit of WASH services?
15 minutes
3 WHY to Measure (and Improve)?
Analyzing the Data: Main Findings
Functional (77%)
Non-functional (17%)
Limited functionality (6%)
• Representative sample of Living Water systems over 15 years (1999-2013) • Study was carried out in Q4 2014, including 792 water points in 12 countries • 95% confidence level, 2% +/- margin of error • Mechanical functionality only
Analyzing the Data: Regional Variances
88% 67% 78%
Latin America & Caribbean Africa India
Analyzing the Data: Technology Choice
77%
Protected Spring
Hand Pump
Borehole /Piped
Rainwater Harvesting
78% 78% 75%
Financial Contributions
YES NO
YES 146 45
NO 99 77
No Payment Scheme With Payment Scheme
76% 56%
Africa Region
Community Management
0
50
100
150
200
250
Committee Institution
75% 0
10
20
30
40
50
60
Committee Institution
No or Low Functionality Functional
56% 85% 70%
Africa India
Why do water systems fail? » Recurring themes: » Poor siting or low demand (possibly indicating poor
pre-implementation community mobilization)
» Poor or under-equipped community management » Poor technical support and supply chain
Impact on Organization
Pursue Excellence
Makes sector knowledge “real” to organization
Builds knowledge & will to improve program practices
Builds culture of measurement & learning at all levels
Creates a sense of urgency to improve policies & systems
Table Discussion 3 Question: » If YOU could accurately answer YOUR
question about the lasting benefit of WASH services, what would the implications be for the public, your board, your donors, and (especially) your program implementation?
10 minutes
0%
10% 10%
40%
31% 31%
23%
8% 8%
17% 17%
9%
22%
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
Pastor lives in village
In last week > 1 week > 1 month > 6 months
Time since last pastor visit
Bihar
Jharkhand
Total
0% 0% 4% 5%
20% 20% 18%
39% 40%
25%
75%
20%
17% 28%
21%
11%
60%
44%
0%
20%
40%
60%
80%
100%
120%
1 2 3 4 5 6 7 8 9 11
Well Age Years
Repair Needs By Well Age
Minor Repair
Major Repair
41% of wells (80) were in need of repair
2015 SUMMIT on EXCELLENCE IN WATER, SANITATION & HYGIENE