managing the transition between emergency · pdf filemanaging the transition between emergency...

MANAGING THE TRANSITION BETWEEN EMERGENCY WASH & DEVELOPMENT

AGENDA:

An evaluation of ECHO- Funded Water, Sanitation and Hygiene (Urban WASH- II)

Project in Monrovia, Liberia

Whs

WASHealth Solutions

Stephen Ntow

[email protected]

Cover Page: Picture of girls harvesting water from a liking overhead water tank

February 2013

mailto:[email protected]

14022013 – Final Report Evaluation of Urban WASH-II Monrovia

2

Profile of the Consultant Stephen Ntow MSc., ADHE, Dip. PH. Stephen Ntow is currently, the Director of WASHealth Solutions (WHS), an international development consulting firm. He is a public health practitioner with specialization in Health Education and Health Promotion. He was the Country Representative of WaterAid, an international NGO specialised in water and sanitation based in the United Kingdom. He had previously served in the public sector, Ministry of Health and later Ghana Health Service (GHS) for 18 years in various capacities. He is associated with landmark operational research and training programmes that provided the much needed drive behind the health sector reforms in Ghana between the late 1980 to the early 2000s. His experience span diverse development areas including environment, population studies, HIV/AIDS prevention education, mainstreaming gender, ensuring equity and inclusion in pro-poor programmes. Stephen has undertaken several multi-sector training in health education and health and hygiene promotion. He has lead the design, implementation and monitoring of several public health, water, sanitation and hygiene promotion projects. He has also undertaken research assignments; baseline and evaluation of programmes in many African countries. The outcomes of many such assignments have since been published and turned into invaluable learning products. Others have greatly influenced sector policies and initiated reforms. Profile of the assignment The object of the assignment is an evaluation of European Commission for Humanitarian Aid (ECHO) 2012 Urban WASH project targeting cholera prone urban slums in Monrovia (urban WASH II). The assessment was conducted between December 2012 and February 2013. Data collection was carried out between 7th and 29th January 2013. The design, processing and analysis were done in collaboration with Liberia Institute of Statistics and Geo-Information Services (LISGIS)

Above: Picture of a newly constructed school Toilet at Dr. H W Reeves Memorial School, Monrovia.


3

Table of Contents

Contents

Executive Summary .............................................................................................................. 6

List of Acronyms and Abbreviations .................................................................................... 14

1.0 Introduction ................................................................................................................... 15

1.1 Background to the assignment ...................................................................................... 15

1.2 Objectives of the assignment ........................................................................................ 15

1.2.1 Objectives ............................................................................................................... 16

1.3 Targets of the assignment ............................................................................................. 16

1.4 Methodology ................................................................................................................. 16

1.4.1Sampling and sample size ....................................................................................... 16

1.4.2 Data collection methods ......................................................................................... 17

1.4.3 Respondents and stakeholders .............................................................................. 17

1.4.4 Enumerators and field supervisors training ............................................................. 18

1.4.5 Quality assurance and coordination ........................................................................ 18

1.4.6 Data processing and management ......................................................................... 18

1.4.7 Specific activities expected as part of the assignment per ToR ............................... 18

1.5 Gender and ethical considerations............................................................................. 19

1.6 Audiences .................................................................................................................. 19

1.7 Work Plan ..................................................................................................................... 19

1.7 Validation workshop ...................................................................................................... 19

Section 2: Findings ............................................................................................................. 20

2.0 Introduction ................................................................................................................... 20

2.1 Profile of respondents ................................................................................................... 20

2.2 Project outputs and outcomes ....................................................................................... 21

2.3 Water ............................................................................................................................ 21

2.2.1 Household Drinking water ....................................................................................... 21

2.2.3 Functionality and water quality ................................................................................ 22

2.2.4 Water storage containers and treatment of drinking water ...................................... 23

2.2.5 Estimated volume of water per household .............................................................. 23

2.3 Sanitation ...................................................................................................................... 24

2.3.1 Access to latrines ....................................................................................................... 24

2.3.2 Distance and time to access latrine ........................................................................ 24

2.3.3 Accessibility: physical and financial ........................................................................ 24

2.3.4 New and emerging challenges ................................................................................ 25

2.4 Household garbage (Solid waste) ................................................................................. 25

2.5 School WASH project .................................................................................................... 26


4

2.6 Hygiene Promotion ........................................................................................................ 27

2.7 Handwashing ................................................................................................................ 27

2.7.1 Availability of hand washing facility............................................................................. 27

2.7.2 Handwashing with soap .......................................................................................... 27

2.8 Perceived decline in prevalence of diarrhoea ................................................................ 29

2.9 Hygiene behaviour change motivators .......................................................................... 30

2.9.1 Preferred channels for hygiene education ............................................................... 31

2.9.2 Benefits and satisfaction ......................................................................................... 31

2.9.3 Sustainability .......................................................................................................... 32

Section 3: Conclusion, Lessons learnt and Recommendations ........................................... 35

3.0 Conclusions .................................................................................................................. 35

3.1 Coherence .................................................................................................................... 35

3.2 Relevance ..................................................................................................................... 35

3.3 Beneficiary satisfaction.................................................................................................. 36

3.4 Efficiency ...................................................................................................................... 36

3.5 Effectiveness ................................................................................................................. 36

3.5.1 Performance of IPs ................................................................................................. 36

3.6 Accountability ................................................................................................................ 37

3.7 Sustainability ................................................................................................................. 37

3.8 Lessons Learned........................................................................................................... 38

3.9 Recommendations ........................................................................................................ 38

References ......................................................................................................................... 42

Appendixes ......................................................................................................................... 43

Appendix 1: Excerpts of TOR: Objectives of Assignment ................................................ 43

Appendix 3: Proposed Work Plan – Evaluation of Urban WASH – II ................................ 67

Appendix 4: Map Showing Parts of Montserrado and Monrovia ....................................... 68

Appendix 4a: Map of Monrovia Showing Project Communities in shades ........................ 68

Appendix 4b: Geo-Referenced Map of Monrovia ............................................................. 69

Appendix 5: Project indicators and baseline results ......................................................... 70

Appendix 6: List of Implementing Agencies ..................................................................... 73

Appendix 7: List of Persons Contact During the Assignment .......................................... 73

Appendix 8: Summary of Project Outputs ........................................................................ 74

Appendix 9: Financial Sustainability Estimates for WASH Facilities and Services ........... 75

Appendix 10: Implementing partner performance related results ..................................... 77

Table 1: Reported daily income (US$ Equivalence) - ………………………………………..19

Table 2: What education have you received as a result of the Urban WASH project........... 26

Table 3: Results - State of Latrines and User Observation ………………………………….. 27


5

Table 4: Major household WASH decision makers ... ...........................................................29

Table 5: What benefits or changes as a result of Urban WASH WASH – II... ......................30

Table 6: Project indicators and Summary Results Card ……………………………………...33

List of figures

Fig1: Household sizes .........................................................................................................19

Fig 2: Main sources of drinking water ..................................................................................21

Fig 3: Time & distance to acess water .................................................................................21

Fig 4: Evidence of treating household water (Pool Test)......................................................22

Fig 5: Main household defecation options ...........................................................................23

Fig 6: Distance and time to access latrine …………............................................................23

Fig 7: Household garbage disposal methods ......................................................................25

Fig 8: Reported regular handwashing compared ................................................................27

Fig 9: Reported diarrhoea among children .........................................................................29

Fig 10: Preferred channels of communication ...................................................................30


6

Executive Summary

In the year 2012 UNICEF in partnership with NGO partners embarked upon implementation of an ECHO-Funded Urban WASH project in deprived communities around Monrovia. Although the project was contracted under emergency WASH conditions there were far reaching development and sustainability considerations at all levels of the project. The project has become known as Urban WASH-II (UW-II: SM120084). The project was meant to deliver sustainable water, sanitation and hygiene promotion services to deprived urban slums to ensure continuous availability of the facilities and voluntary change in WASH behaviours long after the project is over. This assignment therefore is to establish the extent to which the facilities and processes proposed have been met as part of the UW-II project. 1. Objectives of the assignment The purpose of the assignment is to provide data which should enable UNICEF report to ECHO on the indicators and targets listed in Urban WASH-II SM120084. In addition the evaluation was meant to assess the impact, coverage, coherence, relevance/appropriateness, effectiveness and efficiency of the project.

1.1 Specific objectives i) To provide analysis of urban WASH II (SM120084) project achievements and

setbacks against planned results, assess the project’s intervention logic, promote

accountability, and assist future improvements in performance.

ii) To provide data which will allow UNICEF to report to ECHO on the indicators and

targets listed in the urban WASH II: SM120084 proposal.

2. Methodology A stratified random sampling method was adopted to represent different sections of communities and to reflect the following characteristics of the population under review. These include; population, social, literacy (formal education), cultural, economic indicators and also access to WASH facilities and services. A convenient sample of 456 was drawn from 12 out of 18 communities selected at random. A combination of conventional and participatory methods was used to gather both quantitative and qualitative. The software Statistical Package for Social Sciences (SPSS) was used to process and generate frequencies and cross tabulations. This was done in collaboration with Liberia Institute of Statistics and Geo-Information Services (LISGIS).

3. Findings

The major source of primary data was collected from mothers with children less than five

years old. Almost all of them have lived in the community more than six months preceding

the date of interview. A period considered long enough to express relevant opinions about

Urban WASH – II project. They have a varied background ranging from none formal

education to tertiary level education.

Almost all the planned outputs and services have been completed and those under

construction are close to completion but the intended contribution to reduction in suspected

cases of cholera is yet to be realised in the numbers reported at the Cholera Unit.


7

3.1 Project outputs and outcomes impact,

Key actions and services provided as part of project’s requirements include the provision of

safe drinking water, promotion and free distribution of WaterGuard (water treatment

chemical), provision of improved shared latrines and hygiene promotion services through

interpersonal communication, print and electronic media. An estimated 30,000 beneficiaries

was reported. A summary of key interventions carried out has been attached as appendix 8

of the main report.

Information available suggests a steady increase in suspected cases of cholera from the

affected communities and the country as a whole. This situation may be due to increased

vigilance in reporting and documentation of suspected diarrhoea diseases. From an annual

count of 1277 cases by the end of the year December 2011, the number of cases increased

to 4730 by the end of December 2012 as presented in the matrix below.

Reported Diarrhoea Diseases at Cholera Unit 2011-2012

Year 2009 2010 2011 2012

No.of Cases 1052 1546 1277 4730

3.2 Household drinking water

The main sources of drinking water for most households are safe and have increased from

70.1% (226) at baseline to 91.6% (418). The situation shows an improvement over the

baseline status. The number of households relying on unprotected sources also increased

marginally. More than 73% (337) of respondents travel less than one hundred meters or less

than 30 minutes to access drinking water. The distance to drinking water sources has

reduced compared to baseline indicators. It takes the remaining up to 500 meters to access

drinking water but 6.1% (28) are unable to estimate how long it takes them to do same.

At the start of the project, 39.1% (126) compared to 36% (170) of respondents indicated that

their main sources of drinking water did not last all year round. Out of this 46.2% (67) fall

back on unprotected sources for the rest of the year. Although this is an improvement over

baseline statistics, there still are lessons to be learnt in order to improve upon the current

situation.

The use of appropriate containers, the recommended 22 litre (5 gallons) container with caps

to store drinking water has increased appreciably from 76.4% (243) at the start of the project

to 84.6% (386).

The proportion of respondents reported regular treatment of drinking water increased by

18% above baseline indicators. The households whose drinking water showed some

presence of treatment with WaterGuard but recorded values lower than the recommended

levels increased from about 14% (45) to 34% (155). However, results of recommended

dosage of chlorine at the various homes indicate much lower percentage 4.4% (20). This is a

decline from 12.7% (41) as presented in the table below.


8

3.3 Estimated volume of water per household

Most households 75.4% (344) used containers that hold 20 litres of water or more. Majority

of them 61% (277) drew water twice a day hence the estimated volume of water drawn and

possibly used each day per household and individual is about ≥ 40litres and 8 litres

respectively. This situation is a marked improvement in access to drinking water compared

with 4.4 litres per/head/day in the era before the project. 4. Sanitation 4.1Access to latrines More than one third (195), of respondents had access to Family latrines (WCs, pour flash, septic tank latrines). As many as about 50% (217) use shared latrines and about 8% (36) resort to open defecation. A comparison between the current statistics and those reported in the baseline report shows an improvement as illustrated with the bar chart below. Access to family (household) and shared latrines has decreased. Open defecation rate has also declined. A comparison between the current statistics and those reported in the baseline report shows an improvement as illustrated with the bar chart below.

4.2 Household garbage (Solid waste)

The outcome of education on garbage disposal was mixed. Only 7.7% (35) respondents kept

their household garbage in a manner that seems appropriate. It represents a decline of

about 5.7%. Majority 60.7% (277) kept unwanted household materials in open containers

exposing the contents to flies. About 18.4% (84) practiced dig and bury.

No presencePresence of

chlorine

< 0.2 FreeResidualChlorine

Residual Chlorine ≥ 0.2mg/L

I do not / nowater to treat

Not Applicable

32%

14% 12.7%

41.3% 44.7%

34%

4.4%

11.20%

5.7%

Evidence of treating household water (Pool Test)

Baseline % Post Interv'n %

39.4 32.3

1.9

18.6

0.9

42.8 47.6

0.9

7.0 1.0

Family Latrines Shared latrine Dig and bury Open defecation Others

Main household defecation options (%) Baseline Post Interv'n


9

4.3 School WASH

School health clubs were formed and members trained in 37 schools. The clubs are the

centre of hygiene education and other water and sanitation promotion activities in the

schools. There are positive indications that activities of the SHCs could be sustained if

effective monitoring is carried beyond the project phase. However, the officials of the

Ministry of Education are not aware of the schools selected and will like to be part of the

selection process in future. Some of the schools are effectively combining the services

provided by janitors with school children’s learning and self help activities. These will help

provide practical experiences for the children and must be emulated by other schools. Some

negative WASH practices were also observed and these must be changed.

Some of the school health club (SHC) coordinators punish children to undertake

water and sanitation services. This may send negative perceptions that WASH

services are not desirable. This must be stopped and other forms of punitive action

should be considered.

Some of the schools (including communities) have provided containers in the toilets

for holding anal cleansing materials. This practice can facilitate transmission of

diarrhoeal diseases and must be stopped.

5. Hygiene Promotion Marked progress has been recorded in some departments of the hygiene promotion including awareness, increased knowledge and some changes in behaviours. 5.1 Handwashing with soap Access to handwashing facilities is a factor which enhances opportunities for people to practice that action but 45.4% of households did not have the facility. Almost all respondents 95.6% (436) indicated that they regularly washed their hands with soap. Reported handwashing at critical moments has been captured and compared with baseline statistics. The bar chart below shows an increase in knowledge of hand washing in general but has declined when it came to washing hands before feeding a child. In future hygiene promotion may have to focus on this critical moment alongside the general message on the benefits of regular handwashing with soap.

88%

13%

43%

11%

70%

26%

2%

94% 96%

81%

26% 25%

72%

29%

18%

94%

Reported Regular Handwashing Compared

Baseline Post-Interv'tn


10

5.2 Reported versus observed handwashing Although 72% (314) reported regular hand washing with soap after using a toilet, discreet

observations by enumerators unknown to another set of randomly selected subjects did not

support this assertion. Only 36.3% (53) persons actually washed their hands after using the

latrine.

5.3 Perceived decline in prevalence of diarrhea Mothers of children who are less than 5years old perceive a decline in number of diarrhea bouts since the intervention. They compared the period six months before the intervention and another six months after the start of the intervention. The findings have been presented as fig 9 in the main report.

5.4 Potential sustainability A limited assessment of community WASH committees and an estimation of finances; income and expenditure of the water and sanitation facilities suggest a potential sustainability of the project if effective monitoring and capacity issues are addressed. 6. Conclusions, recommendations and lessons learned To a large extent the project has achieved most of the original objectives. The project has created management capacity and it is financially sustainable in the short to term. It’s objective to contribute to reductions in suspected cholera and other diarrhea diseases have however not been achieved. Also, some specific targets have performed below average performance of other sections. The least performing areas included:

a. Garbage collection and disposal.

b. Promotion, proper use and monitoring the use of WaterGuard.

c. Increase in diarrhoea diseases despite the intervention.

The project was found to be coherent having complied with most sector policies and the provisions in project proposal. It was found to be relevant and appropriate as it sought to reach the poor and diarrhea prone communities of Monrovia. Monitoring arrangements put in place to ensure project accountability was followed closely. The project was also found to be effective and efficient. There is no information on how lessons from the previous project (urban WASH-I) have been adopted and systematically implemented as lessons to enhance the outcome of the current and future project. Some WASH committees lack some capacities and so do the communities lack the ability demand and hold their committee to account for their stewardship. Some sector agencies have expressed dissatisfaction with their partial involvement during project implementation.

7. Lessons Learned

There is no information on how lessons from the previous project (urban WASH-I) have been

adopted and systematically implemented as lessons to enhance the outcome of the current

and future project. As part of this report however, the following are lessons identified so far:

1. Although sector policies are widely circulated, the mere awareness does not

necessarily ensure compliance.

2. The creation and subsequent training of WASH committees will not naturally lead to

effective performance. They need some hand-holding, support and post

implementation/construction monitoring to ensure good performance.

3. When poor communities have several WASH needs. Working with them to identify

and solve initial problems empowers them to identify and articulate new and

emerging issues.


11

4. The implementation of WASH projects do not immediately and directly relate to

declines in suspected cases of Cholera and diarrhoea diseases.

8. Recommendations

8.1 Reviewing project outcomes for purposes of learning

Obviously some project components did not perform as expected. It will be appropriate for

UNICEF and IPs in consultation with other sector partners to review the underperformed

components of the project with the view to identifying causes, learn lessons and share with

other stakeholders. They include: d. Garbage collection and disposal.

e. Promotion, proper use and monitoring the use of WaterGuard.

f. The increase in diarrhoea diseases in the affected communities despite the WASH

interventions implemented in the selected communities. This should be done in

consultation with the Health Unit of Child Survival section of UNICEF and other

Government public health agencies.

8.2 Participatory feedback sessions with project communities

The high level of satisfaction expressed by the communities over the facilities constructed

under the project and related outcome is a show of confidence. It will however, be

necessary for IPs to organise feedback sessions with the communities. The sessions should

adopt participatory methods to learn from them specifics of what went well and worth

replication. This should provide opportunities to learn about some knotty points and

challenges which can be improved.

8.3 The need to improve targeting and focusing on the poor UNICEF should lead a process of developing a set of criteria in consultation with other

sector partners to help target and focus on the very poor and persons suffering from other

forms of disability and impairments even under the current project to ensure that the poorest

are reached under this and subsequent projects.

IPs should seek assistance from development Partners including UNICEF to help them

identify needs of very poor persons who cannot afford to pay for water and user fees at the

various points attract further discussions, identification and proper targeting of such people

and community actions implemented to provide them access.

8.4 Specific sector requirements and further consultative process a. UNICEF should include in the PCAs the need for implementing partners to consult

the Hydrology Unit of the Ministry of Land, Mines and Energy to:

Select appropriate locations to site hand-dug-wells to drill boreholes.

Submit relevant data including well logs to the Hydrology Unit of MLM&E.

b. Implementing partners are also expected to conduct and submit water quality test

results after completion of water facilities.

c. As part of future project designs, it is recommended that women, children and

persons living with physical disability should be actively engaged at all levels of the

project cycle namely, concept, planning, implementation, monitoring and evaluation.

d. It is also of utmost importance to emphasize post construction engagement with

target audiences as mention under the previous section labelled “c”.


12

e. Under the leadership of UNICEF the School Health Unit he Ministry of Education

should be actively involved in the selection of beneficiary schools to facilitate

monitoring after project completion.

f. An active involvement of all Government agencies at the point of entering and exiting

communities is recommended to ensure a smooth handing over and post project

implementation and accountability.

8.5 Regular review of IP capacities and effective monitoring of PCAs

The delays in meeting some project timelines require responsive partnership and

implementation arrangements. UNICEF should introduce punitive clauses into PCAs as

motivation to get implementing partners to perform as expected.

It is also recommended that UNICEF in consultation with its Government Partner agencies

should assess capacities of partners and the results used to select IPs for each project. One

of the important indicators to be considered under such circumstances should be the ratio

between existing capacities on one hand and volume of existing demands on those

capacities. This should be used to review or develop terms of PCAs.

UNICEF should ensure that monitoring of IP performances should be followed with

responsive management decisions and implementation of such decisions.

8.6 Further capacity building for WASH committees

It recommended that UNICEF will invest in post project monitoring and capacity building for

WASH committees and community leaders to address specific issues including:

Mobilisations of project communities to take necessary actions preventing or likely to

affect access to WASH services and management of WASH facilities provided under

the project. Particular reference the example in Struggle Community where the. The

newly constructed latrine is grossly underutilised.

Proper constitution of community WASH Committees and to ensure that these

committees:

o Meet regularly and keep proper records especially financial records.

o Compile and render proper accounts of WASH facility user fees to the

community.

o Open bank accounts and pay-in WASH facility user fees.

Further mobilization of communities to demand accountability.

8.7 School WASH promotion issues School health clubs (SHCs) are at the centre of hygiene education and WASH promotion activities in the schools. There are positive indications that the performance SHCs could be sustained if effective monitoring is carried beyond the project phase. Some of the schools are effectively combing the services provided by janitors with school children’s WASH learning and self help activities. These will help provide practical experiences for the children and must be emulated by other schools. The negative WASH practices observed and reported in the previous sections will require urgent and collective remedial actions to ensure sustainability of the SHCs and continuous availability of the facilities provided. They are:

1. The state of some schools latrines outside the UW-II project was found to be unhygienic. These schools are reported to rely mainly on janitors who do not seem


13

sufficiently motivated. A more stringent monitoring arrangement be establish to promote hygiene. Students should be motivated to use the facilities properly and become part of the cleansing processes.

2. School authorities should ensure that minor repairs and replacement of little but important components like faucets/taps should be done regularly. The School Health Directorate of the Ministry of Education in collaboration with the IPs should encourage school authorities to take up this responsibility.

8.8 Visibility for European Commission for Humanitarian Aid (ECHO) Most of the facilities constructed under UW-II especially, posters, latrines and water systems carry the marks and insignia of the donor – ECHO. This level of acknowledgement has been done with other printed materials as provided in the contract documents. However, two of the hand-dug-wells sampled do not have these marks of acknowledgements and needs to be addressed.

New toilets help increase school enrollment

Rev. Henry W. Reeves Memorial School is one of 37 schools which benefited

from the provision of WASH facilities under the Urban WASH-II project funded by

ECHO. This schools has a population of 670 students. Previously, both teachers

and students relied on 4 four toilet cubicles. This was not enough and thus

presented many problems. According to the Principal of the school, only few

people could access the facility during short recreational periods. Pressure on the

four cubicles also meant there was limited time for proper use and the cleaning

was not well done. “It was also a great problem to the bigger girls who needed

privacy during some periods of the month. As a result, their attendance was not

regular. I believe this may have affected their performance in the past years. We

are grateful to UNICEF. Since the new toilet [8 cubicles] were constructed and the

School health club trained, enrolment in the school has increased”.

- James T. W. Dogba, Principal, Dr. HM Reeves Memorial School, Monrovia.


14

List of Acronyms and Abbreviations

BH/HDW Bore Hole / Hand-Dug-Well

CHT County Health Team

CODES Community Development Services

CLTS Community- Led Total Sanitation

CWW Concern Worldwide

ECHO European Commission Humanitarian Organisation

FGD Focus Group Discussion

FAAL Foundation For All Ages Liberia

GoL Government of Liberia

HWTS Household Water Treatment and Storage

HWWS Hand Washing With Soap

IP Implementing partners

JICA Japan International Cooperation

JMP Joint Monitoring Platform

KAP Knowledge Attitude and Practice

KII Key Informant Interview

LDHS Liberia Demographic and Health Survey

LICH Liberia Care for Humanity

LISGIS Liberia Institute of Statistics and Geo-Information Services

LWSC Liberia Water and Sewerage Corporation

MoE Ministry of Education

MoG&D Ministry of Gender and Development

MoHSW Ministry of Health and Social Welfare

MoPW Ministry of Public Works

NGO Non-Governmental Organizations

PCA Program Cooperation

PRS Poverty Reduction Strategy

PSI Population Services International

SPSS Statistical Package for Social Sciences

ToR Terms of Reference

UNICEF United Nations Children Fund

WASH Water Access Sanitation and Hygiene


15

1.0 Introduction

In the year 2012 UNICEF in collaboration with its NGO partners embarked upon implementation of an ECHO-Funded Urban WASH project in deprived communities around Monrovia. The project has become known as Urban WASH-II (UW-II: SM120084). The project was meant to deliver sustainable water, sanitation and hygiene promotion services to deprived urban slums to ensure continuous availability of the facilities and voluntary change in WASH behaviours long after the project is over. This assignment therefore is to establish the extent to which the facilities and processes proposed have been met as part of the UW-II project. As part of the project implementation arrangements, an external evaluator is expected to determine the potential sustainability of projects Outcomes, identify gaps and challenges to future urban interventions. The structure of this report was shaped largely by the Adapted UNEG Evaluation Reports Standards (UNICEF 2010) and the terms of reference issued to guide the implementation of this assignment.

1.1 Background to the assignment

Liberia is in a period of transition after over 14 years of conflict and humanitarian

emergencies. The effects of these years were devastating to most of its sectors:

infrastructure was heavily damaged and poverty rates are extremely high. Health and other

social indicators are also very poor with under-five and maternal mortality rates amongst the

worst in the world.

Between 1990 and 2008 the percentage of Liberia’s population with access to improved

water sources in urban areas declined from 86% in 1990, to 82% in 2000, and 79% in 2008,

due to conflict related rural to urban migration, and the degrading of urban pipe based water

supply systems. The population of Montserrado has increased by 44% over the last 25

years from 491,078 in 1984 to 1,118,241 in 2008, mainly in the capital city of Monrovia

(Census 2008). Indeed, the city has experienced a simultaneous decline in pipe based

water supply of 73% from 16 million gallons/day in 1985 to 4.3 million gallon/day in 2008

(LWSC).

At the start of the project, baseline indicators showed an improvement upon previous research findings as you will find in subsequent sections of the report. According to the JICA study, the average water consumption in targeted urban slum is around 9 l/c/day (about 2 gallons per household of 5 persons. Indeed a number of peri-urban communities presently rely on communal tap-stands and hand-pump wells for drinking. Given that the ground water level fluctuates greatly with season, many wells dry up and cannot be considered as safe and sustainable water sources. Dwellers usually use hand-pump wells and hand-dug wells separately for drinking and other usage.

Although the project was contracted under emergency WASH conditions (Sphere Project

2011) there were far reaching development and sustainability considerations at all stages of

the project (GoL 2010). These included; gender, inclusion, management and financial and

sustainability considerations.

1.2 Objectives of the assignment The purpose of the assignment is to provide data which should enable UNICEF report to ECHO on the indicators and targets listed in Urban WASH-II SM120084. In addition the evaluation will assess the impact, coverage, coherence, relevance/appropriateness,


16

effectiveness and efficiency of the ECHO-funded urban WASH project: SM120084 grant in Monrovia.

1.2.1 Objectives iii) The external evaluation will provide analysis of urban WASH II (SM120084)

project achievements and setbacks against planned results, assess the project’s intervention logic, promote accountability, and assist future improvements in performance.

iv) The external evaluation will provide data which will allow UNICEF to report to

ECHO on the indicators and targets listed in the urban WASH II: SM120084 proposal.

1.3 Targets of the assignment In order achieve the objectives set above the external evaluation will assess: o The impact of the project -measure and assesses the extent to which the

project contributed towards achievement of the principle and specific objectives detailed in the project proposal/s.

o The coverage of the project -measure and assesses the extent to which the project reached groups targeted in the proposal, including children and their caregivers.

o the coherence of the project - assess whether the project was implemented in line with the plan outlined in the proposal

o the appropriateness/ relevance of the approach - Assess whether the project was in line with identified needs and priorities of the target population; Assess the satisfaction of the beneficiaries with the project; Assess the level of beneficiary involvement including women in the project design, implementation and monitoring- participatory approach developed/ used for assessment and project implementation?

o The efficiency of the project - Measure the qualitative and quantitative outputs achieved in relation to the inputs and compare strategies to see whether the most efficient approaches were used; assess the level of cost effectiveness/ appropriate use of the budget.

o the effectiveness of the project - Assess any factors that affected the effectiveness of the project and how the factors were addressed during implementation

o the accountability of the project -Assess monitoring systems put in place for quality control

o The sustainability of the project -Measure the extent to which the beneficiary bears ownership of the project, local capacities is strengthened and sustainable management arrangements have been put in place.

o lessons learnt conclusions and recommendations

1.4 Methodology

1.4.1Sampling and sample size A stratified random sampling method was used to represent different sections of communities and to reflect the following characteristics of population under review. These include; population, social, literacy (formal education), cultural, economic


17

indicators and also access to WASH facilities and services. A convenient sample of 456 was drawn from 12 out of 18 communities selected at random.

1.4.2 Data collection methods

The Consultant gathered and triangulated data from three main sources. They included data

from review of literature, collection of primary data through interviews and observations. A

description of each of these has been presented below:

Review documents (literature review). Information was gathered and reviewed to place the evaluation in its appropriate context. Materials reviewed are those that provided the peculiar WASH situation in Liberia and the selected communities in Monrovia. Some references have also been made to some experiences in other countries in order to situate the evaluation within a global urban and emergency WASH framework. The thematic issues covered demography, geography, factors determining sustainable access and barriers to the delivery of WASH services for different segments of deprived populations including the poor and persons with disability. Great care was devoted to examining the contexts of a country in emerging from war. .

Interviews and others primary data collection instruments

Information was collected from primary sources from multiple stakeholders and

intended beneficiaries of the WASH project using about five different instruments. The

survey instruments were developed or adapted after reviewing project documents and

some existing measurements including MICS4 (2012) and Liberian population census

enumeration instruments (GoL 2007). The Consultant developed and deployed the

following instruments for primary data collection: o Key informants interviews (KII) guides - 47 o Focus Group Discussants (FGD) - 12 o Structured Interview (Households) questionnaires - 456

Observation guides For purposes of triangulation selected audiences including service providers, contractors, women, children and other users of the facilities provided under the project were observed in their natural settings. Information from sources like these provided opportunities to compare on the one hand perceptions and responses elicited, and on the other hand practices and behaviours.

1.4.3 Respondents and stakeholders

Information was solicited from the individuals and agency representatives mentioned below:

• Community leaders • Members of WASH committee • Ministry of Gender • Ministry of Public Works (MoPW) • Ministry of Education (MoE) • Ministry of Health and Social Welfare (MoHSW) • Liberia Water and Sewerage Corporation (LWSC) • Monrovia City Council (MCC) and • Implementing partners (local INGOs & NGOs). • School pupils and authorities. • Representative of National Union of Disabled persons.


18

1.4.4 Enumerators and field supervisors training

A two-day refresher training sessions for enumerators and field supervisors was

carried out for persons previously associated with the baseline assessment. The

relatively short duration was meant to revise and learn lessons from the previous

assessment. A one day field testing session was embarked upon after which all draft

all data collection instruments were finalised. The topics taught or discussed during the

training sessions were;

• Objectives of the assignment. • Introduction to project and other background information including donors

UNICEF and ECHO. • Concepts and principles of formative studies. • Updates on characteristics of project communities • Data collection • methods and interviewing techniques • Review and revision of data collection tools. • Role plays on the use of various data collection tools. • Field trip and administration of draft tools. • Revision of data collection tools and process • Preparation /developing plans for actual field activities and supervisory

arrangements and reporting channels.

1.4.5 Quality assurance and coordination

The Consultant used high professional standards at all levels to assure quality of the

assignment. These included but not limited to:

Diligent literature review.

Rigorous training of enumerators.

Strict supervision during data collection.

Adherence to high standards of compilation, labelling and handling of data.

1.4.6 Data processing and management

Data processing was done using SPSS. It was done in collaboration with Liberia Institute

of Statistics and Geo-Information Services (LISGIS). The aim is to ensure compliance to

national standards, ensure acceptability of the findings and to enhance the chances of

implementing recommendations made at the end of the assessment.

1.4.7 Specific activities expected as part of the assignment per ToR

a. Collection and analysis of secondary information including: donor proposal,

project documents, progress reports, distribution and sales records of WaterGuard, data from health facilities, data collected by NGOs, financial records of water and sanitation facility management entities, IEC materials used, GPS data, etc.

b. Collection and analysis of primary information through: i. Structured interviews, focus group discussions, questionnaires, and

meetings with key stakeholders including: NGO partners, and UNICEF


19

project staff, local representatives, local health workers and

community leaders, beneficiaries, non-beneficiaries etc.

ii. Structured observation of quality of facilities, usage, and hygiene

practices etc.

iii. Sample collection and analysis of drinking water to measure free

residual chlorine.

c. Provide evaluation and analysis of progress against targets and objectively verifiable indicators detailed in the proposal for urban WASH II: SM120084 – Refer section 11 of Appendix 1 for TOR of the assignment and Appendix 4 for project indicators and baseline results against which evaluation results will be compared.

1.5 Gender and ethical considerations

Socially empowering techniques were used during the data collection process. This may be extractive and stressful for some respondents. Great caution was exercised in seeking respondents’ consent to provide responses. This was done purely on voluntary basis. Above all, the processes and issues discussed were not sensitive and did not affect the sensibilities of respondents in any negative way. The specific needs of the poor and disadvantaged segments of the population were comprehensively assessed and addressed at all stages of the assessment.

1.6 Audiences

The primary audience of this evaluation were residents of the cholera prone urban slums

around Monrovia. Others were WASH actors in Liberia including UNICEF, the donor, officials

of the Government of Liberia and civil society organization (CSOs). The secondary

audiences of the evaluation are stakeholders of UNICEF actors outside the WASH sector of

Liberia, including UNICEF Regional Office and donor representatives. Most of the audiences

are familiar with the project objectives and locations but refer appendix 3 to review various

maps including a geo-referenced map showing project locations and implementing partners

involved.

1.7 Work Plan

The evaluation is expected to commence in December 2012 and concluded by the end of 12 February 2013. Do find the details in the Appendix 2.

1.7 Validation workshop

A stakeholder workshop was held verify, clarify and to reach consensus among stakeholders

on different perspectives documented from the different respondents. All the findings were

corroborated by stakeholders. Additional information was provided all of which have been

incorporated in the final report.


20

Section 2: Findings

2.0 Introduction This section contains key findings from the evaluation, some background information on the respondents, WASH outcomes as a result of the interventions, key indicators contained in ECHO project proposal and specific target prescribed in the Terms of Reference (TOR). The section also discusses implications of some key findings and how these can influence sustainability of the project executed.

2.1 Profile of respondents Three out of every four respondents was married or in co-habitation with a partner. About 28% (127) of respondents did not receive any formal education and are less likely to be literate. The ages of FGD participants range from 17 to 67 years. The remaining have between then primary and tertiary education. The average household size of the sampled

communities is five. Majority of respondents are self employed, traders 48.2% (220). The rest were employed in the formal sector as teachers and public servants but 26.3% (120) were unemployed. The reported incomes by respondents ranged from less than equivalence of one US dollar to more than twenty a day (<US1.00 to >US$20.00). Most respondents reported household sizes of five 26.3% (120) and four 26.1% (119) as presented in fig1 below. Almost all respondents have lived in the community more than six months before the day of interview. A period considered long enough express relevant opinions about Urban WASH – II project

Fig1: Household Sizes

Two Three Four Five Six Seven Eight Nine Ten ormore

12

69

119 120

69

23 19 8 14

Fig1: Household Sizes

Table 1: Reported daily income (US$ Equivalence)

Amount in US$ % Freq

a <1.00 0.66 3

b 1.00 -2.00 7.89 36

c 2.01 - 5.00 32.89 150

d 5.01- 10 24.78 113

e 10.01- 20.00 14.91 68

f > 20.00 8.99 41

g Not stated or Unemployed 9.87 45

h Total 100 456


21

2.2 Project outputs and outcomes Information available suggests a steady increase of suspected cholera cases from the

affected communities and the country as a whole. It thus shows an increase rather than a

decline. This situation may be due to increased vigilance in reporting and documentation of

suspected cases. From an annual count of 1277 cases by the end of the year December

2011 the cases increased to 4730 by the end of December 2012.

Key actions and services provided as part of the project’s requirements include the provision of safe drinking water, promotion and free distribution of WaterGuard (water treatment chemical), provision of improved public toilets (shared latrines) and hygiene promotion through interpersonal communication, print and electronic media. An estimated 30,000 beneficiaries was reported. A summary of key interventions carried out has been attached as appendix 8. These include:

Community mobilisation in 32 neighbourhoods.

Hygiene promotion (water, sanitation and personal hygiene)

Water facilities - 16

Shared /Public Toilets (with bath rooms)

WASH in projects in 37 schools (population of 12,976) o Provision of latrines o Training coordinators and members of school health clubs

Promotion and distribution of WaterGuard.

Production and distribution of posters

Production of hygiene education videos and shows in video clubs and

Monitoring project implementation including HWST.

2.3 Water

2.2.1 Household Drinking water

The main drinking water sources for most households are safe and have increased from

70.1% (226) at baseline to 91.6% (418). The situation shows an improvement over the

baseline status. The number of households relying on unprotected sources also increased. Others include and water trucks/cart vendors also called “push-push”, sachet and bottled water use have reduced. The number of persons with house connections has also increased as presented in Fig 2.

Fig 2: Main Sources of Drinking Water

0.3

55.3

0.9

13.7

3.1

23.3

3.4 7.7

52.6

1.4

30

1.1 5 2.2

Fig 2: Main Sources of Drinking Water Baseline Post Intervention


22

2.2.2 Distance More than 73% (337) of respondents

travel less than one hundred meters

or less than 30minutes to access

drinking water. The distance to

drinking water sources has reduced.

It takes others up to 500 meters but

about 6.1% (28) of respondents

were unable to estimate how long it

takes them to access drinking water.

However, some communities still

walk ling distances. Some focus

group discussants in Point 4

reported sources more than one

hour away. The proportion of

households that make shorter

distances to access water has

increased over the baseline situation

although the proportion of those with

house connections has decreased.

Finding the reasons may require

additional information.

Fig 3: Time & Distance to Acess Water

2.2.3 Functionality and water quality

At the start of the project, 39.1% (126) compared to 36% (170) of respondents indicated that the main sources of drinking water did not last all year round. Out of this 46.2% (67) fell back on unprotected sources for the rest of the year. Although this is an improvement over baseline statistics, there still are lessons to be learnt in order to improve upon the current situation.

1. OnPremises

2. <100metres (Less

than 30minutes inand out).

3. 100meters -

<500 metres(30 mins).

4. 500 metres – I kilometers

About 1 hours)

5. Morethan 1

Kilometres(more than

1 hour)

6. Don’t Know/Not

stated.

28

52

9 4 7

0 8

65.8

14.5 4.2 1.1

6.1

Fig 3: Time & Distance to Acess Water

Baseline Post

I am happy!

Annie lives in New Kru Town and recounts the tremendous change in their lives. The 51 year old women narrated how they formed long queues from “6.00 am to 6.00 pm waiting to draw water”, she said. She also remembers how they used to go the government toilet and wait since there were many people there. Those who cannot wait go the swamps. “Since this new toilet was built by UNICEF in our area you can’t see toilet [feces] around. At first people use to puupu into plastic bags and all sorts of things, and chunk it around”. She concludes on a happy note “the community used to be porlu [polluted] now you go to the place and everything is clean”


23

Some newly constructed hand-dug-wells are not functioning as expected, the flow and quality of water from two water points have been questioned by some members of communities living in New Georgia Block “B” and “D”. “The water is not good for drinking … You can pump for small people and the water can’t come again so we close it and open in the evening”.

– Says a key informant in New Georgia. A small water system cited at Pipeline community – Moses booth is reported to be “smelling funny” according to the caretaker. This situation has been addressed as soon as the Consultant informed the IP responsible for the initial construction.

2.2.4 Water storage containers and treatment of drinking water

Information gathered suggests that the use of appropriate containers to store drinking water

has increased from 20.8% (67) at the start of the project to 84.6% (386). The proportion of

households treating their household drinking water increased by 18% above baseline status.

Those whose drinking water showed some evidence of treatment with WaterGuard but

recorded values lower than the recommended levels increased from about 14% (45) to 34%

(155). However, results of recommended dosage of chlorine at the various homes indicated

a much lower percentage 4.4% (20). This is a decline from 12.7% (41) recorded in the

baseline report. Further details have been presented in fig 4. Some households indicated

they did not have access to WaterGuards or did not know where to buy it. In the face of this

decline, about 80% (360) respondents claimed they can afford the cost of treated water for

their households. Only 18% (82) mentioned that they were visited by implementing partners

to test the quality of their drinking water and educate them on the use of WaterGuard. Fig 4: Evidence of treating household water (Pool Test)

2.2.5 Estimated volume of water per household

Most households 75.4% (344) use containers that hold 20 litres of water or more. Majority of them 61% (277) draw water twice a day hence the estimated volume of water drawn and possibly used each day per house and individual is about ≥ 40litres and 8 litres a day respectively. This situation is a marked improvement in access to drinking water compared with the era before the project. Refer to summary of key indicators results card table 6.

No presencePresence of

chlorine

< 0.2 FreeResidualChlorine

Residual Chlorine ≥ 0.2mg/L

I do not / nowater to treat

Not Applicable

32%

14% 12.7%

41.3% 44.7% 34%

4.4% 11.20%

5.7%

Fig 4: Evidence of treating household water (Pool Test)

Baseline % Post Interv'n %


24

2.3 Sanitation

2.3.1 Access to latrines More than one third, (195) of respondents had access to household latrines (WCs, pour flash, septic tank latrines). Close to 50% (217) use shared latrines and about 8% (36) resort to open defecation. A comparison between the current statistics and those reported in the baseline report shows an improvement as illustrated in fig 5. Access to family latrines (household latrines) and shared latrines have increased. On the other hand, open defecation rates have declined. About 40% (166) respondents indicated that latrines are cleaned by male adults in the communities. Another 26% (108) mentioned children thus making these groups the most appropriate audiences for hygiene promotion and training on effective cleansing and maintenance. Most latrines have separate cubicles for male and female users. Rails have been fitted in the disability cubicles to provide support for the aged, disabled and those who are heavy so cannot rely on the strength of their knees. Almost half of the sampled population, (226) indicated paying for the use of latrines. Out of this figure, about 82% (186) are able to do so for members of their households. This demonstrates ability to pay for latrine use and could be considered a sustainability factor. Fig 5: Main household defecation options: Fig 6: Distance and time to access latrine

2.3.2 Distance and time to access latrine

About 40.9% (168) have access to latrines at home and it takes less than 15minutes or walk

up to 100 meters for 53.3% (219) of respondents to access their regular place of defecation.

There is a moderate decline in the distances travelled to access latrines as displayed in fig 6.

There is also a decline in the number of households using family latrines but this may be a

result of members finding alternatives in newly constructed shared latrines. It will also reduce

pressure in household (family) latrines.

2.3.3 Accessibility: physical and financial

All the latrines constructed under the project have provided ramps to facilitate the movement of persons who are physically disabled and rails to support them. However, persons who are visually impaired will still find it difficult to access and effectively use the facility. Children in some of the schools are unable to use the toilets provided them. There are also persons who cannot afford the cost of water and use of latrines. It has been suggested that person who are very heavy and cannot squat may have been partially excluded. Inspection of the water facilities revealed they do not have facilities for persons suffering physical disability. Such persons may have to depend on others. This situation according to Ministry of Gender and Development sources are disempowering. Same source suggests that if lack of access to

39.4

32.3

1.9

18.6

0.9

42.8 47.6

0.9

7.0 1.0

FamilyLatrines

Sharedlatrine

Dig andbury

Opendefecation

Others

Fig 5: Main household defecation options (%)

Baseline Post Interv'n

Indwelling

house

<50metres

away (15Mins)

50 ormore

away (30Mins)

Don’t know.

42.6 47.8

6.5 3.1

40.9

53.3

2.4 3.4

Fig 6: Distance and time to access latrine (%)

Baseline Post Interv'n


25

WASH facilities mostly affects the deprived, then they must be part of the decision making processes concerning the designs and locations. Specific mentions include children, women, physically disabled persons and the visually impaired. These omissions may have resulted in what has been described as “design inadequacies” and “lack of targeting.” Some indications of exclusion have been identified and mentioned by the some community informants. Also, some focus group discussants explained that people practice open defecation because they cannot pay the user fees at the public toilet.

2.3.4 New and emerging challenges

Having dealt with the major sanitation and water issues and raised the level of WASH

awareness, the community is able to identify and ask for other WASH related needs. Some

of these issues have equity and related undertones. They are not negative. These should be

seen as positive outcomes of hygiene community sensitization and hygiene promotion.

“We that [who are] on this side are not enjoying the toilet. Because of the smell” – FGD

participant, Battery factory.

“People are not using the flush latrine, we want latrine on water to be destroyed”. – FGD

participant, Struggle community.

2.4 Household garbage (Solid waste) The outcome of education on garbage disposal was mixed. Only 7.7% (35) respondents kept

their household garbage in a manner that seems appropriate. It represents a decline of

about 5.7%. Majority 60.7% (277) kept unwanted household materials in open containers

exposing the contents to flies. About 18.4% (84) practiced dig and bury. The others burnt or

simply threw rubbish about their vicinity. That notwithstanding, 71% (324) considered their

methods of solid waste appropriately. This presents a real test for hygiene promotion.

Respondents mentioned dirty surroundings 53% (246) as the one of the challenges

confronting them in the community. Others are fly nuisance 40.8% (186) smelling

neighbourhood 37.1% (169). The baseline and post-interventions results have been

presented in figure 7.

“Not everyone can pay, sometimes we drink our well water” – FGD participant, Chocolate City.

“It is good for us but the water is expensive” – FGD participant,

Pipeline, 72nd

community.

“The blind man can’t see so we need Kamo” – FGD Chicken Farm. “I can see toilet around in my house. Plenty people toilet in plastic bags due to the hard cost” – FGD participant, Duala Market


26

Fig 7: Reported household garbage disposal methods

The communities seem to feel some relief with the provision of latrines and currently

identifying other needs.

“The dirt business are too much, we need garbage tank and since it has not come, dirt is everywhere” – FGD participant Battery factory. “There is no place for dirt collection” - FGD participant chocolate city. “We are asking for a garbage can to help us clean our community” – FGD, Jamaica Road.

2.5 School WASH project

A total of 37 schools have benefited from the project. Water supply, sanitation and Hygiene promotion are the range of service provided. School health clubs were formed and members trained in the schools. WASH Training was conducted for selected teachers as School Health Club Coordinators. Subsequently, a step-down training for members of the school heath clubs were organized thus involving the pupils (school children) in managing and maintaining WASH facilities. School health clubs (SHCs) are at the centre of hygiene education and WASH promotion activities in the schools. There are positive indications that the performance of SHCs could be sustained if effective monitoring is carried beyond the project phase. However, the officials of the Ministry of Education are not aware of the schools selected and will like to be part of the selection process in future. Some of the schools are effectively combining the services provided by janitors with school children’s learning and self help activities. These will help provide practical experiences for the children and must be emulated by other schools. The complete reliance on janitors for delivery of all school WASH services may not only be difficult but cannot be sustained in the light of declining quality of services and low conditions of service for some of the janitors engaged in public schools. Some negative WASH practices were observed and these must be changed.

1. Only a few schools had plans and schedules to ensure proper monitoring and follow-up actions of school health clubs. Many do not have it.

2. All the handwashing facilities inspected were not functioning. 3. Some parts of the facilities had broken and were not functioning. Examples include

faucets and taps.

13.4

61.8

3.7 13.4

2.2 5.6 7.7

60.7

18.4

1.1 8.3

3.8

Fig 7: Household garbage disposal methods

Baseline Post Inter'tn


27

4. The numbers of handwashing facilities are inadequate compared to the enrolled population.

5. The latrines are not child-friendly. Smaller children are unable to use them. 6. Some of the school health club (SHC) coordinators punish children to undertake

water and sanitation services. This may send negative perceptions that WASH

services are not desirable. This must be stopped and other forms of punitive action

should be considered.

7. Some of the schools (including community toilets) have provided containers for anal

cleansing materials to be kept. These can facilitate transmission of diarrhoeal

disease and must be stopped.

2.6 Hygiene Promotion Marked progress has been recorded in some departments of the hygiene promotion including awareness, increased knowledge and some changes in behaviours. When respondents were asked to mention the hygiene education they received as a result of the project, their responses were categorical as presented in table 2 below (multiple responses).

Table 2: What education have you received as a result of the Urban WASH project?

What are doing Differently as a result of hygiene education?

Educational themes/topics % Freq. % Frequency

1. Regular hand washing with soap. 49.1 224 5.2 24

2. Safe Treatment and storage of water

41.4 189 5.2 24

3. Use of latrine / abandoning open defecation.

46.3 211 58.9 269

4. Others (Specify) 17.3 79 3.7 17

2.7 Handwashing

2.7.1 Availability of hand washing facility

Access to handwashing facilities is a factor which enhances opportunities for people to practice that action but 45.4% of households did not have the facility.

2.7.2 Handwashing with soap

Almost all respondents 95.6% (436) indicated that they regularly washed their hands with

soap. Responses to reported handwashing at critical moments has been captured and

compared with baseline statistics. The bar chart in fig 8 shows an increase in knowledge of

hand washing in general but declined when it came to washing hands before feeding a child.

In future hygiene promotion may have to focus on this critical moment alongside the general

message on the benefits of regular handwashing with soap.

When they were asked, respondents mentioned the following actions as the key changes in behaviours and intentions made as a result of handwashing education they received. These include:

Wash hands with soap regularly - 61.4% (264)

Build or buy hand washing facility – 1% (4)


28

Educate child (ren) to wash hands regularly – 28.4% (122)

Don’t know – 11.2% (48) Fig 8: Reported Regular Handwashing Compared

Although 72% (314) reported regular hand washing with soap after defecation, discreet

observations by enumerators unknown to their subjects did not support this assertion. Only

36.3% (53) persons actually washed their hands after using the latrine.

Other handwashing and related factors observed have been recorded and presented in table

3 below.

Table 3: Results - State of Latrines and User Observation

1. State of latrine (Describe the state of latrine) Frequency %

a. Condition of immediate environment, clean? 1. Yes 110 76.4

2. No 34 23.6

Total 144 100.0

b. Proper disposal of anal cleansing 1. Yes 106 73.6

2. No 38 26.4

Total 144 100.0

c. Is there Odor? (Foul smell)

1. Yes 61 43.0

2. No 81 57.0

Total 142 100.0

88%

13%

43%

11%

70%

26%

2%

94% 96%

81%

26% 25%

72%

29%

18%

94%

Fig 8: Reported Regular Handwashing Compared

Baseline Post-Interv'tn


29

d. Is it accessible to children? 1. Yes 66 46.2

2. No 77 53.8

Total 143 100.0

e. Is there any defect on the structure? 1. Yes 20 14.1

2. No 122 85.9

142 100.0

2a. Is there hand washing facility near latrine? 1. Yes 89 61.8

2. No. 55 38.2

Total 144 100.0

b. Hand washing facility has soap/ash?

1. Yes 87 63.5

2. No 50 36.5

Total 137 100.0

c. Is there water available for washing? 1. Yes 56 38.9

2. No 88 61.1

Total 144 100.0

d. Availability of soap on site? 1. Yes 89 61.8

2. No 55 38.2

Total 144 100.0

3. Observation of an individual latrine user:

f. The person using the latrine wash hands?

1. Yes 53 36.3

2. No 93 63.7

Total 146 100

g. Hand washing done properly?

1.Yes 55 39.3

2. No 85 60.7

Total 140 100

2.8 Perceived decline in prevalence of diarrhoea

As a proxy to obtaining information on impact of the project on its ultimate goal of reducing cholera and other diarrhea diseases among children, mothers were asked to mention the number of episodes their children (0-months to 5years old) suffered during the past one year dived into parts; six months before the intervention and another six months after the start of the intervention. The findings presented in fig 9 below suggest a general decline in perceived cholera and other diarrhea disease.


30

Fig 9: Reported diarrhoea among children

Similar views were expressed by other stakeholders including, Community WASH Committee Leaders, FGD participants, School health coordinators and Community Health Animators. “The number of people with diarrhea is decreasing, … People are responding to our message …they are using the latrines” – Community Animator ,New Kru Town. “…the people are washing hands with soap” – Community Animator, Point Four.

2.9 Hygiene behaviour change motivators Almost all 95.4% (435), of respondents expressed a sense of high self esteem as they

indicated that they were capable of taking good hygiene decisions. The assessment did not

test the assertion. Apart from themselves 64% (292) and their children 22.1% (101), their

landlords/ladies 7.9% (36) matter most when it comes to taking household WASH decisions.

About 21.1% (96) respondents also mentioned their heads of families as persons who

influence their WASH related decisions. Important WASH related decisions in the

households are ultimately taken by themselves and others mentioned in table 4.

Table 4: Major household WASH decision makers

1. Landlord/lady 7.5% 34

2. Head of family 12.7% 58

3. Husband 13.8% 63

4. Myself 71.7% 327

5. Other tenants / family members 2.9% 13

6. Public Health Inspectors 0.0% 0

7. Community volunteers 1.0% 1

8. Others 1.3% 6

Behaviour change communication on WASH related issues should therefore be premised on these factors as prime motivators among women household heads. Landlords/ladies are therefore influencers who must be targeted both as primary and secondary audiences.

1. None 2. Once 3. Twice 4. Thrice 5. More thanthrice

80.9%

12.6% 4.7% 0.7% 1.2%

71%

54%

9.1% 5.6% 1.2%

Fig 9: Reported diarrhea among children During & After Before


31

2.9.1 Preferred channels for hygiene education

Most FGD respondents mentioned community meetings and house-to-house dissemination of hygiene information and education but the responses solicited through the structured questionnaire include those presented in fig 10 below. Fig 10: Preferred channels of communication

2.9.2 Benefits and satisfaction

Various levels of satisfaction were expressed by beneficiaries. They range from very

satisfied to unhappy individuals. It is very easy to notice the satisfaction brought about by

the project. Most respondents expressed their satisfaction with different aspects of the

project. The highest level of satisfaction was one for less medical expenses attributed to the

gains of the project. Table 5 contains further details.

Table 5: What benefits or changes as a result of Urban WASH WASH - II

Responses % Frequency

1. Improved water quality. 3.1% 14

2. Easy access to water. 0 0

3. Decline in diarrhea diseases. 26.50% 121

4. Less medical expenses. 13.80% 63

5. Clean environment/less houseflies. 41.40% 184

6. Improved sanitation/No improvement (15), etc 11.20% 51

There were more satisfied voices but these could not drowned by the passionate appeals of those whose WASH needs are yet to be met. “We have a toilet built by LICH, thank you UNICEF and LICH” – FGD participant Struggle Community. “Children [school children] wash hands now than before” – School teacher

7%

79.8%

3.1%

23.5% 23.5% 13.2%

41.2%

8.8% 2.9%

Fig 10: Preferred channels of communication


32

“Complains of running stomach among the children is not like before. It has reduced” – School Health Coordinator,

. These notwithstanding there are voices of dissatisfaction. Some of these have been captured from different communities and presented below: “There is no toilet for us, the first latrine has been own by those who own the land” “We suffering for [from lack of] drinking and the mosquitoes and flies are giving us hard time” “We don’t have safe drinking water and proper dirt disposal …we have toilet but it placed in the school name” “The bath room should not be in the toilet” “I am satisfied with the water and toilet” “As for me, I am satisfied” “Not satisfied the cost if too hard” “We need kamo [Commode - Handwashing facility] place to wash our hands” – FGD, Jamaica Road - NKT

2.9.3 Sustainability

For purposes of this assessment, discussions on sustainability have been limited to only two

factors; management and financial sustainability.

1. WASH facility management

a. The management of both water and sanitation facilities are expected to be

done by community WASH committees. One of such potentially viable WASH

committee was identified at Struggle Community. In many other communities,

the management of the facilities are centre around individuals who provided

land for the project.

b. Decisions around the exemptions from paying user fees have been left in the

hands of caretakers. Such information and decisions are not in the public

domain.

c. There are no records on the activities carried at the water kiosks and latrine

sites. Although some of the caretakers and community indicated the

availability of such records, none of these were available for verification.

Information on user numbers and amounts realised each day could not be

provided and where these were mentioned the basis for arriving at the figures

were not provided or they were not consistent.

d. The issues raised above are conditions that go contrary to WASH sector

guidelines issued by the government of Liberia GoL (2010) and can

undermine the sustainability of the facilities provided.

2. Financial sustainability

a. The consultant gathered some information from caretakers, representatives of

implementing partners and users of the facilities. Some assumptions have

also been made as basis making some projections. The outcomes of such

calculations suggest that nearly all water and sanitation facilities are

financially viable and sustainable beyond the short to medium term (1 to 5

years) as presented below. Some further assumption can be made to

determine their sustainability over a long term period a lifecycle cost

assessment is a preferred option. Refer to appendix 9 for detail projections.


33

i. Water

Between the water tariff charged by LWSC (30 gallons for $5LD) and

the rates charged by the caretakers/operators (10-15 gallons for

$5.00LD), there is a mark-up enough to cover operational cost such

as paying the caretaker and replacement of taps, other minor repairs

and lost through spillage.

ii. Latrines

Information gathered from which estimates have been provided

suggest that the reported expenditures are less than the incomes

generated. It thus provides excess income over recurrent

expenditures. Calculations on short to medium term (1-5 years)

suggest that if prices and all other factors held constant. The facilities

should break even and within 1 to 5 years and be able to replace the

facility in the long term (5 – 10 years) at an estimated cost of

US$9200.00.

It is good UNICEF gave us water! It is usual to provide sanitation, water and hygiene services to mainstream populations but meeting the needs of the most vulnerable has often eluded most development agencies engaged in the water and sanitation sector. Having consciously identified and targeted the WASH needs of the poor and most vulnerable around Monrovia, UNICEF has attracted very kind comments from the aged located in high density and swampy neighbourhoods of Point Four, Monrovia, Liberia. Madam Mokombe is a 72 years old woman. She lives with 2 grand children in a house which does not have a household latrine (called family latrine). With very little resources at their disposal, the two children defecate in the open near the beach and take their bath also in open spaces on their compound. Madam Mokombe is unable to walk the entire stretch to the beach every morning so she finds comfort and privacy in the one room she shares with her grand children and take her bath before down and or after dusk. Speaking to Madam Mokombe as a special informant, she could not hide her appreciation. She said, “… it is good you gave us the water. Thank you for the water … and God bless you my son [meant for UNICEF] for the toilet. Now I can go to the toilet and help myself. The beach is too far away”. It is worrying to note how easy it has become to refer to the beach as the standard location for defecation.


34

Table 6: Project indicators and Summary Results Card

Indicators Baseline Indicators Results of Evaluation

1. Contribution to the reduction of incidence of cholera and other diarrheal diseases in slums in Monrovia.

Number of Cholera and other Diarrhoea disease December 2011 - 1277

Number of Cholera and other Diarrhoea diseases: December 2012 - 4730

2. Users /improvement in access to water points.

Improved sources – reported 70.1% (226)

Improved sources – reported 91.6% (418)

3. Users/latrine cubicle

Planned number of cubicles to be

constructed 226

(11,300 beneficiaries)

Number cubicles constructed:

174 completed (8700)

52 in progress

4. Percentage of households using a pit latrine or flush latrine.

Family Latrines (pit and pour Flush) 39.4% (127) Public shared latrine 32% (104) Open defecation – 18.6% (60)

Family: 42.8% (195) Shared: 47.6% (217) OD: 7.9% (36)

5. Percentage of drinking water samples tested in the home with free chlorine residual >0.2mg/l

Pool Test Results # %

There is no indication of presence of chlorine 103 32.0%

< 0.2 Free residual Chlorine 45 14.0%

Free residual Chlorine > 0.2mg/L 41 12.7%

I do not treat water 133 41.%

# %

204 44.7%

155 34%

20 4.4%

104 22.8%

6. Increased percentage of households transport and store their water in 5 gallon (20 litre) plastic jerry cans.

Jerry can with cap (Estimated) 67 20.8%

368 84.6%

7. Percentage of persons observed washing hands with soap after using latrine.

Observed hand washing after defecation: 30.3% (20)

Observed hand washing after defecation: 36.3% (53)


35

Section 3: Conclusion, Lessons learnt and Recommendations

In the section, conclusions have made in line with targets and expected outcomes set in the

terms of reference (ToR).

3.0 Conclusions

To a very large extent the project has achieved its objects. The project according to all

stakeholders is relevant and appropriate but there are challenges and lessons to be learned

so that the outcome of future projects will turnout better.

3.1 Coherence

The review of selected project records and related literature as well as the outcome of a

wide consultative process suggest the project met key sector demands and high project

management standards. The project was implemented largely as proposed with slight

changes only to improve the quality of delivery and to meet the needs of persons with

disability. However, the specific needs of very poor persons who cannot afford to pay for

water and user fees at the various points need further discussions, identification, proper

targeting of such people and community action to provide them access.

There are some government partners who indicated that some implementation procedures

guiding the delivery of WASH facilities have not been fully complied with and these have to

be addressed. They include;

1. The Hydrology Unit of the Ministry of Land, Mines and Energy which expects

implementing partners to consult that Unit for advice on appropriate areas to cite

wells or drill boreholes and to submit the relevant data after completion of the facility.

2. The Environmental and Occupational Health Unit of the Ministry of Health and Social

Welfare is not happy that implementing partners are not conducting and /or not

submitting water quality test after completion of water points.

3. The Ministry of Gender and Development suggests active involvement of women,

children and persons living with physical disability at all levels of project cycle;

concept, planning, implementation, monitoring and evaluation. They emphasized

post construction engagement with target audiences.

4. The Ministry of Education (School Health Unit) wants to be actively involved in the

selection of beneficiary schools to facilitate monitoring after project completion.

5. It has also been observed that effective monitoring and coordinating WASH facilities will further guarantee sustainability if government partners including the Monrovia City Council (MCC) are adequately informed and involved very early stages of the project.

3.2 Relevance Most sector agencies knew about and they suggest the project was relevant and timely. The

government agencies think the project was in line with the key sector policies including the

Poverty Reduction Strategy (PRS). A visit to the sampled communities and subsequent

transect walks revealed the real WASH needs and the level of deprivation the project sought

to address. There is no doubt therefore that the project was implemented in very deprived

and poverty stricken areas in Monrovia. That the project has addressed the needs of

women, care givers, children and persons suffering from physical disability and deprivation is


36

an understatement. However, a lot more needs to be done to target and focus on the

sanitation needs of children in school and the very poor.

3.3 Beneficiary satisfaction

From the results presented this far, there is ample evidence to suggest that beneficiaries are

satisfied with the project. Their leaders and to some limited extent sections of the

communities were actively involved in the project design, implementation and participatory

monitoring. The outcome of this achievement is the communities’ realization that there are

other WASH needs to be met. However, communities in which projects are yet to be

completed have also expressed some level of dissatisfaction.

3.4 Efficiency

A critical examination of planning and resource allocation processes suggest a system and

transparent arrangements established by UNICEF were adhered to. The consultant

assessed randomly selected outputs; water and sanitation structures and found them

generally to be of high quality. A cursory review of some comparison with other project

outputs including the going market prices suggests the project was efficiently executed and

managed. There was no overrun of budget rather there was a higher number of beneficiaries

above the project target mentioned from 12 to 32 communities. Unicef also provided

additional resources to fund the evaluation.

3.5 Effectiveness

Information available to the consultant points to an effectively implemented project. The

management team identified challenges and revised original project designs and plans to

accommodate the needs of physical disabled persons and to increase the number of project

communities thus ensuring high value per dollar invested. However, the slow pace of

implementation by one of the implementing partners has resulted in delays in meeting

project timelines.

For example, two sampled projects have fallen behind schedule. One latrine project in

Struggle Community is only about 50% finished one month after the end of an extended

finishing line. Project monitoring reports also indicate that as many as 23% of latrine cubicles

are not yet completed.

There are some design or construction challenges which may require further examination

and remedial actions taken. The ramp meant to provide access for persons suffering

physical disability to a latrine constructed for school near battery factory seem too steep

even for able bodied persons.

3.5.1 Performance of IPs

Performance of implementing partners has been measured from two different perspectives.

The first is performance set against project objectives and the second is based on

perspectives of responses from beneficiary communities.

Performance set against project objectives;


37

By December 3, 2012 all partners except Oxfam had completed all deliverables.

Some of the reasons provided by Oxfam for state of affairs are due to limited

capacity. They are;

o Inability to identify very competent contractors to construct WASH facilities.

o Difficulty in obtaining land space for construction.

Perspectives of beneficiary communities.

Reviewing IP performance from community perspectives has provided useful results

and lessons for designing future projects. Key results areas have been presented in

appendix 10. From the matrix, the best performing IP was is LICH and the least

performing IP is Oxfam.

3.6 Accountability

A monitoring process was put in place and has assisted the WASH Section of UNICEF

assess progress and ensure quality during implementation. Regular monitoring visits were

carried out to the project sites, progress reports and actions taken have been documented.

Most sector partners have participated in regular sector meetings where project reports are

shared with members of the WASH Consortium in Monrovia. It does not seem that the

systems in place and the actions taken have been that responsive to the emerging

challenges.

3.7 Sustainability

Some very important actions have been carried out to ensure the sustainability of the

project. They include but not limited to;

Frequent and regular consultation with major sector players including government

partners.

Reasonable budgets allocated for community mobilisation and consultations with

community leaders /opinion leaders

Creation and training of community WASH committees.

Identification, training of and deploying Community Animator and Community Based

Health Volunteers

Development and production of relevant print and electronic materials for creating

awareness to ensure diversification of hygiene promotion approaches.

A simulation of income and expenditure patterns suggests that with open and

transparent management of the WASH facilities provided, the facilities are financially

viable in the short to medium term.

Together the interventions and conditions listed above are likely to build relevant capacities

at all levels for the proper management and functioning of the facilities over time. They can

also engender ownership by communities and empower the poor, disadvantaged and

deprived members of the community to access the WASH facilities and services.

Notwithstanding, the actions listed above there are early signs that some capacity gaps

exist.

There are communities which are yet to meet and take collective decisions to

promote the effective use of WASH facilities provided in their communities. An


38

example is the Struggle Community. The newly constructed latrine provided under

UW-II is facing undue competition from the infamous “latrine on the water front” and

the community seems helpless.

Most communities visited do not have the full complement of WASH Committee

members. The management of facilities revolves around a few people especially

those who provided land for the construction of the facilities. This can lead to turning

the facilities into private hands. The water facility at Moses Booth, Pipeline

Community and the latrine cited at Battery Factory are examples. A follow-up

discussion with the implementing (CODES) has since led to resolving the challenge

at Moses Booth but this may have to be followed up by UNICEF subsequently.

Some committee do not hold regular meetings. Others do not meet at all. None of the

committee members sampled have ever held themselves out to account to other

members of the committee neither have they met with the wider community.

None of the caretakers are yet to show proof of a bank account into which they pay

proceeds for the day. Indeed most of the care-takers put in-charge of the WASH

facilities were reluctant to discuss facility user numbers. It is worse when incomes

generated from facility user fees seem far less than the designed capacity of the

facility.

These are factors which can compromise the functions and sustainability of the facilities

provided.

3.8 Lessons Learned

There is no information on how lessons from the previous project (urban WASH-I) have been

adopted and systematically implemented as lessons to enhance the outcome of the current

and future projects. As part of this report however, the following are lessons identified so far:

1. Although sector policies are widely circulated, the mere awareness does not

necessarily ensure compliance.

2. The creation and subsequent training of WASH committees will not naturally lead to

effective performance. They need some hand-holding, support and post

implementation/construction monitoring to ensure good performance.

3. When poor communities have several WASH needs. Working with them to identify

and solve initial problems empowers them to identify and articulate new and

emerging issues.

4. The implementation of WASH projects do not immediately and directly relate to

declines in suspected cases of Cholera and diarrhoea diseases.

3.9 Recommendations

1. Reviewing project outcomes for purposes of learning

Obviously some project components did not perform as expected. It will be appropriate for

UNICEF and IPs in consultation with other sector partners to review the underperformed


39

components of the project with the view to identifying causes, learn lessons and share with

other stakeholders. They include specific areas such as:

a. Garbage collection and disposal.

b. Promotion, proper use and monitoring the use of WaterGuard.

c. The increase in diarrhoea diseases in the affected communities despite the WASH

interventions implemented in the selected communities. This should be done in

consultation with the Health Unit of Child Survival section of UNICEF and other

Government public health agencies.

2. Feedback sessions with project communities and schools

2.1 Community

The high level of satisfaction with the facilities constructed under the project and related

outcomes are signs of confidence. It will be necessary for implementing partners to organize

feedback sessions for communities with the aim to further build their capacities (Castro et. el

2009). The sessions should adopt participatory methods to learn from them specific events

and process which went well and are worth replication. This should provide opportunities

learn about some knotty points and challenges identified in the previous sections to be

addressed. 2.2 School sanitation and related issues School health clubs (SHCs) are at the centre of hygiene education and WASH promotion activities in the schools. There are positive indications that the performance of SHCs could be sustained if effective monitoring is carried beyond the project phase. The negative WASH practices observed and reported in the previous sections will require urgent and collective remedial actions to ensure sustainability of the SHCs and continuous availability of the facilities provided. They are:

a. The state of some schools latrines outside the UW-II project found to be unhygienic s are reported to rely mainly on janitors who do not seem sufficiently motivated. A more stringent monitoring arrangement should be establish by the MoE to promote hygiene. Students should be motivated to use the facilities properly and become part of the cleansing processes.

b. School authorities should ensure that minor repairs and replacement of little but important components like faucets/taps should be done regularly. The School Health Directorate of the Ministry of Education in collaboration with the IPs should encourage school authorities to take up this responsibility and expect external agencies to come and fix these.

c. UNICEF should assist IPs so they can adapt at least two of the cubicles/squat holes (one on each on male and female sections) so the children can access and use the toilets on their own. It will also be extremely helpful if an additional hand washing station will be provided in the schools to further motivate the students to wash hands and also ease pressure on existing facilities.

d. It is strongly recommended that IPs should return to all the schools and urge them to discontinue the use of baskets and other receptacles for keeping anal cleansing materials. The practice is a potential means of transmitting diarrhoeal diseases.

e. School health club (SHC) coordinators and schools authorities should identify other

ways of motivating students and desist from assigning WASH responsibilities are a

way of punishing students for wrongdoing. They should adopt innovative approaches

to motivate students to clean and maintain WASH facilities voluntarily.

3. Improve upon targeting and meeting WASH needs of the poor


40

UNICEF should lead a process of developing a set of criteria in consultation with other

sector partners to help target and focus on the very poor, persons suffering from other forms

of disability and impairments to ensure that the poorest are reached under this and

subsequent projects.

Implementing partners (IPs) should seek assistance from development Partners including

UNICEF to help them identify needs of very poor persons who cannot afford to pay for water

and user fees at the various points. This should attract further discussions, identification and

proper targeting of such people and community actions implemented to provide them

access.

4. Specific sector requirements and further consultative process a. UNICEF should include in the PCAs the need for implementing partners to consult

the Hydrology Unit of the Ministry of Land, Mines and Energy to:

I. Select appropriate locations to site hand-dug-wells to drill boreholes.

II. Submit relevant data including well logs to the Hydrology Unit of MLM&E.

b. Implementing partners are also expected to conduct and submit water quality test

results after completion of water facilities.

c. As part of future project designs, it is recommended that UNICEF should

select/develop a set of criteria for monitoring the involvement or otherwise of

women, children and persons living with physical disability to ensure that they are

actively engaged at all levels of the project cycle namely, concept, planning,

implementation, monitoring and evaluation.

d. It is also of utmost importance to emphasize post construction engagement with

target audiences as mentioned under the previous item marked “c” above.

e. Under the leadership of UNICEF the School Health Unit the Ministry of Education

should be actively involved in the selection beneficiary of schools to facilitate

monitoring after project completion.

5. Regular review of IP performance and capacities

The delays in meeting some project timelines require a responsive revision of partnership

and implementation arrangements.

a. UNICEF should introduce punitive clauses into PCAs as motivation to get

implementing partners to perform as expected. Examples include;

I. Inserting clauses in the PCAs requiring defaulting IPs to refund or forfeit

proportions of the cost of project components not completed within/on the

delivery date; and

II. Immediate appraisal and down grading the level of partnership to “least

performing” IP.

b. It is also recommended that UNICEF in consultation with its Government Partner

agencies should assess capacities of partners and the results used to select IPs for

each project. One of the important indicators to be considered under such

circumstances should be the ratio between existing capacities on one hand and

volume of existing demands on those capacities. This should be used to review or

develop terms of PCAs.


41

c. UNICEF should ensure that monitoring of IP performances is followed with

responsive management decisions and implementation of such decisions each time

performance fell below set standards or targets. Where IPs excel in performance,

they should be recognized in a special way reworded accordingly.

6. Further capacity building for WASH committees

It recommended that UNICEF will invest in post project monitoring and capacity building for

WASH committees and community leaders to address specific issues including:

a. Mobilisation of project communities to take necessary actions likely to remove

obstacles barring access to WASH facilities provided under the project. Particular

reference is made to example in Struggle Community. The newly constructed latrine

which is grossly underutilised.

b. Proper constitution of community WASH Committees and to ensure that these

committees:

o Meet regularly and keep proper records especially financial records.

o Compile and render proper accounts of WASH facility user fees to the

community.

o Open bank accounts and pay-in WASH facility user fees.

c. Further mobilization of communities to demand accountability.

7. Visibility for European Commission for Humanitarian Aid (ECHO) Most of the facilities constructed under UW-II especially, posters, latrines and water systems carry the marks and insignia of the donor – ECHO. This level of acknowledgement has been done with other printed materials as provided in the contract documents. However, two of the hand-dug-wells sampled do not have these marks of acknowledgements and needs to be addressed.

Left to right: Work in progress by Oxfam's sub-contractor


42

References

1. Castro V, Msuya N and Makoye C (2009) Sustainable Community Management of Urban

Water and Sanitation Schemes (A Training Manual): Empowering communities to

improve water and sanitation services. WSP assistance to and DAWASA.

2. Government of Liberia (2012). Water, Sanitation and Hygiene Sector Strategic Plan.

MLM&E.

3. MICS4 Indicators: Numerators and Denominators MICS4 List of Indicators, v3.0, 16

January 2012.

4. Population Services International - Liberia (2012). Expansion of Household Water Treatment and Hygiene in Liberia Final Report. June 1, 2011 – May 15, 2012.

5. Republic of Liberia (2010). The Guidelines for Water and Sanitation Services In Liberia Ministry of Public Works In Collaboration With The Liberia Wash Consortium And Unicef

6. Government of Liberia (2010) Water, Sanitation and Hygiene Sector Strategic Plan

(2012-2017).

7. Government of Liberia (2009) National Integrated Water Resources Management Policy. Ministry of Lands, Mines and Energy in collaboration with other agencies. Sustainable Sanitation in Urban Slums: The path to permanently reducing sanitation-related diseases.

8. Government of Liberia (2008) Population and Housing Census.

9. Liberian Hydrological Service. Government of Liberia. Ministry of Lands, Mines and Energy. Well Data Card.

10. The Sphere Project (2011) Humanitarian Charter and Minimum Standards in

Humanitarian Response.

11. UNICEF (2010) UNICEF- Adapted UNEG Evaluation Reports Standards.

12. UNICEF and World Health Organisation (2012). Progress on Drinking Water and

Sanitation 2012 Update.

13. WHO (2010). UN-Water Global Annual Assessment of Drinking Water-Water.


43

Appendixes

Appendix 1: Excerpts of TOR: Objectives of Assignment

VISION Contract reference: _________________________

PART A: DESIGN OF THE CONSULTANCY

I.TERMS OF REFERENCE

1. Purpose of the assignment.

To evaluate the ECHO funded 2012 Urban WASH project targeting cholera prone urban slums in Monrovia

(urban WASH II).

2. Validity start date: 12th

December 2012

3. Validity end date: 12th

February 2013

4. Section, duty station: CSD/WASH, Monrovia, Liberia

5. Consultant’s manager - name and title: Zainab Al-Azzawi, M&E Specialist

6. Authorised officer - name and title: Isabel Crowley, Resident Representative

7. Background and Justification.

UNICEF has implemented an ECHO funded WASH project in 2012, in collaboration with four NGO partners to improve access to water and sanitation and to increase better hygiene practices in targeted cholera-prone urban slum communities in Monrovia (urban WASH II: SM120084). The proposed external evaluation is an agreed component of the ECHO urban WASH II project and will help to identify gaps and challenges to enhance future urban WASH interventions. This evaluation is planned to take place in December 2012 in order to coincide with completion of the project work and in order to meet donor reporting requirements agreed in the project proposal.

8. Objective and targets of the assignment.

v) The external evaluation will provide analysis of urban WASH II (SM120084)

project achievements and setbacks against planned results, assess the project’s

intervention logic, promote accountability, and assist future improvements in

performance.

In order to do this the external evaluation will evaluate:

o the impact of the project -measure and assesses the extent to which the

project contributed towards achievement of the principle and specific

objectives detailed in the project proposal/s.


44

o the coverage of the project -measure and assesses the extent to which the

project reached groups targeted in the proposal, including children and their

caregivers.

o the coherence of the project - assess whether the project was implemented

in line with the plan outlined in the proposal

o the appropriateness/ relevance of the approach - Assess whether the project

was in line with identified needs and priorities of the target population;

Assess the satisfaction of the beneficiaries with the project; Assess the level

of beneficiary involvement including women in the project design,

implementation and monitoring- participatory approach developed/ used for

assessment and project implementation?

o the efficiency of the project - Measure the qualitative and quantitative

outputs achieved in relation to the inputs and compare strategies to see

whether the most efficient approaches were used; Assess the level of cost

effectiveness/ appropriate use of the budget.

o the effectiveness of the project -Assess any factors that affected the

effectiveness of the project and how the factors were addressed during

implementation

o the accountability of the project -Assess monitoring systems put in place for

quality control

o the sustainability of the project -Measure the extent to which the beneficiary

bears ownership of the project, local capacities is strengthened, and

sustainable management arrangements have been put in place.

o lessons learnt conclusions and recommendations

vi) The external evaluation will provide data which will allow UNICEF to report to

ECHO on the indicators and targets listed in the urban WASH II: SM120084

proposal.

The primary audience of this evaluation will be WASH actors within Liberia including UNICEF, the donor, GoL and CSOs. The secondary audiences of the evaluation will be actors outside of Liberia, including UNICEF Regional Office, Headquarters, and donor representatives.

9. Specific activities to be completed to achieve the objectives

d. Collection and analysis of secondary information including: donor proposal, project documents, progress reports, distribution and sales records of WaterGuard, data from health facilities, data collected by NGOs, financial records of water and sanitation facility management entities, IEC materials used, GPS data, etc.

e. Collection and analysis of primary information through:


45

iv. Structured interviews, focus group discussions, questionnaires, and

meetings with key stakeholders including: NGO partners, and UNICEF

project staff, local representatives, local health workers and

community leaders, beneficiaries, non-beneficiaries etc.

v. Structured observation of quality of facilities, usage, hygiene practices

etc.

vi. Sample collection and analysis of drinking water to measure free

residual chlorine.

f. Provide evaluation and analysis of progress against targets and objectively verifiable indicators detailed in the proposal for urban WASH II: SM120084 – see Section 11 below.

10. Tangible and measurable outputs of the work assignment

a. Final written report (preceded by detailed work plan and interim report) b. PowerPoint presentation of evaluation results to UNICEF, ECHO, and the implementing

Partners

11. Performance indicators for evaluation of results

Timely submission of comprehensive final report and PowerPoint presentation written in English containing the following elements:

Cover page

Table of contents

Executive summary (Describing the programme, main findings of the evaluation,

conclusion and recommendations.)

Main body (Background information, methodology, findings)

Conclusions and recommendations

Annexes

Report and presentation to cover indicators/targets listed in proposal to ECHO for Urban WASH II, SM120084: • Contribute to the reduction of incidence of cholera and other diarrheal diseases in slums in

Monrovia. • Users/improved water point • Users/latrine cubicle • %age of population reporting always or frequently washing their hands with soap before

feeding children; after cleaning up following children’s defecation; before preparing food; and after using toilet

• Percentage of drinking water samples tested in the home with free chlorine residual >0.2mg/l

• Percentage of households transport and store their water in 5 gallon (20 litres) plastic Jerry cans

• Percentage of households using a pit latrine or flush toilet


46

• Percentage of population observed to wash hands with soap after using the toilet

III. SIGNATURES

1. Consultant’s manager

Name and title: Zainab Al-Azzawi, M&E Specialist

Signature:

2. Head of section

Name and title: Laura O’Hara, Chief CSD

Signature:

3. Deputy Representative (for programme owned SSAs)

Name and title: Fazlul Haque, Deputy Representative

Signature:

4. HR Review

Name and title: Marcela Madero, HR Specialist

Signature:

5. Representative

Name and title: Isabel Crowley, Resident Representative

Signature:


47

Appendix- 2: Household Questionnaire- Urban WASH – II

GOVERNMENT OF LIBERIA - UNICEF

QUESTIONNAIRE FOR WOMEN HOUSEHOLD HEADS

EVALUATION OF URBAN WASH-II PROGRAMME IN MONROVIA

Introduction:

UNICEF has in collaboration with other national and international organisations implemented an

Urban Water Access, Sanitation and Hygiene (WASH) programme in selected communities

around Monrovia within the past 18 months and more.

The project under review is Urban WASH-II. It sought to provide access to 60,000 beneficiaries.

The aim is to ensure a halt and subsequent decline in cholera and other WASH related illness

such as diarrhoea diseases in the targeted communities presenting as the most affected,

especially among children, women and disadvantaged members of the different communities.

You have been selected as a member of an evaluation team charged with the responsibility to

gather relevant information using this questionnaire. This questionnaire will require that you

systematically follow instructions that will lead you to an eligible household. You will then select a

mother with at least one child who is five years old or less in that household. You will need to be

thorough, polite and diligent in administering the questions as provided below.

Read the questions and instructions carefully and record the responses legibly. Refer to you

supervisor when you encounter any challenge or when you are in doubt.

Time: Started: _______________

Section A: Particulars 1. Questionnaire no: MT/UB2/-------/----/0--

2. Initials of Enumerator:_____________ 3. Date (DD/MM/YY): __/__/2013

4. Name of Community/EA__________________________________________

Good morning/afternoon. My name is ……………………….. I am a member of a research

team collecting information on behalf of the Government of Liberia in collaboration with

support from UNICEF. The project in this community has an aim to improve access to safe

Water, Sanitation and Hygiene situation. I would like to speak with the head of household or a

mother who has a child not more than five years old about the water, sanitation and

Hygiene situation in your compound/unit and also in this community. I will be grateful if you

take some time to answer them. You are under no obligation to answer any of the questions.

However, if you choose to participate, nothing you say will be used against you now or in the

future. I also promise that the information you give will be used solely for the purposes of the

project only. Do I have your permission to proceed with the discussion?

a. Yes, agreed (continue with Interview) b. No, (Discontinue and look for respondent).

7. How long have you lived in this community? ___/___/ (YY/MM) Note: (If less than 3months

discontinue and find another respondent.

8. How old is your youngest child? ………….

a. If the child is 5 years or less (Continue with interview).

b. No child or more than 5 years (Discontinue and find another respondent).


48

Section B: Background Information

1. Name of Respondent:____________________________

2. No. of household members: _________ (List names of members of the household overleaf)

3. State of occupancy

1. Owner occupied 3. Squatter

2. Tenant 4.Living with a host family

5. Others (Specify)……………

4. Highest Educational

level:

1. None 2. Primary 3. Secondary 4. Tertiary/College. 5. Others

5. Occupation: 1-Farmer. 2-Trader. 3-Teacher. 4-Public Servant. 5- Unemployed. 6-Other

(specify)………..

6. What is your legal marital status?

1-Single; 2-Married; 3-Separated; 4-Cohabitation; 5-Widow(er); 6-Others (Specify………..)

7. How much money do you earn in a day?

( Record any amount in LD$ ……….. and convert to nearest equivalence in US$ ……….

Section C: HWTS drinking water

1. What is the main source of drinking water for your household?

1. Pipe in dwelling 2. Public tap 3. Tube well or Borehole 4. Protected hand-dug-well (fitted with hand pump). 5. Rain water. 6. Unprotected wells/Open wells. 7. Surface water (Pond/River/stream etc). 8. Tracker/vendor 9. Others (Specify)……………………………………

2. How long did it take to repair your water facility the last time the facility broke

down?

1. Three days or less

2. Between 4 to 7 days

3. Between 1 – 2 week

4. Three weeks or more

5. Never had a breakdown (Explain why?....................................................................)

6. Not Applicable

7. Others (Specify)………………………………………………………………………..)

3. How far is your main source of drinking water from your dwelling place?

1. On Premises 2. <100 metres (Less than 30 minutes in and out). 3. 100 meters - <500 metres. 4. 500 metres – I kilometers.


49

5. More than 1 Kilometres 6. Don’t Know.

4. Does your main source of drinking water serve you throughout the year?

1. Yes (Skip to Q6)

2. No

5. If no, what other sources are accessible to you?

1. Protected (Public Tap /Small water system, Hand dug well or borehole with pump)

2. Unprotected underground water (hand dug well, etc)

3. Unprotected surface water (River/stream/pond)

4. Others (specify)………………………………………………

6. Who fetches/draws drinking water for your household? (Accept multiple entry)

1. Myself 2. Children 3. My partner 4. Siblings/Family members or friends 5. Others (Specify)…………………………………………………………………….

7. In what containers do you mainly store your household’s drinking water?

(Note for EA: Verify container before you select an answer (Accept multi-entry).

1. Bucket with lid.

2. Jerry can with cap

3. Jerry can without cap.

4. Wide containers with lid

5. Others (specify)……………………………………..

8. What quantity of drinking water do you use in your household a day?

Note: Indicate actual volumes reported following the steps listed below;

a. Type of container

b. Estimated volume (or size)

c. Number of times water is drawn

d. Estimated volume per day:………………………….

9. Is there anything you expect to be done in order to make your drinking water safe to

prevent cholera and other diarrheal diseases?

1. Yes (Explain) ……………………………………………………………………

2. No (Explain) ……………………………………………………………………

3. Not sure (Why?) ……………………………………………………………….

10. Do you fix (treat) your household drinking water?

1. Yes (skip to Q12)

2. No

11. Why do you not treat your drinking water?

1. There is nothing I can use to fix (treat) (skip to 13). 2. I cannot afford the cost of treatment (skip to 13). 3. It takes too much time to treat (skip to 13). 4. Water treatment gives water a different taste or smell I do not like (skip to 13).


50

5. Others (Specify)………………………………………………………

12. How often do you treat your household drinking water?

1. Each time I fill the container with fresh water from the source. 2. As often as I feel there is a need. 3. Anytime I have access to the chemical /treatment agents. 4. I do not treat my drinking water.

13. Are you able to buy adequate safe drinking water for your household?

1. I am able 2. I am not able due to lack of resources / finance 3. It is not my priority. 4. Others (Specify)………………………………………………………………..

14. Did you receive any education on hygiene in the past 9 to 10 months?

1. Yes

2. No. (Skip to Q.16)

3. Can’t remember. (Skip to Q.16)

4. Others (specify)………………………………………………. (Skip to Q.15)

15. What are you doing differently because of the hygiene education received (Accept

multiples responses).

1. Acquired jerry can. 2. Fix (treat) drinking water with WaterGuard/PUR/AT. 3. I use bleach powder 4. Others (Describe…………………………………………………………..)

16. Do officials come to test the quality of drinking water in your home?

1. Yes

2. No (Skip to 18)

17. What were you told the last time any official visited your house to test the quality of

water in the past few months (six months)?

1. I was told to treat my water. 2. I was asked to obtain a jerry can 3. I was not told anything. 4. There was no such visit in the past ten months.

18. Did you pay for safe drinking water at any one moment?

1. Yes 2. No 3. Others explain (………………………………………………………….)

19. Are you and your household able to pay for safe and adequate volumes of water

always?

1. Yes

2. No.

3. Don’t know


51

Section E: Sanitation

1. Where do you and your household members defecate regularly?

1. Family Latrines

2. Public / Communal (Shared latrine).

3. Dig and bury (Skip to Q6)

4. Open defecation (At the beach, in bush, poly bags, others (Explain)?(Skip to Q6)

5. Others (specify)…………………………………………………………… (Skip to Q6)

2. How far do you walk in order to use a latrine?

1. In dwelling house

2. Up to 100 metres away (15 min)

3. More than 101 metres (30 minutes or more)

4. Don’t know.

3. Do you have hand washing facilities attached or close to the latrine?

(Note for Enumerator: Verify the availability of HW facility before you select

option 1 (Facility must be seen from the latrine or less than 10 steps to see it).

1. Yes 2. No

4. Who cleans or maintains the latrine you use? 1. Female adults in the community 2. Male adults in the community 3. Children 4. Other members of the family

5. The community or community groups/ Members of the camp. (specify)

5. Do you have separate latrines or designated places for males and females to defecate? 1. Yes 2. No

6. How do you dispose of children’s feces in this household? 1. Throw it along bath water into the open 2. Dig and bury 3. Use chamber bucket and drop it in the latrine 4. Others (Specify)…………………………………

7. Do you pay to use a latrine at any one moment?

4. Yes 5. No…(Skip to Q9) 6. Others explain (……………………………………………………(Skip to Q9.)

8. Are you and your household able to pay for each time you use the latrines?

1. Yes

2. No

3. Don’t know.


52

9. What education have you received as a result of the Urban WASH project? (Multiple

response) 1. Regular hand washing with soap at critical moments (after latrine, before cooking and

eating, after cleaning a child’s feces and before feeding a child) 2. Safe Treatment and storage of water (fixing of drinking water at the point of use) 3. Use of latrine and abandoning open defecation. 4. Others (Specify)…………………………………………………………………………..

10. What are you doing differently as a result of the hygiene education you received?

(Accept multiple responses).

1. I avoid open defecation. 2. Dig and bury my feces. 3. Intend to build or built a household latrine. 4. Keeping the environment clean. 5. Not applicable. 6. Others (Specify)……………………………………………..

11. What Challenges do you face with access to and use of latrine your use household?

………………………………………………………………………………………….

………………………………………………………………………………………….

………………………………………………………………………………………….

_______________________________________________________________

Section F. Garbage disposal

1. How do you dispose household garbage in the house? 1. Covered household refuse container 2. Open household refuse container 3. Dig and bury (Skip to Q3) 4. Indiscriminate disposal 5. Burn. (Skip to Q3) 6. Others (specify)……………………………………

2. Where do you finally dispose of garbage in this household? 1. Public refuse skip 2. House to house collection 3. Dig and bury 4. Burn 5. Throw away in the open 6. Others (Specify)…………………………………………………………….

3. Do you consider your methods of disposing of garbage in this household appropriate? 1. Yes …………………………………………………………………… 2. No. (Why Explain?) …………………………………………………


53

4. What challenges confront your household on garbage disposal? (Accept multiple

responses) 1. Dirty surroundings 2. Fly nuisance 3. Smelling neighborhood 4. No challenge 5. Others (Specify)……………………………………………………….

_______________________________________________________________

G. Hand Washing With Soap (HWWS)

1. Do you wash your hands with soap regularly?

1. Yes 2. No (skip to Q3)

2. Under what circumstances would you wash your hands with soap? (Accept

multiples responses but do not suggest any of the options below!!)

1. Before meals, 2. Before feeding a child. 3. Preparing/serving a meal. 4. After latrine 5. After cleaning a child’s feces or changing dippers 6. After meals 7. Others (Specify) …………………………………………………………….

3. Do you think there is any benefit in washing your hands with soap regularly?

1. Yes 2. No (skip to Section H).

4. What are you doing differently as a result of the hand washing education?

1. Wash hands with soap regularly. 2. Build/built/would buy hand washing facilities. 3. Educate child (ren) to wash hands regularly. 4. I was not educated/Don’t know. 5. Others (Specify) ……………………………………………

5. How many episodes of diarrhea have your child (under five years experienced in

the past six months? (During / After the WASH programme).

1. None

2. Once

3. Twice

4. Thrice

5. More than thrice.

6. Others (Specify) ………………………………………………………...

6. How many episodes of diarrhea has you child (under five years) experienced six

months previously (Before the WASH programme)?


54

1. None

2. Once

3. Twice

4. Thrice

5. More than thrice

6. Others (Specify) ……………………………………………………….

7. Probe if there is more reported diarrhea during/after the project than before.

1. Yes ( Explain)…………………………………………………………..

2. No ( Explain)……………………………………………………………

H: Assessing behavior triggers

1. Are you able to making good hygiene decisions to prevent diseases and improve your

health? 1. Yes 2. No. 3. Not sure / Don’t Know

2. Who matters to you most when it comes to taking household WASH related decisions? 1. Myself 2. My children 3. Landlord/lady 4. Religious leader 5. Traditional leader 6. Head of family 7. Political representatives 8. Government Officials 9. Others (Specify)…………………………………………………………………..

3. Who influences water, sanitation and hygiene related behaviors in this unit or household? 1. Myself 2. Landlord/lady 3. Head of family 4. Community leader 5. Husband 6. Other tenants / family members/settlers. 7. Public Health Inspectors 8. Community volunteers 9. Others

4. Who takes the important water, sanitation and hygiene related decisions in this

household? (Accept multiple responses) 1. Landlord/lady 2. Head of family 3. Husband 4. Myself 5. Other tenants / family members 6. Public Health Inspectors


55

7. Community volunteers 8. Others

___________________________________________________________________

_____

I. Preferred sources of information and education

1. Which sources do you receive information including WASH education programs from?

(Accept multiple responses. Note; There should be no prompting).

1. Community Leaders

2. Radio

3. Cinema/Video shows

4. Health / Environmental workers

5. Community Volunteers

6. Friend/relatives

7. Posters

8. Billboards

9. Others

2. What benefits or changes have occurred in your life and those of your household

members since the WASH programme was implemented in this community? (Note: Accept

multiple entry).

1. Improved water quality.

2. Easy access to water.

3. Decline in diarrhea diseases.

4. Less medical expenses.

5. Clean environment/less houseflies.

6. Others (Specify)…………………………………………..

3. Is there any information you feel like passing on to help you access WASH facilities and

services?

………………………………………………………. ………………………………

………………………………………………………. ……………………………… End of the interview. Thank you very much.

Section J: Pool Test

Take a sample of water from the household’s drinking water source. Apply test and record

the result below.

After testing I found:

1. There is no indication of a presence of chlorine (Water did not change colour). 2. There is some residual chlorine but it is below 0.2 mg/l. 3. Residual chlorine >0.2 mg/l exist. (Comment ………………..) 4. There is no drinking water to be tested.

Time Ended: __________(hh/mm)

____________________________________________________________________


56

Section K: Latrine User Observation checklist

County: ………………………………… District…………………………..

Community……………………… …… Type: ……………………………

(1. Household 2. Cty 3.School)

Observations: Comments:

1. State of latrine (Describe the state of latrine);

a. Is the immediate environment clean? 1. Yes 2. No (…………………………)

b. Proper disposal of anal cleansing? 1. Yes 2. No (…………………………)

c. Is there Odor? (Foul smell) 1. Yes 2.No (………………………….)

d. Is it accessible to children? 1. Yes 2. No (………………………….)

e. Is there any defect on the structure? 1. Yes 2. No (………………………….)

2. Is there hand washing facility near latrine? 1. Yes 2. No (………………………….)

a. Hand washing facility near latrine? 1. Yes 2. No. (………………………….)

b. Hand washing facility has soap/ash? 1. Yes 2. No (………………………….)

c. Is there water available for washing? 1. Yes 2. No (………………………….)

d. Availability of soap on site 1. Yes 2. No (………………………….)

3. Observation of an individual latrine user:

f. The person using the latrine wash hands? 1. Yes 2. No (…………………………)

g. Hand washing done properly? 1. Yes 2. No (………………………….)

Initials of observer:…………………….. Date:…………………………………….

Supervisor’s Initials: ………………….. Date: ……………………………………


57

Section L: Key Informant Interview Guide for Community Animator (KII 1)

County……………………. District ……………………… Community………………

Name of Informant……………… Date:………………… Time stated…………

1. Did you receive any training or orientation on Urban WASH service delivery?

1. Yes.

2. No. (Skip to Q6)

3. Others (Specify)……………………………………………………………………………

2. Who carried out the training?

1. Name(s) ………………………………………………………………………………………. 2. Organization) ..………………………………….. ………………………….……………….

3. How long did you train;

1. A few hours,1 Day,2 Days, 3 days and Others (Specify)…………………………………………………...

4. What were the contents of Urban WASH Programme training you received(Accept

Multiple responses):

1. HH water treatment 2. Sanitation (Explain………………………………….) 3. Hand Washing with Soap 4. Monitoring water quality 5. Others (Specify)……………………………………………………………………..

5. Benefits derived from the training: (Knowledge/skills, Allowance, Other incentives,

Nothing, Others. etc,)

1. Nothing …………………………………………………………………………………………

2. Allowance/Cash………………………………………………………………………………

3. Collaterals and other incentives… and 4. Others (Specify)………………….

6. Activities planned to be carried out at the end of the training:

o HH promotion of Urban WASH programme o Community promotion of Urban WASH programme o Train other colleagues, Others

7. Activities carried out:

o …………………………………………………………………………………………… Note: Verify plans

6. Achievements (list below)

……………………………………………………………………………………………………………

7. Challenges encountered.

8. Suggestions and future actions.

9. If you have another opportunity what will you do differently?

End of Interview - Thanks you very much.


58

Interview Guide for Community Health Volunteer (KII 2)

URBAN WASH Evaluation - MONROVIA

County…… …………………….. District ………………… Community………………………..

Name of Informant…………………… Date:………………… Time stated……………………

1. Contents of Urban WASH training received:

a. HH water treatment b. Sanitation c. Hand Washing with Soap d. Monitoring water quality

2. Training was carried out by (Name(s) & organization) ……………….

How long did you train;

a. ------Days b. ------Hours per day (time).

3. Benefits derived from the training: (Knowledge/skills, Allowance, Other incentives, Nothing,

Others. etc,)

……………………………………………………………………………………………

4. Activities planned to carry out at the end of the training:

a. HH promotion of Urban WASH b. Community promotion of Urban WASH c. Train other colleagues d. Others

5. Activities carried out:

o ………………………………………………………………………………………Note: Verify plans

6. Achievements (list below)

…………………………………………………………………………

7. Challenges encountered.

………………………………………………………………………………

8. Suggestions and future actions.

………………………………………

9. Any other comments?

…………………………………………………………………………………….……………

End of Interview - Thank you very much.


59

Key Informants Interviews –WASH Committee Leader/Member (KII 3) Evaluation of Urban WASH

County…………………………. District ………………… Community………………………..

Name of Informant…………………… Date:………………… Time stated: __/__/ (hh/mm)

Introduction

Introduce yourself and project and explain the purpose of the interview.

Permission to take notes and start.

1. WASH Package Training

Date and duration and trainers

List & number of CHVs/ WASH Committee members trained.

Contents of training / Promotion of package;

o Distribution /sale of WaterGuard o Monitoring of water quality o Handwashing with soap o Training in CLTS

2. Post training activities planned (Ask for plans)

3. Post training implementation activities carried out (reports)

List activities implemented (for each activity ask for outputs or number of people reached).

1. ……………………………………………………………………………………………

2. ……………………………………………………………………………………………

4. HouseHold Water Treatment (HHWT)

Source and state of drinking water in the community since the last 6 months

Use of WaterGuard.

5. Sanitation status of communities

6. Knowledge of outputs produced during the program:

hand – dug - wells / bore holes

Latrines and hand washing facilities

Supplies; Soap and WaterGuard, etc

7. Perceived benefits from the program to the community (List): …………………………..

…………………………………………………………………………………..………………….

8. Challenges encountered suggestions and lessons learnt.

…………………………………………………………………………………………….

…………………………………………………………………………………………….

9. End of Interview. Thank you. Time ended.___/___/ (hh/mm)


60

Urban WASH Program Evaluation Key Informants Interview Guide for School Club Coordinators (KII 4)

County…………………………. District ………………… Community………………………..

Name of Informant…………………… Date:………………… Time stated……………………

Themes

1. Urban WASH Training

Date and duration and trainers

List & number of CHVs, WASH / SHC members trained.

Post training implementation plans (verify plans)

Outputs (structures and supplies provided under the project); o hand – dug - wells / bore holes o Latrines; Schools/wells o Hand washing facilities o Supplies; Soap, WaterGuard

2. Activities carried out as promotion of Urban WASH after training.

Distribution /sale of WaterGuard

Monitoring of water quality

Follow-up activities: types and frequency 3. Functions of WASH committees/clubs Explore: Promotion of WASH education, Manage WaterGuard stocks, Manage Waterguard revolving fund (Verify books or minutes). 4. Water sources: adequacy and quality

Source of drinking water in the school;

State of hand pumps.

Quality and quantity of water. Water treatment with WaterGuard; who, how, frequency, etc Information onWaterGurd Training on WaterGuards. Procurement of WaterGuards in the School

5. Maintenance of hand pump (plans & implementation)

who, when, how much

Breakdowns and duration. 6. Sanitation – latrine School latrine (Availability/Type, Number/adequacy, State of latrine, etc). 7. Hand washing with soap (HWWS)

Adequate hand washing facilities with soap

The state of HW facility Community in the schools since the last 6 months. o Current state; available / flow of water / availability of soap

End of interviews. Thank you.


61

Key Informant Interview Guide Partner Organizations (KII 5)

Evaluation Urban WASH - II Monrovia

UNICEF - Liberia

Name of Organization: ……………………… Name of Informant: ……………… Date:…….

County: …………………………. District: …………………….. Community: …………………

Key Themes of Urban WASH - II:

1. Community Mobilization for WASH

How was community mobilized

When and for how long this was done

Any training in the community and purpose of training

Social and management structures in place.

2. Hygiene

How Hygiene promotion was: o Planned o Implemented o Quality of construction and other WASH services provided o Monitored o Evaluated

Support provided for promoting HWWS

3. Sanitation

Type and number of training

List and address of trainees.

Contents of training.

Post training activities implementation plans developed.

Adherence to plan. 4. Planned sustainability of latrines in community and schools

Training on WASH in schools for Principal and leaders.

Use fees

Number of user a day

Cost per use

Frequency of de-sludge

Cost of de-sludge latrines

Cost paying caretakers and detergent.

Cleansing, maintenance and rehabilitation of school latrine: o Who o Frequency

Support provided for proper use, cleansing and rehabilitation of latrine.

5. Promotion and sale of HWTS

How is promotion of WaterGuards done?

Activity reports available?

How many HHs visited

Types of community events held to promote WaterGuard o Education & distribution of WaterGuards


62

o Monitoring of waterGuard Distribution Proper use

o Availability of WaterGurads. 6. Management and Maintenance of Hand pumps/water kiosks

How facilities are managed

Maintenance and rehabilitation of hand pumps in communities and school o Who o Frequency o When

7. Management Arrangements

Satisfaction with implementation arrangements and support from by UNICEF

Design of water points/latrines meet needs/customs/traditions, disabled, poor, aged

etc.

Assessment of works

Program Cooperation Agreement (PCA)

Monitoring and Reporting 8. Performance/Capacity of WASH committees to sustain activities

Meetings

Decisions and Actions carried out

Verify minutes and some of the activities 9. Coordination and collaboration

Type

Frequency

Names of organizations

Outcomes of collaboration

10. Challenges and suggestions

Challenges and how these were solved: 11. Other matters

End of Interview Thanks


63

Key Informant Interview

WASH Sector Agency Government / Development Partners (KII 6)


UNICEF - Liberia



Introduction 1. Knowledge of Urban WASH II, Agencies responsible and contribution sector strategic. Contribution to the sector & impacts if any. 2. Coherence /Relevance of the project. Consider Designs, construction and access to most needy urban poor.

Compliance & adherence to sector policy/strategy and standards:

Water supply and technology options

Sanitation and latrine options constructed

Hygiene education and promotion 3. Appropriateness of the project to location and beneficiaries 4. Sustainability and related issues. 5. Effectiveness of the project; Quality of work and involvement of communities 6. Efficiency of Urban WASH –II 7. Coordination and collaboration with other sector players. 8. Satisfaction with level of implementation 9. Performance and best practice issues. 10. Lessons from previous and current Urban WASH projects. 11. Challenges and actions taken. 10. Any suggestions


64

Key Informant Interview

Special Audiences and Stakeholder Organizations (KII 7)


UNICEF - Liberia



Introductions: This guide applies to special audiences like People Living With Disability and Disable Association, etc.

Children’s Access to WASH facilities

Disability access to WASH facilities

Access by minority groups

Access by aged and poor

PLWA/HIV 1. Knowledge of Urban WASH – II and other WASH projects 2. Relevance and level of satisfaction 3. Specific WASH needs that are unmet and WASH expectations. Consider;

Involvement

Processes

Construction and other outputs

Religious/Cultural needs. 4. Challenges and suggested solutions. 5. Other issues


65

Client (UNICEF) Perspectives

Key Informant Interview Guides

Evaluation Urban WASH - II Monrovia (KII 8)

1. The impact of the project Changes observed in project communities and likely impact 2. The coverage of the project

Project reports o Estimated pop o Process of selecting of communities o Facility drawings o Planned/reached.

Progress reports o No. designed for o No. reached and covered

Performance of IPs o Satisfaction with level and quality of work by IPs o Quality of work o UNICEF monitoring reports o Comparing plans versus drawings with structures

3. The coherence of the project

Compliance with GoL WASH policies: o Strategy/Policies. o Unicef policies/Medium Term Plan o Implementation standards o International: Sphere, etc.

Design and implementation of Hygiene Promotion in Urban WASH II. o Inter-personal personal Communication o Mass media materials:

Audio Visuals (Posters, etc) Frequency Costs etc

Extent of adherence to project proposal. Changes to original proposal and reasons.

PCAs.

Facility drawings.

Project status/Progress reports: o To ECHO o From partners o Coordination and collaboration with Government and sector agencies.

Challenges: o Natural disasters/Accidents. o Delayed take-offs o Differences between plans and realities on the ground.


66

4. Efficiency Extent of deviation from original estimates (per project proposals)

o PCAs o Project cost and competitive prices. o Distances / accessibility o UNICEF financial report for the period.

5. The appropriateness / relevance of the approach

Extent to which identified needs and priorities of the target population has been planned and met.

Beneficiaries involvement in project activities.

including women in the project design, implementation and monitoring

participatory approach developed/ used for assessment and project implementation

5. Accountability of the project

Project monitoring arrangements; o Water quality o Construction of WASH facilities o Hygiene Promotion

Population per facility.

User statistics. 6. Lessons learnt Proportion of lessons ploughed into design and implementation from Urban WASH I to

Urban WASH-II

Changes were made to improve program design and implementation

Best practices and lessons learned from Urban WASH-II for future projects 7. Extent to which recommendations of and lessons learnt from UW-I & II and Baseline assessment were implemented; Consider.

Developed implementation plans

Incorporated into existing plans.

Other related projects

9. Suggestions End of Interview


67

Appendix 3: Proposed Work Plan – Evaluation of Urban WASH – II

Activities Date Dec 12 Jan 13 Feb 13

1. Planning and finalization of Administrative processes

12th Dec – 17th Dec 12

2. Development of detail project work plan and recruitment of enumerators.

19th Dec – 21st Dec 12

3. Literature review/ review of relevant project documents

19th Dec – 5th Feb.13

4. Arrival 27th Dec 2012

5. Review and revision of data collection tools.

20th Dec – 28th Dec 12

6. Scheduling KIIs interviews with stakeholders

27th Dec.- 30th Jan 2013

7. Refresher/Training and field testing data collection instruments

8th –11th Jan 2013

8. Printing of data collection instruments

11th – 12th Jan 2013

9. Data Collection 14th – 20th 2013

10. Develop tabulation and data analysis plan

14th –20th January 2013

11. Field supervision and review of completed questionnaire

14th –20th January 2013

12. Data Analysis and draft reports and conducting KII interviews

20th – 29th Jan 1013

13. Data Entry and processing 30th Jan 2013

14. Validation workshop/ Submission of draft reports 30th Jan 2013

15. Unpaid Break 31st Jan – 7th Feb 2013

16. Feedback from client (UNICEF:M&E& WASH) 7th Feb 2012

17. Revision of draft evaluation report.

8th Jan – 11th Feb 2013,

18. Submission of final evaluation reports. 11th Feb 2013

19. Departure 12th Feb 2013


68

Appendix 4: Map Showing Parts of Montserrado and Monrovia

Map of Liberia with Portion Shaded Brown Showing Broad Project Areas Where Urban

WASH II was implemented

Appendix 4a: Map of Monrovia Showing Project Communities in shades


69

Appendix 4b: Geo-Referenced Map of Monrovia

Geo-Referenced Map of Monrovia Showing Project Locations and Implementing NGOs

Sample of WASH Facilities provided under the project


70

Appendix 5: Project indicators and baseline results

Indicators Baseline Indicators

1. Contribution to the reduction of incidence of cholera and other diarrheal diseases in slums in Monrovia.

Number of Cholera and other Diarrhoea disease 2012 - 1277

2. Users/improvement in access to water points.

Improved sources -reported (226) 70.1%

3. Users/latrine cubicle

There is no data on this and a survey

methodology could not have provided one except

for total enumeration of the project area.

4. Percentage of population reporting to be washing their hands always or frequently with; soap before feeding children; after cleaning up following children’s defecation; before preparing food; and after using toilet.

a. Reported regular hand washing soap (285) 88%.

Before meals 13%

Before feeding a child. 43%

Preparing/serving a meal. 11%

After latrine 70%

After changing dippers 26%

After meals 2%

5. Percentage of drinking water samples tested in the home with free chlorine residual >0.2mg/l

Pool Test Results # %

There is no indication of presence of chlorine 103 32.0%

< 0.2 Free Residual Chlorine 45 14.0%

Residual Chlorine > 0.2mg/L 41 12.7%

I do not treat water 133 41.%

6. Increased percentage of households transport and store their water in 5 gallon (20 litre) plastic jerry cans.

Jerry can with cap (Estimated) 67 20.8%

7. Percentage of households using a pit latrine or flush latrine.

Family Latrines (pit and pour Flush) – 39.4% (127) Public shared latrine – 32% (104) Open defecation – 18.6% (60)

8. Percentage of population observed to wash hands with soap after using latrine.

Observed hand washing after defecation – 20 (30.3%)


71


72

Appendix – 5: List of stakeholders

UN:

UNICEF – M&E, CSD & Education

Government:

MoE – School Health Unit

MPW

MoH

NGOs:

FAAL

CODES

Oxfam GB

LICH

Donors:

SIDA

Government of Japan

European Commission Government WASH Agencies

• MPW • MoH&SW • Director Hydro Div / Energy and mines. • LWSC • MCC

Private WASH service providers:

o LIBRA Sanitation o NC Sanitors


73

Appendix 6: List of Implementing Agencies

URBAN WASH IMPLEMENTING NGOs WITH CONTACT - URBAN WASH 2011

Implementing

Agency

Category Nane of

Contact Person

E-mail Telephone

1. Community

Development

Services (CODES)

National Tamba Bundor [email protected] 0886514325

2. Foundation For

All Ages Liberia

(FAAL)

National Daogogoe H.

Fahnbulleh

[email protected] 0886570760

3. Oxfam Great

Britain (OGB)

International Precious Dennis [email protected] 0776364397

4. (LICH) National Jennis Barclay 0886531920

Appendix 7: List of Persons Contact During the Assignment

Name Designation/Address Tel/Cel Contact

Zainab Al-Azzawi M&E Specialist

Sam Treglown WASH Specialist, UNICEF Liberia

Faisah Sadat M&E Specialist, UNICEF Liberia

Edwin Rogers WASH Specialist, UNICEF Liberia

James Massaqoui WASH Specialist, UNICEF Liberia

Paul Jablai WASH Specialist, UNICEF Liberia

Joe Nyan LISGIS, Monrovia

Jefferson W. Wylie Hydrology. Min. Land, mines and

Energy

Dehwehh Yeabah MoH&SW

Amos Gborie MoH&SW

M. Gailey Barclay LICH

Den Vamba LICH 0886434097

Zoe Kanneh Ministry of Education (School

Health)

Dala T. korkoyah Ministry of Gender and Development

J. Diggs St. John River Schools

John Payne Suwetha Memorial Public school

Jeff Nyadibo NC Sanitation Services 0886581363

Tamba CODES

Hassan Fambulleh FAAL

Caroline Page Dir. Env Health Dept, MCC Monrovia 0886415951





74

Appendix 8: Summary of Project Outputs

1. Water

Water Facilities Number Estimated Beneficiaries

Taps 66 16,500

Water points 9 4500

Totals - 21,000

2. WaterGuard

WaterGuard Distribution

Date in 2012 Qunatity Distributed

June 12,000

September 19,032

October 11,046

November 32,040

December 32,040

Total 106,158

3. Latrines Facilities

Number of Cubicles

Locations Under-construction Completed Total

Communty 36 102 138

Schols 16 72 88

Total 52 174 226


75

Appendix 9: Financial Sustainability Estimates for WASH Facilities and Services

URBAN WASH-II - Financial Sustainability for Water and Sanitation (Latrines)

Assumptions:

1. Price is constant

2. Number of Users remain at current levels or increase

3. Average expenditure per/capita declines or remain constant.

4. Care takers keep basic data on number of users, amounts realized and account on daily/regular basis.

5. Community WASH Committees

Tariff based on estimated 2000gals per day

Water Tariff in

$LD

Volume/Gal Tariff

per gal

Month

Quarter Annum US$Equiv. comments

1. Bulk water supply

from LWSC

5 30 0.17 12200 36600 146400 2091.43 Simulation

2. Tariff at water kiosk 5 15 0.33 19800 59400 237600 3394.29

3. Expenses

(Cleaning & minor

repairs)

2000 6000 24000 342.86

4. Gross Balance 0.16 7600 22800 91200 1302.86 (sum of 2&

3) -1

Sanitation Work sheet

Particulars/Items Oxfam* FAAL CODES LICH

Cost of Latrines

Consumables:

Detergent and T/rolls 40 40 40 40

Payments for cater

taker LD$3500 /month 116.67 83.3 63.33 50

De-sludging (US$

150) X Frequency

(2x/month) 700 466.67 466.67 373.33

Totals 856.67 590.00 570.00 463.33

* Estimates


76

Sustainability of latrines

Number Rates/

User

Amounts

in $LD:

Particulars Users/Day $LD Day Monthly Quarter Annual US$

Equivalence

Comments

a. Income

1. Oxfam (King

Gray)

65 5 325 9750 29250 117000 1671.43 Yet to receive

completed data

2. FAAL 180 5 900 27000 81000 324000 4628.57

3. CODES 250 5 1250 37500 112500 450000 6428.57

4. LICH (Clara

Town)

200 5 1000 30000 90000 360000 5142.86 Estimated cost

of

consumables

5. Total 695 3475 1251000 17871.42 Sum: A1 &A4

Expenditure Average

cost/User

Amounts

in LD$:

b. Expenditure Day $LD Day Monthly Quarter Annual US$ Equiv. Comments

1.Oxfam (King

Gray)

856.67 13.18 856.67 25700.1 77100.3 308401.2 4405.73

2. FAAL 590 3.28 590 17700 53100 212400 3034.28

3. CODES 570 2.28 570 17100 51300 205200 2931.42 Not started de-

sludging yet

4. LICH (Clara

Town)

464 2.32 464 13920 41760 167040 2386.28

5. Totals 2480.67 2480.67 893041.2 12757.73 Sum:B1-B5

c. Balance - Rates per/day Month Quarter Annual US$ Equiv. Comments

1. Oxfam (King

Gray)

-8.18 -531.67 -

15950.1

-

47850.3

-191401 -2734.30

2. FAAL 1.72 310 9300 27900 111600 1594.28

3.CODES 2.72 680 20400 61200 244800 3497.14

4. LICH (Clara

Town)

2.68 536 16080 48240 192960 2756.57

5. Totals 994.33 357958.8 5113.70 A5-B5


77

Appendix 10: Implementing partner performance related results

1. Highest proportion of respondents with protected sources of drinking water

CODES FAAL LICH Oxfam

a. CODES - Overall access 4 2 1 3

b. Oxfam - Protected HDW/BH,

c. FAAL – Public Taps (Kiosks) and Pipes in Dwellings

d. LICH

2. Highest proportion with all year round sources of drinking water

a. Oxfam, b. CODES, c. FAAL, d. LICH 3 2 1 4

3. Highest proportion making shortest distances

a. CODES, b. Oxfam, c. LICH, d. FAAL 4 1 2 3

4. Highest Proportion of respondents who reported use of recommended containers

a. FAAL, b. LICH, c. CODES, d. Oxfam 2 4 3 1

5. Highest proportion of respondents reporting use of treated of water with WaterGuard.

a. LICH, b. Oxfam, c. FAAL, d. CODES 1 2 4 3

6. Proportion of respondents who reported IP WaterGuard monitoring visits

a. LICH, b. FAAL, c. CODES, d. Oxfam 2 3 4 1

6a. Proportion of positive pool test results indicating appropriate used of WaterGuard by IP

a. LICH, b. FAAL, c, CODES, d. Oxfam. 2 3 4 1

7. Proportion of Respondents who reported access to Shared latrine and Open Defecation by IP

a. LICH, b. FAAL, c. CODES, d. Oxfam. 2 3 4 1

8. Proportion of respondents reported covering household garbage appropriately by IP

a. Oxfam, b. LICH, c. CODES, d. FAAL 2 1 3 4

9. Proportion of Respondents who reported regular HWWS by IP

a. LICH, b. FAAL, c. CODES, d. Oxfam 2 3 4 1

CODES FAAL LICH Oxfam

Total 24 24 30 22

Notes:

a. Highest mark (4) indicates best performnace/ Lowest mark (1) indicates least performnce

b. Blue = Hardware

c. Green = Sofware

d. Refer graphs below as supporting documents.


78

Oxfam FAAL LICH CODES Total

7.9 15.1

4.4 0.9 7.7

26.3

66.4 59.6

44.7 52.6

61.8

9.9 14

51.8

30

Fig 1: Source of drinking water by IP Pipe inDwelling Public Tap Protected HDW/BH

Total Oxfam FAAL LICH CODES

62.3

72.4

58.6 57.9 64.9

Fig 2: All year drinking water source


8.1 11.8 10.5 5.3 5.3

65.8 72.4

51.3 57

89.5

14.5 10.5 19.1 23.7

1.8

Fig 3. Distance to Source of Drinking Water by IP In Dwelling <100 metre/30 Min 100 metre/1Hr


79

Bucket withlid

Bucket withlid

Jerry canwith cap

Jerry canwithout cap

Widecontainers

with lid

Others

16.7 15 16.1

0

25

12.5

33.3 35 31.3

71.4

42.5 37.5

25 24.4 26.4

14.3 20

25 25 25.6 26.2

14.3 12.5

25

Fig 4: Containers for water storage Oxfam FAAL LICH CODES


77 84.2 78.9 86.8

59.6

Fig 5: Treat Drinking Water with WG Treat Drinking Water with WG


82

8 18

46

10

Fig 6: WaterGuard Monitoring - IP Performance

No Chlorine There isChlorine<0.2 m/l

Thiere is Clorine>0.2m/l

There is no waterto test

8.1 5.6

0

4

16.7

8.6

2.1 4

5.6

15.6

2.3 1.6

17

6.3

0.2 2.3

Fig 6a: Evidence of Treating HH drinking Water (Pool Test) by IP Oxfam FAAL LICH CODES


80

7. Proportion of Respondents who reported access to Shared latrine and Open Defecation by IP

Family latrine Shared latrine Dig & bury O. defecation Others

42.8 47.6

0.9 7.9

0.9

55.3

11.8 5.3

26.3

1.3

35.5

57.9

0 4.6 2

29.8

70.2

0 0 0

57

35.1

0 7.9

0

Fig 7: Access to Toilet by C'ty & IP



7.7

13.2

4.6 8.8 7 8.3

3.9 3.9 6.1

19.3 18.4 19.7 17.1

20.2 17.5

3.7 3.9 3.9 3.5 3.5

Fig 8: How HH Garbage is Stored by C'ty & IP Covered Burn Dig & Bury Others


95.6

92.1 96.1

98.2

94.7

Fig 9: Reported Regular Hand washing by IP

managing the transition between emergency · pdf filemanaging the transition between emergency...

Documents