ESTABLISHING IMPACT EVALUATION

FOR THE SUSTAINABLE WATER FUND

Commissioned by the Netherlands Enterprise Agency

ESTABLISHING IMPACT EVALUATION FOR THE SUSTAINABLE WATER FUND (FDW1)

PROGRESS REPORT - JUNE 2016

CO-AUTHORS OF ESTABLISHING IMPACT EVALUATION TEAM REPORT

John Cameron (Team Leader and Ethiopia household survey and institutional analysis)

Getnet Alemu (Ethiopia household survey and institutional analysis)

Niek de Jong (Colombia institutional analysis)

Elena Gross (Ghana farm survey and institutional analysis)

Carley Pennink (Ghana institutional analysis)

Maximiliane Sieverti (Colombia farm survey)

1 See http://english.rvo.nl/subsidies-programmes/sustainable-water-fund-fdw

1

TABLE OF CONTENTS

EXECUTIVE SUMMARY 2

SECTION ONE: OVERALL REPORT INTRODUCTION 4

SECTION TWO: INSTITUTIONAL ANALYSIS OF FIVE FDW PPPs 7

SECTION THREE: COLOMBIA HOUSEHOLD SURVEY 34

SECTION FOUR: HOUSEHOLD SURVEYS FOR IMPACT EVALUATION OF SUSTAINABLE WATER SERVICES (SWSH) IN HARAR, ETHIOPIA 91

SECTION FIVE: HOUSEHOLD SURVEY BASELINE REPORT ON INTEGRATED WATER MANAGEMENT AND KNOWLEDGE TRANSFER IN SISILI KULPAWN BASIN IN THE NORTHERN REGION OF GHANA 121

2

EXECUTIVE SUMMARY The five FDW Phase 1 projects selected for this FDW impact evaluation (FDWIE) are now about halfway through their planned lives. Most are lagging in terms of implementation and it is expected that activities will accelerate in the coming two years, though in several cases it now seems unlikely that all planned activities will be completed by the planned completion date. Therefore it is too early to draw any firm conclusions on the performance of the partnerships and their impact on people’s lives. But some preliminary insights on sustainability may be useful in assessing future calls for projects in the FDW programme.

To organise these insights a variant on the Netherlands government’s FIETS framework is utilised.

a. Finance: the subsidies from the Dutch government are unlikely to be substituted by additional PPP partner funds after the projects come to a formal end. The subsidies are playing a significant role in physical investment that should have a life well beyond formal project completion. The partners are certainly financial capable of operating and maintaining these investments into the indefinite future and in most cases have good reason to do so. A second use of the subsidies is to reduce the risks in innovative activities, especially by private sector partners. Whether this has been successful in terms of future unsubsidised activities will be clear by the time of project completion; current experience is hopeful in this respect. Many FDW project activities are very close to core activities of partners and therefore funding is likely to be available for these activities after the project end.

b. Institutions: the partners had to be well established institutionally to be accepted on the FDW programme and therefore are institutionally secure. Many partners had pre-FDW positive relationships and these are likely to continue independently of the FDW programme. But there are partner relationships in all the projects that can be attributed to the FDW programme and it is the sustainability of these relationships that will be of especial interest in the final impact evaluation.

c. Environment: One of the selected projects has an explicit concern with environmental sustainability. The other projects have physical environmental implications (some potentially positive, others negative) that do not appear to be being explicitly addressed currently. This dimension of sustainability will need to be explored in the final impact evaluation, including possible explanations of why it was neglected if that is the case.

d. Technology: this dimension raises important issues of ‘appropriateness’ and ‘replicability’. All projects are engaged with introducing new technology in capital investment activities, human resource skills, and/or monitoring and evaluation information generation. Whether these innovations can be ‘locally’ sustained will be important for the impact evaluation.

e. Social: there are gaps in terms of gender sensitivity in all the projects, But there are elements of encouraging women’s participation in some of the training activities. The confusions over poverty impact are well described in the FDW Mid-Term Review.

3

This variant on the FIETS framework, updated to include future thinking on sustainability in the Netherlands and globally, will help frame the final FDWEI impact evaluation in 2018. The semi-structured interview and direct observation approaches, augmented by formal questionnaires when appropriate, used in the last two years will continue with a final impact evaluation exercise in 2017/2018.

Baseline household surveys for three FDW projects in Colombia, Ethiopia, and Ghana have been completed as required and descriptions of the survey designs and preliminary results are included in this Report. Impact household surveys will be conducted at appropriate times between now and the middle of 2018.

4

SECTION ONE: OVERALL REPORT INTRODUCTION

The FDW Establishing Impact (FDWEI) project contract was signed in June 2014. The key Term of Reference for the FDWEI project was summarised as follows:

’The Dutch Sustainable Water Fund (FDW) aims to contribute to sustainable economic growth, self-reliance and poverty reduction in developing countries through public-private partnerships (PPPs) in the water sector. The objectives of this evaluation are to estimate the impact and assess the sustainability of a selection of projects in the fund’s portfolio, to assess the value added of the PPPs to achieve results, and, indirectly, to determine the level of success of the fund as a whole.’

The FDWEI team were contracted to study five FDW projects out of thirteen in Phase 1 of the FDW programme. These projects are summarised and their relationship to the whole FDW Phase 1 portfolio are summarised in Table 1.

The Table shows the selected FDW projects account for 5 out of the 13 FDW Phase 1 projects, with CDW/EWU characteristics distributed similarly to the whole population, and budgets distributed around the overall mean for all thirteen projects. Additional considerations were that the selection had to include the Colombian project as it alone accounts for a quarter of the total budget, and that having two projects from both Ethiopia and Ghana was sampling efficient in terms of logistics and, to some extent, controlled for national institutional contexts while keeping some variation. Also all three selected countries were well known by members of the FDWEI team.

The Terms of Reference required two forms of data collection:

a. Institutional data exploring the Public-Private Partnership (PPP) relationships from inception to project completion with projections of sustainability for all five selected FDW projects.

This data was primarily collected through semi-structured interviews with ‘representatives’ of the PPP partners and direct observations of day-to-day PPP activities and formal partner/stakeholder meetings. Timing of baseline interviews varied depending on availability of ‘representatives’.

As the representatives were asked to use recall and there was documentation recording the histories of the projects, the timing of the base-line interviews could be flexible. For two of the projects, a second round of interviews were conducted in the second half of 2015 to explore the dynamics in partnership relationships and observe the creation of wider stakeholder platforms.

The institutional analysis of the PPPs for all five selected FDW projects is contained in Section 2 of this Report and presented in an integrated format combining insights from all five Projects. The semi-structured interviews were conducted on conditions of ‘reasonable’ anonymity so that partner representatives could be frank with minimum risk of information entering the public domain and damaging reputations and/or revealing commercially sensitive information.

5

THE FIVE SELECTED PROJECTS AND THEIR RELATIONSHIP TO THE WHOLE FDW PHASE 1 PORTFOLIO

Lead partner Project code Project title Country Total budget (Euros millions)

Theme (EWU: Efficient Water Use; CDW: Clean Drinking Water)

Colombian Coffee-Growers Federation

FDW12C001

Intelligent Water Management

Colombia 24.5 EWU

Wienco (Ghana) Limited

FDW12GH02

Integrated Water Management and Knowledge Transfer in SK Basin

Ghana 11.8 EWU

Vitens Evides International

FDW12ET06

Source to Tab and Back (STT&B)

Ethiopia 7.1 CDW

Vitens Evides International

FDW12ET03

Sustainable Water Services in Harar, Ethiopia

Ethiopia 5.5 CDW

IRC International Water & Sanitation Centre

FDW12GH06

Mobile Monitoring of Rural Water and sanitation services that last

Ghana 3.8 CDW

Total number of other projects

Other countries

Mean budget (all)

Number EWU/CDW in other projects

8 Kenya, Vietnam, Rwanda, Palestine, Malawi, Sri Lanka, Tanzania, South Africa

7.3 EWU: 2 CDW: 6

b. Three formal randomised questionnaire household surveys to establish and attribute developmental impact on FDW targeted households.

Two FDW Efficient Water Use agricultural projects in Colmbia and Ghana had target populations of farming households that could be effectively sampled and surveyed. One FDW project in Ethiopia aimed at improving drinking water availability and quality was also selected for a household survey, though in this case the overall target population was less well defined..

The timing of the three baseline household surveys has depended on the very different contexts and implementation progress in the three projects in Colombia,

6

Ethiopia, and Ghana. They also had to be separately designed to take account of the very different objectives and activities in the three FDW project proposals. Therefore the baseline surveys are described in three separate sections of this report in their specific contexts - Section 3 (Colombia), Section 4 (Ethiopia), and Section 5 (Ghana).

Questionnaires, semi-structured interview schedules, and direct observation tools are not included in this Report, but will be included as a separate methodology document with the final FDW Evaluating Impact Team Report in 2018.

7

FDW ESTABLISHING IMPACT REPORT, JUNE 2016

SECTION TWO: INSTITUTIONAL ANALYSIS OF FIVE FDW PPPs2

1. Introduction

The FDW Establishing Impact evaluation project (hereafter FDWEI) has evaluating the value added brought by Private/Public/Third Sector project governance arrangements to a range of water interventions subsidised by the Dutch Government with matching funds from the Partners. The FDWEI was instigated to explore only the first phase of the FDW programme in which thirteen projects were instigated. The FDWEI focused on five of these projects in Colombia (one), Ethiopia (two) and Ghana (two).

The FDWIE original Terms of Reference stated:

‘The choice of working with PPPs is motivated by the assumption that the PPP-approach will create synergies between the different partners and therefore add value in reaching the intended development objectives. Determining the success of the fund therefore revolves around verifying if the funded interventions reach the intended development results and what the value added of the partnerships have been in reaching the results.’ (Terms of Reference – Impact Evaluations for the Sustainable Water Fund (FDW), undated, p.3)

This section of the FDWEI June 2016 report builds on the PPP section of the March 2015 Inception Report to include further findings and reflections from semi-structured interviews with Partner representatives. Semi-structured interviews were supplemented by a sample survey of local officials expected to facilitate project interventions (in the largest project where there are substantial delegated powers), and direct observations of Partner and wider stakeholder meetings (see Methods sub-section).

This Section of the Report also includes findings from a substantial FDW Mid-Term Review (MTR) that collected data from Phase 1 FDW projects in late 2015 (van Woersem et al 2016). Also the PPP-LAB of the Dutch Partnership Resource Centre is exploring the PPP aspect of the FDW programme on an on-going basis.

The criteria for creating partnerships in the FDW programme were revised for the second phase, but the FDWEI remit does not include exploring the effects of these changes on FDW second phase projects, though the critical findings reported here are relevant for Phase 2 and future Phases. .

As with the FDWEI Inception Report, this Report will attempt generally to anonymise more critical reflections (as was done de facto in the MTR report). The FDWEI team is very conscious that interpretation and analysis that could be read as negative in 2The FDWEI team’s thanks go to all the Partners’ representatives, other stakeholders’ representatives, survey supervisors and interviewers, and, people who gave survey information, all of whom gave us invaluable assistance – the total number of all these people is currently over two thousand. The font chosen for this report (Arial 12) was chosen because it is widely acknowledged as most suitable for visually impaired readers.

8

comparing particular Partners or the three countries would be counter-productive in causing emotions of humiliation/self-satisfaction to no good purpose in terms of understanding why some types of PPP relationships are not always efficient and/or effective.

This section of the FDWEI Report is divided into eight sub-sections:

1. Introduction (p.7)

2. Key issues in the current general debates on PPP governance drawing on Partnership Resource Centre (PRC) material and interview material (p.8)

3. Insights from the FDW Mid-Term Review Report (MTR) (p.11)

4. Methods used to collect data for the FWDEI institutional analysis (p.12)

5. Key findings on public sector Partners and implications (p.14)

6. Key findings on private sector Partners and implications (p.16)

7. Key findings on ‘third sector’ Partners and implications (p.21

8. Key findings on PPP relationships and interactions – towards a finer mesh typology (p.25)

2. Key issues in the current general debates on PPP governance

The general debate on PPPs was initially very optimistic. Given the shift from predominant confidence in the ‘benign developmental State’ from 1945 to 1975 with the end of the ‘long boom’ towards predominant confidence in ‘efficient market forces’ from 1975 to 2005 ending with the crash of 2007, perhaps it is inevitable that a belief there can be a ‘middle way’, bringing together the virtues of apparent opposites, emerged.

Critical debates on governance of water and wider developmental projects developed in the first decade of the 21st century (see Castro, 2004, 2007, 2008 and Zawahri et al, 2011). In 2008, in his Max Havelaar lecture3, Professor Rob van Tulder (2008) perceived potential limitations of PPPs as an aspect of internationalisation of corporations since 2000. This view sees enlightened corporate self-interest, including Corporate Social Responsibility (CSR) as eliding ‘development’ into the creation of market opportunities. CSR is a significant factor in the FDWEI selected projects and is discussed in some detail in Subsection 54 of this Section of the Report.

3 2008 Rotterdam School of Management Max Havelaar Lecture 2008 on ‘Partnerships for Development’ by Rob van Tulder, http://www.maxhavelaarlecture.org/downloads/Max%20Havelaar%20Lecture%20Tulder2008.pdf 44 CSR is also extensively discussed in a Special Issue of Socio-Economic Review (2012) Vol 10 No. 1

9

Van Tulder also pointed to the conceptual oppositions in the language surrounding PPPs at the time including:

More positive: stakeholders/networks/leverage; dialogue/learning dynamics

More negative: poor governance critique of public sector; market ‘failures’ critique of private sector; elite capture critique of civil society organisations.

The more positive concepts relate to a ‘partnership’ in action. The more negative points describe the behaviour that a Partnership is meant to remove. Unfortunately the journey from poor behaviour to virtuous action is ill defined, as recognised by van Tulder and literature produced by Netherlands Partnership Resource Centre (PRC) and PPP-LAB (see list of web-sites in References at the end of this section of this Report). The positive view of this challenge can be read in the following quote from the PRC 5 (which was established in 2009) current introductory leaflet:

‘Sustainable development requires an approach in which the interests of companies, governments and civil society are balanced in various combinations.’ (PRC introductory leaflet)

The task is then is find the combinations in which the interests are balanced. It was in this spirit that the FDW Programme was launched in 2012. Underpinning the optimistic model is a vision that two (Public and Private) or three (Public and Private and Third Sector) types of organisation can bring complementary motivations and resources to a single umbrella PPP organisation (see following Table).

Type of organisation

‘Ideal’ motivations Resources

Public Effective regulatory framework

Fungible* tax revenues; public borrowing capacity

Private Private profit/calculated responsible risk taking

Returns from previous investments; private credit lines/commercial borrowing capacity

Third sector Popular representation, especially vulnerable people

Civil society support; ‘weapons of the weak’**

*Tax revenues that can be used for any activity as prioritised by public sector decision-makers

** Actions available to affected people to avoid/challenge/sabotage/evade unpopular interventions

Continuing in a relatively optimistic vein, The PRC produced a booklet titled ‘How to make cross-sector partnerships work? Critical success factors for partnering’ 6. The 5 Using PRC publications is consistent with the FDWEI Terms of Reference statement ‘The evaluations will be conducted using the analytical framework for PPPs developed by the Partnership Resource Centre (PRC)’ (ToR p 5). Thinking in the PRC has moved on since the framework was developed and this Report will engage with current PRC thinking on key issues, notably in the final Conclusions to this Section of the whole FDWEI Report.

10

booklet is based on data from 294 Partner organisations from all over the world and is very readable and attractively presented. Five critical factors are identified:

1. Clarity of roles, responsibilities and ground rules;

2. Clear understanding of mutual benefits;

3. Clear vision of objectives;

4. Clear communication, shared planning and decision-making;

5. Good leadership.

Given the data comes from existing practitioners, it needs to allow for the positive biases that partnerships which failed are not represented (similar to the challenge of researching success and failure among small enterprises) and the respondents are likely to believe they possess all these qualities in these rather abstract terms.

There is a risk that such research arrives at ‘motherhood and apple pie’ conclusions that do not engage with practical realities. But the very idea there are ‘critical success factors’ suggests that there are possible ‘other side of the coin’ failure factors. Coming from this less optimistic direction, the challenge for this evaluation is both ‘picking winners’ and ‘avoiding losers’.

But before plunging into doom and gloom cynicism from sunny optimism, it must be said that all five of the FDWEI selected projects are functioning and moving forward with their planned activities, and only two original partners have withdrawn. As such, they all offer examples of at least survival, and the means by which the PPPs have survived various tensions and crises will form an important part of this Report. The longer term question of ultimate sustainability in Financial, Institutional, Environmental, Technological, and, Social terms will be addressed in the final FDWIE Report in 20187.

The PRC pamphlet rightly points out that Partnerships do not exist in a vacuum and ‘a stable macro-economic environment’ is an important exogenous factor in determining success. We would add a permissive (more than simply ‘stable’) political environment is also important. Some commentators might identify ‘permissive’ with ‘liberal’, but the latter term has been so used and abused that ‘permissive’ seems more appropriate. In the three FDWIE countries, economic instability and political authoritarianism/kleptocracy are not absent, but have kept within post-2012 ‘normal’

6 See: https://www.rsm.nl/fileadmin/Images_NEW/Faculty_Research/Partnership_Resource_Centre/brochure-How-to-make-cross-partnerships-work-2013.pdf). 7One FDW project was acutely aware of risks over the comparatively long life-time of the FDW project including elections (candidates using the project to their benefit) and the political changes that might take place due to the outcome of these elections pose a risk to the project. In addition, other external risks relate to severe and uncontrollable weather and environmental events as such as bushfires, droughts or floods. The last progress report mentions weather related risks as affecting the project. Internal risks affecting the project related more to 1) whether the local population would see the benefit of the scheme, and whether sensitisation would be effective 2) the capacity of project staff to take on a PPP project of this sort. This also included being able to understand the local culture and language

11

boundaries in the lifetime of the FDW programme and, other than the dramatic decline of the Euro on the international currency markets, no changes in economic and political factors exogenous to the Partnerships and their activities can be seen as having significant influence on Partnerships’ performances.

3. Insights from the FDW Mid-Term Review (MTR) report

The MTR report is a thorough and insightful reflection on the Phase 1 of the FDW programme which also takes on the basic selection process for Phase 2. The MTR Report covers all the countries and projects in FDW Phase 1 with most being visited by MTR team members. The MTR report concludes with a final positive statement:

‘FDW provides a rich resource of experience from which to draw lessons regarding [institutional?] innovation; how best to balance business interests with development objectives; how best to draw in Dutch expertise, experience and commercial interest ….’ (FDW Mid-Term Review, 2016, p. 24)

The FDWEI project very much perceives itself as such a ‘lesson provider’ adding on to the reflections of the MTR team.

A significant part of the MTR report is concerned with the Dutch para-statal RvO’s process of project selection. This is outside the remit of the FDWEI team.

But from the perspective of the FDWEI, the MTR report makes two very important critical points that lead to its strategic ‘recommendation’:

a. Conventional developmental objectives are not clearly defined and identified in the Phase 1 proposals, poverty reduction, greater gender equality, environmental conservation are generally not explicitly addressed and causally integrated into theories of change as impact variables.

b. The private sector as active profit seeking and calculated risk taking organisations are not well represented in the Phase 1 projects.

These points lead to the understandable conclusion and recommendation that the FDW programme as it stands is ‘caught between two stools’. Thus the FDW programme should consider moving into a more explicit (and thus more conventional) developmental agenda in which activities are valued by their likely contribution to poverty reduction, gender equalisation and environmental conservation demonstrated in a well specified (and quantified) theory of change. Or the programme should adopt a more radical risk-taking, market oriented approach in which fully private sector organisations set much more of the agenda with a vision of ‘game changing’.

Neither vision is hostile to a Partnership approach, but the ‘balance’ in the earlier PRC sense is shifted. In the first case, the public sector and third sector are more significant with the private sector providing ‘contracted’ services’. In the second model, the private sector sets priorities with a supportive public sector and the third sector providing a check on ‘exploitation’ (in one or more of its many forms) of the vulnerable.

12

The five FDWEI selected projects do have different objectives, balances of Partner membership, and models of leadership. The FDWEI evaluation project is therefore in a position to test the strategic MTR proposition ‘on the ground’ in terms of effectiveness in delivery.

However these tests will be towards the centre of a possible continuum of Partner ‘balances’ because, as the MTR Report points out the FDW Phase 1 projects were generally neither ‘development’ nor ‘profit’ led and thus tended to a muddled middle of Partnership possibilities. But, if CWD (Clean Drinking Water in FDW terms) FDW projects can be seen as broadly ‘developmental’ and EWU (Efficient Water Use in FDW terms) as rate of return earning (for Partners as well as target populations) then the five FDWEI selected projects include both types. Though we will discover that this division is by no means tidy!

4. Methods used to collect data for the FDWEI institutional analysis

The three country FDWEI teams were encouraged to make their own decisions on how to investigate PPP relationships in the light of situations on the ground (in terms of both FDW project activities scale, plus available access to Partners). This produced some healthy plurality in methods used:

a. In Colombia, the Partners tended to be from major organisations and Partner meetings were held at a very high level with technical committees below that level. Semi-structured interviews were conducted with Partner representatives. But, in addition, given the scale and locational spread of the FDW Project, a questionnaire survey was conducted with middle ranking officials, who would be crucial in transmitting implementation down through the Project hierarchy. The PPP/institutional analysis is based on a combination of document review and interviews. Important sources of information for the analysis are:

- IWM Project Plan - IWM Colombia’s “Result 1 report”, which summarizes IWM Colombia’s

activities of the first year (1 July 2013-30 June 2014). - IWM Colombia’s “Annual Progress Report Result 2 to 5 Year 2 report”, which

summarizes IWM Colombia’s activities of the second year (1 July 2014-30 June 2015).

- Semi-structured interviews conducted with core PPP partners and with departmental and municipal-level officials involved in the implementation of the programme and other relevant actors.

A first round of semi-structured interviews was held with representatives of most of the PPP partners, both in Colombia and the Netherlands, in 2014. Additional interviews were conducted in Colombia in 2015, in collaboration with researchers of CRECE. In interviews at the central level, a semi-structured interview schedule was used. In interviews at departmental and municipal level we used a combination of fully closed, semi-closed and open questions. IWM Colombia contracted the research institute CRECE for conducting a mandatory external evaluation. During the inception mission, it turned out that CRECE’s evaluation effort pursues very similar ambitions as our evaluation. In consultation

13

with the Client, it was decided to join forces. The cooperation brings on board the extensive knowledge of CRECE on the Colombian coffee sector and water management for coffee production. With the help of CRECE, a total of 48 persons were interviewed between September and December 2015. In a few instances, more than one person was interviewed at the same time. For this reason, the total number of interviews was 45. A partly-structured questionnaire was applied in 31 of the interviews. These were the interviews conducted with:

- 12 IWM extension workers; - 5 IWM regional coordinators (who each cover 5 micro-basins); - 4 leaders of the CDC (departmental committee of coffee growers) extension

workers; - 8 municipal administrators; - 2 representatives of Corporaciones Autónomas Regionales (CARs), which are

regional autonomous public entities in charge of implementing the policies of the Ministry of the Environment.

In addition, interviews were held with representatives of Cenicafé, the FNC central office in Bogotá and a couple of other entities. A second round of interviews among the PPP partners and among departmental and local-level stakeholders will be held at towards the end of the implementation of the IWM Colombia programme.

b. In Ethiopia, the sub-team was welcomed warmly and was able to establish positive on-going relationships with both FDW Projects’ managements. This facilitated repeated semi-structured interviews with Partner representatives in Ethiopia and the Netherlands The sub-team were also able to observe directly three meetings of Partners and one of a Stakeholder Platform. Finally, inspired by the Colombia initiative, the sub-team interviewed local government officials on their understanding of one of the two Partnerships and their roles in Partnership activities.

The Ethiopia FDWEI team found in all these relationships that, given the on-going activities of the FDWEI project, the team was seen as playing an advisory role contributing to the management and implementation of the two FDW Projects. This has not compromised the objectivity of the FDWEI’s team’s judgements, but it is a factor to be considered in designing such institutional evaluations using an ethnographically rooted methodology. There is an inevitable tension in institutional analysis between depth of critical emic understanding and a more comfortable etic detachment that treats documents and one-off interviews uncritically as objective facts. c. In Ghana, initial relationships with one of the Projects centred on the creation of a questionnaire survey which required very sensitive negotiation with the lead private sector, profit seeking Partner unused to external evaluations. This inhibited interaction with Partnership relationships. The second Project was led by a very internationally and locally experienced Partner who deeply embedded the FDW Project in their own pre-existing activities and relationships with other Partners. This more relaxed atmosphere meant that the institutional investigation was more

14

straightforward. In both cases, semi-structured interviewing was undertaken and insights gained from direct observations of Partner interactions.

5. Key semi-structured interview findings on public sector Partners and implications

There is a tendency to see the public sector in principle as uniform and homogeneous based on principles of bureaucratic structures, hierarchy of decision-making, and political accountability. Political accountability may be thought to differ in form between countries with variations in how far political accountability is dispersed to a wider population in reality.

The FDW projects show a wide variation in types of ‘public sector’ PPP partners. This variation can be categorised across a number of dimensions:

Scale: organisations directly employing from under one hundred to more than a thousand and serving thousands to millions of people

Length of hierarchy: organisations with less than five levels to those with more than ten levels between CEO and field employee

Political accountability: organisations with leaders directly elected on the basis of a mass franchise to those whose managers are appointed by, and only accountable to government and elected politicians

Market accountability: organisations directly selling services to customers to those providing goods and/or services at no financial cost to beneficiaries

History: organisations with a long duration culture of bureaucratic behaviour to recently created organisations to meet specific perceived current challenges.

These dimensions are not totally independent of each other in practice, but do have some autonomy. The experience of the FDWEI selected projects suggest that younger, smaller organisations with short hierarchies and more direct market accountability are likely to bring more ‘success factors’ to a Partnership than their older, larger, longer hierarchy, and market less accountable contemporaries. Political accountability is a more complex dimension. A high level of accountability to ‘politicians’ may lend legitimacy to an organisation, but depending on the political culture, this may mean less stability at the top of the organisation and having to de facto re-negotiate the terms of the Partnership with successive CEOs who feel no personal ownership of the Partnership.

It might also be argued that smaller, younger organisations with specific mandates (likely to be ‘parastatals’ in public policy conceptual terms) are less likely to have institutionalised kleptocratic tendencies (‘institutionalised corruption’ in the public policy discourse)8. But evidence backing such a claim is inevitably weak. But putting 8 One such ‘para-statal’s role as a public institution was perceived as crucial. As an example, with reference to

the imports of irrigation equipment and machinery, the para-statal facilitated the process by granting the

partnership custom exemptions under its development mandate. The lead partner was able to clear

equipment without paying taxes as the parar-statal is by law exempt from tax. It is also within the para-statal’s

mandate to facilitate the private sector it was able to apply laws that empower it to make certain decisions.

Institutionally, moving forward was made easier for the partnership and this has kept transaction cost lower.

15

that claim to one side, it would seem ‘small is beautiful’ when picking public sector Partners.

But, if the PPP activities overlap with the mandate of a very large public sector organisation which is not a formal Partner, for instance a national Ministry, then it is likely that at some moment (or moments) in the PPP life that the large public sector organisation will attempt to intervene in the PPP activities. The claim of national priorities will then be difficult to resist and the Partnership will face a crisis which will test its determination and stress relationships. So there is a dilemma on whether or not to include very large public sector organisations as formal Partners.

Another consideration when including very large public sector organisations in a Partnership is that, not only will they de facto treat the FDW project resources as if it was part of their own fungible funding ‘pot’ to be raided when convenient, but also the FDW funding will be too small to exert any leverage on deliberation and decision-making even if all the other Partners combined forces.

To use a metaphor, if Partnerships are meant to be like a herd of gazelles, moving swiftly and responsive to changes and risks in the surrounding environment then putting an elephant into the herd is likely to be disruptive. Even worse is putting two bull elephants into the middle of the herd which is likely to result in conflict, confusion and scattering of the gazelles. And to extend the metaphor to the planned target groups of people as the grass then as the adage has it, when two elephants fight, it is the grass that suffers.

Independent of its size, any public sector organisation with a designated area of operation is likely to claim it has at least a right of veto in its area of nationally given mandate9. This claim can be over all other partners (public, private and third sector), but can be more assertive when the partner is non-national and issues of foreign interference with its inevitable historical cultural tensions arise.

So what are the implications for choice of public sector partners in FDW projects:

Do not treat national public sector organisations as if they are all similar. The para-statal has taken the lead in facilitating relations with other government institutions. In addition,

SADA is able call certain issues to the attention of Ministries.

9 For instance, the pace of implementation is de facto being determined by the priorities of public sector

national Partners and their ‘normal’ pace of implementing. The public sector national Partners have ‘hidden’

motivations both in capturing resources and increasing their fungibility, and also frustrating rival national

public sector organisations inside and outside the Partnerships.

Rivalries and tensions between public sector water authorities sharing watersheds do not appear to have been

diminished by the FDW PPP Projects. In one Project, relevant authorities have not been included in the

Partnership (though a more inclusive stakeholder platform primarily of ‘technical experts’ has been created

that can bring all the authorities into interaction on a regular basis. In the other Project, the relevant

authorities are all Partners, but the level of hostility is such that the FDW Project may not have the leverage to

reduce that hostility – though this would be a great achievement if this is found to have happened in the final

impact observations.

16

Ask for details on national public sector organisations in terms of their size, mandate, and organisational history and check whether their scale and focus are commensurate with the other Partners (remember in this case small and young (and possibly local?) are more likely to be beautiful).

Be cautious if very large national public sector organisations are being put forward as a formal Partner. How is resource capture going to be avoided?

Be even more cautious if two large national public sector organisations are being put forward as formal partners and ask about the relationships between those organisations in terms of conflict/cooperation. There is a limit to how far one FDW project can reconcile deep, historic rivalries/conflicts.

Accept that the national public sector organisation Partner with the strongest mandate close to the FDW Project activities will become the de facto leader of the project whatever the formal lead agency arrangement10. It could be argued this is totally appropriate in terms of ‘sovereignty’ claims. In such circumstances, a Netherlands’ nominal lead agency can play a useful counselling/book-keeping role plus bringing Dutch technical inputs into the project activities at moments appropriate for absorption and sustaining by the mandated national public sector Partner.

6. Key semi-structured interview findings on private sector Partners and implications

The FDW MTR report points out the very broad definition of private sector organisations applied in FDW Phase 1. The definition seemed to centre on being cost and time conscious rather than necessarily profit-seeking. Thus the Netherlands’ water utilities were included in the private sector, and certainly would fit into those criteria. But there were also a number of profit-seeking organisations in the FDWEI selected PPPs. The range of PPP private sector organisations was wide, including very large transnational corporations plus small and medium sized (SMEs) manufacturing and consultant engineering enterprises. The vast majority of these private sector organisations were from the Netherlands. The MTR was concerned about the under-representation of SMEs and national organisations from the ‘recipient’ countries.

There are FDW PPPs in the five selected projects that can be seen as being strongly influenced by private sector rate of return priorities and thus do have efficiency motivations. A positive return on FDW private sector partners’ contributions/ investments, whether the private sector partner is a lead partner or not, can be found through a variety of channels11.

10 The project only amounted to only 1% of the public sector partner’s budget. The public sector partner’s see the FDW project as a small contribution to the overall public policy strategy in the sector with consequent little leverage. 11 Private sector partners have four channels through which investment is repaid and a rate of return can be earned. First, soft channels: connections to FDW project beneficiaries through markets in the longer term. Second, hard channels: direct sales of inputs, initially on credit, to beneficiaries. Thirdly, payments in kind from beneficiaries that are subsequently sold in local markets. Fourthly, producing outputs for sale on the private sector partner’s own account on an experimental basis for possible

17

But a significant issue, mentioned in the FDWEI Inception Report (2015), and worth further development here is the issue of Corporate Social Responsibility (which also appeared in van Tuld’s Max Havelaar lecture pointed out in subsection 1 above). The original role of a private sector enterprise in PPPs was to drive down unit costs and seek a private rate of return to its own investment commensurate with rates of profit in other activities undertaken by that enterprise. Such behaviour would increase the financial efficiency of the PPP activities and this would link to physical efficiency in the use of inputs. This motivation was envisaged overflowing into all the activities of the PPP, including public sector and third sector elements. Indeed in one form of PPP, all investment comes from the private sector and all returns go to the private sector with the public sector providing specifications and regulations. But this is not the case in FDW Phase 1 where project funding can come from all the sectors, with a large ‘matching’ subsidy from the Netherlands’ government. This complex pattern of funding can be seen as diluting the profit motive element in the FDW programme as the private sector cannot claim to be even the majority funder with an associated right to ensure the project makes an overall positive rate of return.

A Corporate Social Responsibility (CSR) element in the private sector funding contribution further dilutes the profit motive as a driver in FDW project decision-making. CSR as a motive within the private sector itself further takes the pressure off making a financial rate of return and pressure towards low unit costs and timeliness. This is not to argue that CSR in itself is undesirable, but to get a more objective, and less morally subjective, perspective, the history of CSR needs to be understood.

Twentieth century Corporate Social Responsibility has its origins in the USA in the 1920s with the growth of large manufacturing corporations with secure flows of profits, unthreatened by organised labour seeking higher wages or government seeking increased tax revenues. The results can be seen in the establishment of Foundations willing to fund a wide range of social ‘causes’, including charitable (for the ‘deserving poor’), cultural, educational, and health facilities. Such philanthropic Foundations undoubtedly increased the status of their eponymous founders which could be seen as a reward for showing social responsibility and thus they were unwilling to experiment in widening governance of the Foundations beyond a small circle of the elite. Also the Foundations tended to operate completely separately from the day to day operations of the corporations, where social responsibility had little or no role.

Economic depression in the 1930s diminished the resources available for Corporate Social Responsibility initiatives by big business. Then the expansion of public sector activities in the Second World War and post-war reconstruction reduced both the need and ideological space for private sector philanthropy. But, though resources

later replication. The ‘return’ on the FDW project investment can also be achieved by improving quality of local products sold through the private sector partner. The extent of improvement required to lever a satisfactory rate of return from the FDW project is reduced by the presence of resources from other partners and the Netherlands government.

18

and space for CSR in this form may have diminished, they had not disappeared and could be expanded when circumstances were right.

Environmental disasters and near-disasters in the 1970s and 1980s forced defensive actions by some corporations to show they were aware of a social responsibility not to damage the physical environment and human lives through incompetence and irresponsibility. Such defensive actions involved not only individual corporations, notably in the energy sector, but also a sector wide initiative in the chemical industry in its Responsible Care programme. But such defensive initiatives, even if genuinely well intended, had little reason to involve wider groups of people in governance of the use of resources thus made available. Social responsibility became more a matter of litigation and compensation rather than new activities requiring new forms of governance.

Outside the USA (and the UK), other models of social responsibility were developing. In the USSR and China, the State had a comprehensive approach in which decisions on both production and distribution of resources were highly concentrated in terms of governance. In Japan, and the newly industrialising ‘flying geese’, there was a division of responsibility between corporations providing ‘cradle to grave’ support for their regular employees and protective State provision for those without such support. In western Europe, corporations were expected to provide tax revenues for ‘welfare state’ provision for all as well as following regulations protecting the physical environment and their employees. In none of these models were issues of governance a serious issue. The authority of corporate managers in the private sector and elected politicians/top public officials in the public sector was unchallenged.

Only with the end of the Cold War around 1990 did global public attention turn towards the immense resources controlled by the largest corporations and the degree to which they were able to avoid (and evade) social responsibility, especially in the economically poorest areas where they were operating. State guarantees of social responsibility were weakening as the political will and resources to hold corporations to account were diminishing. But continuing pressure from various parts of the UN system, ‘fair trade’ and ‘green’ lobbies, and media conscious International Non-Governmental Organisations (INGOs) resulted in many individual corporations developing explicit Corporate Social Responsibility policies and establishing departments to deliver these policies. But the institutionalisation of such developments tended to be managerialist in its form of governance accountable to shareholders rather than wider stakeholders. The phrase ‘enlightened shareholder value’ comes into existence to link CSR with longer term profitability and the need for ‘enlightened management’ rather than increased popular participation.

The most public and global expression of the new CSR initiatives was the Global Compact which came into being alongside and complementing the UN Millennium Development Goals. This was an initiative of the United Nations and is an offer to the

19

larger scale private sector to commit itself to follow socially responsible codes of conduct on human rights, employment conditions, and the physical environment. It was ambitious in scope at the outset and did include statements on ‘good governance’.

The Global Compact does value partnerships between corporations and government and NGOs with the emphasis is on corporations reporting to, rather than deliberating with, partners. But the history of CSR with respect to governance from its foundations to the Global Compact is not one of active inclusion and effective participation of those who are affected by the activities of private corporations.

The implications for the FDW programme are paradoxical. A private sector PPP partner is ‘expected’ to behave in a hard-headed fashion pushing the whole partnership towards greater efficiency. But, evidence from FDWEI semi-structured interviews suggest insofar as the funding derives from a CSR motive, then the profit motive disappears and what was seen as ‘private’ comes to resemble international public sector ‘aid’ or a civil society funding agency. However, although CSR does not seek profits directly, it is not devoid of organisational ‘interest’. The private sector organisation may recognise a need to invest in local political goodwill if it is to sustain its local operations12.

The FDWEI observations suggest other considerations that need to be borne in mind when reflecting on the role of the private sector in PPP relationships:

a. Commercial and technological confidentiality and awareness of competitive threats may inhibit transparent sharing of PPP relevant financial and technical information.

b. A private sector partner feeling isolated may wish to bring more private sector stakeholders into wider stakeholder platforms/fora surrounding the PPP and thus influence the ‘ideological balance’ in deliberations in favour of market forces. This can be seen as appropriate in the FDW programme as a ‘rebalancing’ of the original PPP to include greater private sector influence.

12 In one FDW Project, lead Partner management appears to be controlling potential public sector rivalries by

avoiding all Partner meetings. In another Project, there have been all Partner meetings, but these did not

result in clear decisions on controversial issues, though they may have helped focus attention on issues that

were later resolved in bilateral/trilateral partner negotiations. Also, a private sector organisation extracting

water for its operations in a context of widely perceived or actual water shortage, may wish to use CSR funds

to conserve or increase water supplies for protesting local users as part of persuading local political leaders

that its actions are benign, even if the company has no productive need for that water itself (though these

actions may contribute to global corporation goals of achieving water use neutrality).

20

c. A private sector partner may wish to maximise its claims to PPP project socio-economic/environmental impact in its organisation specific CSR publicity and ‘over-attribute’ its actual share of ‘success’ and thus irritate other partners, especially ‘third sector’ partners.

d. Private sector partners may claim their resource contribution to the FDW are ‘in kind’ such as ‘reductions’ in daily fee rates for their own staff temporarily attached to a FDW project. The financial value of such contributions is open to dispute as the claimed ‘open market rate’ is not market tested in the specific context of the project. Such contributions offer leverage on the Netherlands’ government subsidy and so the whole PPP has an interest in putting a high value on such in kind contributions. Conceptually, it can be questioned whether such contributions represent a genuine private sector investment in the FDW programme.

e. Private sector partners may have had pre-FDW project strong contractual relationships and these may be continuing during the life of the FDW project and probably continue after the formal end of the FDW project. Where the relationship is between organisations of similar scale, this may produce cartel type behaviour and exclusion of well qualified other private sector organisations from participating in FDW project activities. Where the relationship is between organisations of very different scales, then the smaller private sector partners in the PPP may be inhibited from making deliberative contributions that are not in the interest of the larger private sector partners.

The implications of these findings and reflections for the future FDW programme are similar to those of the FDW MTR in pointing out the strategic option of a more private sector driven FDW programme:

Separate CSR elements in private sector contributions to FDW projects. If there are substantial CSR elements then do not count them as private sector investment. If the private sector is mainly contributing CSR funds, then treat the FDW project as a ‘developmental’ activity in the MTR sense of the term with clear and explicit developmental impact goals.

Look sceptically at claims that private sector contributions are ‘in kind’ and inquire how these contributions are valued and in what sense they are investments expecting a rate of return.

For the investment element in private sector contributions ascertain the expected rate of return and how it is to be generated.

Establish how the performance of the FDW project is to be attributed between partners (e.g. on the basis of overall financial contribution) and how the results of the FDW project are to be publicised (this is not specifically a private sector implication).

When new technology is required or desirable from private sector sources, then establish how questions of patents and other property rights are to be handled by the PPP.

21

Where formal private sector partners are a small in number compared with other types of partners, then consider how the private sector can be made more influential on the PPP by creating stakeholder platforms that include organisations such as Chambers of Commerce and/or Industry Associations.

Where multiple private sector partners are included in a PPP, then it may be useful to establish where there are past or present contractual relationships that may inhibit open deliberation in the PPP. This is not intended to be a veto on any partners having pre-existing relationships, rather it is to ensure transparency in the interest of openness between all partners.

6. Key semi-structured interview findings on Third Sector Partners and implications

There is an element of paradox in the term PPP as used in the FDW programme as that the programme is not just about partnering public sector and private sector organisations but includes a third ‘civil society’ (in PRC parlance) sector. What is the rationale for this inclusion?

The past discourse on how to design and operate safer drinking water and effective irrigation schemes has been dominated by the languages of ‘efficient management’ and ‘equitable public policy’. But recently the word ‘governance’ has become more prominent with the implication that the wider global debates about ‘democratisation’ have relevance to drinking water provision. To understand this shift, in this section of the paper we will explore three questions:

Why has the language of management and public policy been so dominant in the past?

What is driving the shift towards the language of governance? What can be learned from the wider debates on governance?

Providing safer drinking/irrigation water can be seen as challenge to be met by combined efforts of civil, mechanical, and water engineers and public health doctors/geologists. All these professions use the language of science and technology to understand and respond to challenges. This language can rightly claim great successes in describing how to modify physical environments to improve human health and productivity. But this language is mysterious and inaccessible for non-specialists which tend to restrict and inhibit debate.

This tendency to closure of debate can spill-over into the language and attitudes in day to day management, whether by public sector officials or profit-seeking private sector companies, or mixtures of the two in various forms of public-private partnerships. A top-down approach using the managerialist language of beneficiaries, clients, and customers justifies lack of consultation with people who could otherwise be described as participants and citizens.

Mainstream economists reinforce this closure of any need for collective influence on drinking water and irrigation schemes by emphasising the potential marketisation of

22

water as a commodity that can be allocated through open market pricing. There is no need for participation beyond the act of individual purchase. A non-economist may claim that drinking water is a public good because all people need to consume it. But mainstream economists deny this claim on the technical grounds that consumption is separable between individuals as it is for irrigation and therefore competitive pricing is possible. Profit seeking providers will sell drinking or irrigation water to all individuals with the ability to pay a price that reflects their individual desire to consume a unit of water providing that price more than pays their marginal cost of providing that unit. In addition, mainstream economists would point to the general availability of alternative water sources and therefore the presence of potential competition as a check on monopolistic behaviour.

The uneasy alliance of engineers and medical/environmental science professionals (frequently in the public sector), and private sector, market advocating economists suggests that safer drinking and irrigation water can be provided without the collective involvement of the people who use the final product. A mixture of public sector technical expertise and market forces can guarantee continuous delivery of appropriate quality water. A pro-poor concern might point to limitations on the ability to pay among some groups and argue for a public subsidy to assist such groups, but this does not require their collective involvement in the delivery system. While the system is operating smoothly, running it only requires ‘light touch’ public policy with minimal subsidies.

But what if a system faces a crisis, and most systems do at one or more points in their life-times, needing decisions that distribute water deficits and their associated costs and benefits unevenly among a population of potential users? We use the term ‘potential users’ here to include all people who live in the catchment area of the water system and not just actual users as some people may be excluded by pricing and/or formal residence or land-holding registration requirements. An initial crisis of ‘physical supply’ can then be transformed into a crisis of ‘political representation’.

One answer to such decision-making challenges could be to meet each such crisis with a specific crisis management intervention, which could or could not involve potential users collectively on an ad hoc basis. The case against such ad hoc responses and more on-going collective involvement of potential users can rest on five arguments:

If people are not involved in the design of a system then they will not feel ownership of that system and could be hostile rather than co-operative when faced with loss of service, whether provided by the private or public sector or a mixture of the two

If people are not involved in the design and operation of the system then they will lack the necessary information and knowledge to contribute to the resolution of crises

23

If people are only to be involved in crisis resolution then creating deliberative institutions for their involvement on an ad hoc basis takes time and in that time some people will be without safe drinking water and their health will be at risk

Resolution of a crisis may require resources from the potential users, e.g. labour and cash, and there could be an understandable reluctance to provide such resources if there has been no consultation

Drinking water systems do not operate in a political vacuum and the tendency towards greater user involvement in provision of public services is now well established and exposing ‘democratic deficits’ in services where ‘techno-bureaucratic’ power is unchallenged

If one, some, or all of these arguments is accepted, then the question remains - what institutional form should such ‘participation’ take and who should be involved? The literature on the political dimension of international development differentiates between levels of participation on the following lines:

Information provision: people are informed on a regular basis about system operation

Design consultation: there is involvement in the overall design of the system, usually focused on point of delivery of a service

Operational and maintenance engagement: participation in the construction and day to day operation of the system – often needing technical capacity building

Representative engagement in executive decision-making: representatives of the potential users sit on committees/boards that make strategic decisions – needs reflection on how representatives are selected with a risk of ‘elite capture’

Mass engagement in executive decision-making: strategic accountability to general meetings of all potential users – needs good quality deliberative processes and transparent decision-making rules, consistent with local better practice ‘community’ decision-making

These levels are often ranked evaluatively in terms of desirability, distinguishing between ‘token’ and ‘real’/’full’ participation. But such evaluative criteria need to be treated with some caution. Differences in technology and related issues of scale may make mass participation in all aspects of the scheme unrealistic in practice. Getting an acceptable and feasible mix of rights and obligations and institutional forms are also challenges when different ‘languages’ and deliberative cultures are being brought into interaction. It is in this broadly defined ‘equity’ role, that ‘third sector’ organisations enter tri-partite Partner organisations.

24

Phase 1 of the FDW PPP initiative settled for a very broad definition of the ‘third sector’ in terms of any formal organisations that were not unambiguously governmental or profit-seeking organisations. This permitted the inclusion of research institutes, para-statal, and national NGOs receiving substantial private sector and/or international funding to be included as ‘third sector’ organisations. The formal procedures for becoming a Partner in a FDW PPP tended to exclude ‘grass-root’, ‘bottom-up’, less formal, but more popularly representative organisations. This appears to have been partially addressed in the second FDW Phase, but it is beyond the terms of reference of the FDW Establishing Impact project to assess the effects of this change on popular accountability.

Implications of the semi-structured ‘third sector’ interviews findings with FDW Phase 1 Partners and the above reflections for the future FDW programme are:

Demanding formal organisational qualifications for ‘third sector’ partners works against popularly accountable grass-roots organisations and in favour of larger well established organisations with histories of close relationships with ‘outside’ funding agencies;

Including research institutes as ‘third sector’ organisations brings independence of government and private sector interests into the PPP deliberations, even if they add nothing in terms of popular accountability and associated equity concerns. They can play a significant role in providing reliable, independent evidence to the PPP and wider society. Some observations of Phase 1 PPPs suggest the PPPs can play a valuable on-going Research and Development role in relation to national public sector partners which have deficits in this area of activity. This function needs recognising and it would be unfortunate if research institutes were excluded from future FDW PPP projects, though they are not ‘third sector’ in the sense of bringing popular accountability and equity concerns into PPPs. Research Institutes could be an explicit separate category of Partner or included in an ‘other’ category which could also include private sector organisations participating solely on a Corporate Social Responsibility basis.

In order to bring more popularly accountable ‘third sector’ organisations (e.g. community development NGOs and smaller units of local government) into FDW PPP projects, it would be appropriate to build post-inception outreach and inclusion of such organisations as a condition in the Project proposals. Identifying such organisations in the proposals is likely to be unrealistic until local interventions are actually being planned in detail and local people can see the reality of what is being proposed13. The stake-holding role of local

13 In one project, ‘The process of the establishment of irrigation farmer groups or water user associations

(FBO’s) has started. A legal expert is formulating the regulatory process and statutes are being formulated. A

manual with operational procedures is being drafted that will guide a proper functioning of all operations,

including input credit and repayment system.’

25

people can then be recognised and their participation be given meaning through institutional representation in the FDW project decision-making structure for those project interventions affecting local people’s well-being14. There will inevitably be questions for FDW management over the accountability of local representation and whether it is de facto captured by a local patriarchal elite. Formal NGO partners in the PPP would then have a role in assessing and improving the quality of local people’s participation.

8. Key findings on PPP relationships and interactions

Creating a partnership as a formal governance arrangement says little about the quality of the relationships that are being created and the potential of those relationships to develop qualitatively into sustained relationships. The three previous sections on types of partners in the FDW PPPs discussed both the general limitations that prevent partnerships from achieving the optimistic outcomes envisaged in the early advocacy of PPPs and the more specific challenges met in the FDW PPP practices in the FDW Phase 1. This sub-section of the FDWEI report will explore the relationships between PPP partners as they have developed in the first two years of the five FDW Phase 1 sampled projects15.

Observations of the influence of the setting up of PPPs in the five projects showed:

a. The PPPs were set up on the basis of there being a ‘lead’ organisation in the partnership. While this is understandable in terms of firmly locating accountability for the use of substantial public funding, this requirement institutionalised inequality between the partners tending to reduce accountability to the PPP as a whole of the non-lead organisation partners.

b. Sub-sets of the partners had a history of positive, established relationships before the FDW PPPs were created16. This had positive implications in terms of lowering

14 An initiative of the programme, the formulation/strengthening of groups at the level of communities is seen as the way to foster sustainability of the programme. 15 This subsection of the Report looks at the interactions between PPP Partners and the patterns of

relationships found in semi-structured interviews and direct observations. It is not surprising that the FWDIE

findings contrast with the written reports sent to RvO which understandably report consensus and

cooperation or are silent on the quality dimension of relationships between partners.

14. The FDW PPTP Phase 1 programme may have underestimated the influence of pre-existing relationships

between Partners. In practice, there is likely to be a core group who have worked closely and amicably

(including profitably) together before the FDW programme and would in all probability have continued to work

closely and amicably together if the FDW programme did not exist. There is a risk of overestimating the value

16For instance, national private sector partners tend to be dependent on the national public sector partners

and/or the Dutch partners through past and present working relationships and prospects for future contracts

after the FDW projects. More generally, one to one relationships within and between most of the private

26

the entry costs in setting up the PPP. It had more negative implications in that, when these relationships did not include all partners, it produced barriers to inclusion of ‘new’ entrants that took time to lower.

c. The insistence that FDW PPPs should have clear statements of separate objectives and an associated well defined division of labour between partners (even if some of those activities were delegated to sub-sets of partners) discouraged open deliberation on the project as a whole involving all partners17.

sector and ‘third sector’ partners have a history of relating pre-dating the FDW Projects. One of the PPP

partners emphasized that it is a results-based programme. The definitive subsidy from RVO is conditional on

meeting agreed results.

14. The FDW PPTP Phase 1 programme may have underestimated the influence of pre-existing relationships

between Partners. In practice, there is likely to be a core group who have worked closely and amicably

(including profitably) together before the FDW programme and would in all probability have continued to work

closely and amicably together if the FDW programme did not exist. There is a risk of overestimating the value

added of the PPP in such cases. But there will be Partners who were not in the core group and a real test of the

PPP is how far inclusive, positive relationships are created by the core partners for those partners. There may

also be partners who were previously alienated from, or even hostile to, each other. Another real test of the

PPP will be if such partners can be induced by the core partners into more cooperative working relationships.

There are mixed signs of this in current FDWEI observations and this will be important for assessing final FDW

PPP impact.

In one project, partners are working closely together as ‘a team’. The partners joke and tease each other which

is culturally appropriate. Discussions take place with a high level of professionalism. Most of the partners have

worked together before and ‘grew into’ this PPP.

17Ideally, a high value-added PPTP arrangement would mean that all Partners feel some responsibility for all

the PPTP Project activities and able to constructively criticise where their technical knowledge/contextual

experiences/equity judgements indicate performance can be improved. As indicated in the Establishing Impact

Inception Report, this ideal can be frustrated when the Partners have concretised a division of labour prior to

implementation. Faced with such a division of labour, Partners may think that is inappropriate to cross

boundaries in this division of labour, seeing it as questioning expertise of other Partners or inviting ‘retaliatory’

unwelcome criticism of themselves.

Breaking through such barriers is an institutional socio-psychological challenge, the story of the Emperor’s new

clothes of unhealthy, uncritical consensus is relevant. As in this story, what may break such a consensus is

someone who comes from an institutional position that honest criticism is seen as unproblematic (in the story,

it is a child), and has the energy and determination to make a statement that may be unpopular. Of course,

beyond the story, it is also important that the point made is logically and empirically/technically defensible and

made in an acceptable style.

Whether such energy and determination exists in a PPTP is an aspect of establishing whether the PPTP

arrangement adds value beyond that achievable by the Partners working independently on specific activities. It

might be expected that A public sector with a custom and practice of working within ‘silos’ is the least likely of

the three sectors to provide such energy and determination, but individual public sector representatives and

very senior public officials may not feel so constrained by a bureaucratic model of interaction.

27

d. The complex resourcing arrangements of the FDW projects with different forms and amounts of resources being donated/invested by different partners18 created a hierarchy of material interest overlapping hierarchies of effective power, hierarchies of activities’ involvement, and the basic hierarchical FDW dichotomy between ‘lead’ organisation and other partners. In all ‘partnerships’, there are multiple ways in which some partners ‘are more equal’ than others.

e. One of the sampled projects conducted a comprehensive risk analysis. Sixty six risks were identified during Year 1, comprising very high, high, low and very low priority risks. In the Year 2 report the status of risks is classified into Active (high impact, high probability, but not yet occurred), Observation (medium/low impact and probability), Materialized (occurred in Year 2) and Mitigated (risks corrected through application of response plans). It is however not always clear whether the risks concerns the public or the private sector (or ‘third sector’), or whether it is a risk that is borne by the PPP as a whole. One partner stressed that the major risk concerns the timely disbursement of financial

These five ‘initiation’ characteristics hindered the development of partnership relationships in terms of inclusivity and deliberative openness19. They worked against development in the PPP as a round table and in favour of intersecting circles with the ‘lead’ organisation at the intersection of all the circles20. But each PPP has its own dynamics and is developing with various mixtures of differing, often unintended, directions in the implementation process.

18 One key challenge cited in one project was the difficulty faced in ‘getting some of the partners to fulfil their partnership obligations on time’, in this case referring to financial obligations. The funds from the government partner, for instance, have been held up and disbursements delayed. This implies, that the lead partner had to take steps to rectify this and to step in to cover the gap. The last progress report indicates that this issue has been resolved”. After two years the programme is still in a preparatory phase. Some delay was caused ba lack of funds (because the government had not released the financial contribution). The delays lasted one year because the key partners were seeking matched funding. 19 For instance, one partnership agreement indicates that the partners will set up a project committee,

containing one representative of each party. Each partner will have one vote, when making decisions and

decisions in the committee will be made with majority vote. The committee has been set up but the strategic

partners have not met formally yet (as at early 2016), meetings have been individual. One related challenge

mentioned was a seeming confusion or lack of clarity on the allocation of roles and responsibilities of parties

involved. Mention was made of the partnership agreement and the need to revise this document to introduce

a more regular schedule of meetings and to define more clearly the role of various committees.

20 Several projects identified a challenge in the geographical spread of the partners. It is not easy to get all

partners seated around a table and discussing. Occasionally this distance affects response times (to emails). In

addition, the lead partner is responsible for coordinating all inputs and currently all communications are

between partners and the lead partner. The other partners have minimal contact with each other.

In one project, two partners, though also key partners, still see themselves as ‘service providers’, they are

contracted to provide a service for a fee.

28

a. getting the FDW proposal implemented: all the PPPs are implementing the interventions as specified in their original proposals. Timing has slipped on a number of interventions, but there have been no changes in the nature of the interventions themselves, notwithstanding some changes in context that a more unified partnership could have recognised and consequently requested modifications. This style of implementation is associated with pre-existing relationships between partners in which there is a mutual respect for specialised competence.

In this role, there is a question about whether a PPP structure was needed and whether the same results could have been achieved on the basis of individual intervention specific contracts or sub-contracts with the lead organisation21.

This question was frequently posed in the 1970s about ‘integrated rural development’ programmes in which cross-sector interventions (in agriculture, processing, transport, health and education) were seen as having symbiotic interactions. In the 1980s, such integration was seen as inferior to marketising the sectors separately. PPPs seen as a revival of a more integrated multi-intervention approach then depend symbiosis between activities as positive relationships between partners. PPPs operating on a strict division of labour with little communication between partners can justify their existence on this basis.

b. reducing transaction costs22: the commitment to partnership relationships generates space for the exercise of goodwill between the partners which can reduce 21 FDW projects do not rule out sub-contracting and one project mentioned comparative success (compared with other similar projects in the same country) is due the fact that the project was able to identify and choose the correct people with whom to work. In this case, reference was being made to the correct subcontractors and consultants. These were chosen with an eye on past performance on past projects, and results achieved. At the start of work there was a reasonable level of confidence that they would deliver according to expectation and contract.

22 There was consensus in one project that the activities could not have been accomplished without the close

interaction in the PPP. Arguing that they did more with the PPP than could have done if the partners worked

separately one respondent stated “. . . Yes, scaling up. This is what the government wanted to do. They have

had too many ‘pilots’; they wanted a big project”.

There were, however, some negative aspects related to local absorption capacity of some of the players and

the difficulties of application:

“ . . . Sometimes there were also disadvantages. There were too many ‘professors’, too many ideas, too

many IT people. We did not know how to consume the ideas, we might not be able to keep up. We were

sometimes frustrated; the ideas could not be picked up. There were lots of discussions about open data

(for instance. Though several respondents mentioned that cooperation in the PPP led, in their

perception, to the streamlining of activities, and therefore a reduction in transaction costs, there were

also some negative aspects attributable to the interaction in the PPP.

“ . . . There were different interests around the table. Decision making could be long and winding. It was

frustrating if your idea was not picked up . . . . . Ideas are good, but there are multiple realities, this is

problem. Some things may be beautiful, but some things may not be realistic”.

29

transaction costs when interventions do not go precisely to plan. In this sense, PPPs act as moral, high quality deliberative ‘communities’. Such goodwill would be absent in cases of contracts with strict compliance and penalty clauses23.

23Lead partners have very different facilitation and mediation skills. Higher skills had positive impacts on

project performance at critical moments. In a more tightly knit PPP where the partners are working closely

together as a ‘team’, interaction is positive and productive. The partners joke with and tease each other (a

national cultural style). Partners put issues on the table and discuss solutions. Discussions take place with a

high level of professionalism. There is perception on the part of the partners that the team will find a solution

to problems. The successes that the project has realised have given the partners confidence. There is a

common perception that the partners are there for a common purpose, that the project is of importance and

of value, and that each partner has a clear and important role in the process. There is clear inter-dependence

in this PPP. There are several factors that have contributed to the positive nature of the PPP and the belief on

the part of the partners that the PPP has added value:

1. Most of the partners have worked together before and ‘grew into’ this PPP. The working relationship has

become natural and there is a substantial level of trust. That synergy existed already that really helped the

partnership. Because we sincerely believed that there was value added because of the partners.

2. In the project design there is a clear linkage and rationale for the involvement of the partners, e.g. in

completing a value chain.

3. The lead Partner has invested great deal of time (and money) in building relations. It has had an office in

the public sector Partner’s building for a number of years and the local presence. It shows sensitivity to

the local culture and ways of working

4. The lead Partner has put as a priority that this should be a government-driven PPP. So even though the

lead partner relates to RVO, and signs all contracts, the public sector Partner has been given and has taken

over the ownership of the project. A lead Partner staff member is seconded to the public sector Partner

5. Linking knowledge transfer to execution: building capacity of the staff of the public sector Partner to

monitor services (collection of data, cleaning of data, etc.) and providing support when needed in the

execution of activities. Slowly tasks are being transferred to the public sector Partner. This is in contrast to

previous years and projects the current FDW lead Partner actually acted more as consultants for CWSA.

6. There is a common belief amongst the partners, as a result of the PPP, that the success of interaction

has been dependent on a substantial investment in relationship building (links to the last point).

Leadership is important. Also you need to pay attention to the relationship. And nurturing the

relationship so that partners feel like they are valued; that they bring something to the table, that feeling

of belongingness. So this is not just a collection of people just for the sake of accessing funding from the

Dutch government. Really, we have things that we bring on board to improve the process. Linked to the

last point, success has come from the fact that the relationship between the partners has been given a

change to evolve. Deliberative processes are no longer positional, partners now put things on the table

and resolve them without the feeling that they need to defend themselves.

30

c. collecting evidence: a FDW project which is ‘heavy’ on technical expertise (national and/or Dutch private sector consulting firms and/or para-statal/university Research Institutes) with a broad range of planned interventions can use its continuing presence close to policy making in public sector national partners to provide a de factor Research and Development role where this is lacking or weak in the public sector partners. The lead organisation’s ability to mobilise a range of partners’ knowledge and skills to advise on a range of planned interventions at appropriate times can increase the evidence base for public sector decision-making as a spin-off from the original proposal24.

d. policy ‘nudging’: associated with collecting evidence is the possibility of ‘nudging’ public sector organisations to act autonomously on FDW project concerns25. The FDW PPP proposals contain developmental aspirations that the partnership can use to encourage service delivery public sector agencies to think ‘outside the box’ and take initiatives that were not explicit in the original FDW proposal, but contribute directly or indirectly to developmental goals26.

e. private sector and technology transfer: the FDW PPP financing structure substantially reduces risk and uncertainty for technologically innovative, private sector partners by spreading risks among partners and leveraging the Netherlands government through its subsidy. This can encourage private sector partners to try out new technology and institutionalising new market relationships, once they feel confident in the support of key partners (especially partners who are public sector regulators)27.

24 The public sector partner saw an opportunity to have data. Now the data is accessible. They can now speak confidently as a public servant. 25 Though not the lead Partner, a public sector Partner may be in the ‘driver’s seat’, and therefore positioned

centrally in the figure. A public sector Partner may have the overall responsibility for the in country, day to day

coordination. This can ‘include coordination, operations and progress reporting’. The public sector Partner may

be responsible for activating local networks and key stakeholders as well ensuring political ownership.

26 The lead partner put as a priority that this should be a government driven PPP. This contrasts with previous projects where the lead agency acted as a consultant. 27 One project hired an independent consultant to conduct an independent assessment of proposed new

technology. The assessment looked at the economic viability of the system. The report was used as a basis for

discussion and reduced the emotional aspects of the meeting. The meeting gave the partners a chance to give

full updates on what they were doing and to look for areas of convergence, ways to improve on linkages

(between activities) and to agree on the way forward. In one project, there is a perceived need for public

sector intervention to either intervene on pricing (in the shorter term) on behalf of a private sector Partner or

to ease entry into the market for other suppliers (in the longer term). Of note is that the only organisation that

FAM is willing to listen to is CWSA on the negotiations on price.

31

f. wider stakeholder inclusion28: the original creation of a FDW PPP had to have an element of opportunism in creating a ‘coalition of the willing’. This inevitably means that some ‘stakeholders’, that is groups of people/organisations that could have enhanced or can damage achieving the objectives of the PPP will be missing from the formal PPP. Arguably, missing stakeholders who have interests opposing the PPP interventions are both likely not to be in the PPP and to reduce the value added that the PPP can achieve29. But identifying missing stakeholders (with positive or negative powers to influence the FDW project) and exploring how they are included or not included in PPP decision-making and the effects of this inclusion/non-inclusion on project activities and outcomes are proving significant in understanding the value added of the PPP itself. For instance, the creation and practical operation of ‘Stakeholder Fora/Platforms’ which would not have existed without the PPPs is very relevant here, though relationships may be built through less formal interactions may be just as important. For one sampled Project, it is the intention that 5 additional PPP projects will come from the Partnership Agreement, which will be carried out with the same partners or with other partners and for which separate agreements will have to be signed. For specific projects they sign a Project Management Agreement with the same or other partners. The core FDW PPP does not include all relevant partners, but allows for an inclusive platform to involve other relevant institutions. The design of the programme was broadened to include also social aspects and the other current core PPP partners were involved in the discussions, but it was decided to limit the FDW PPP to a few core partners with an opening to other institutions to participate in the programme. The PPP partners formed a platform which other institutions can join, but will not form part of the PPP. It is envisaged that the platform will, apart from the six PPP partners, have 50 members towards the end of the programme implementation period (i.e. June 2018). These institutions will participate with contributions in money and kind (labour hours) for implementing the so-called Complementary Projects. The external institutions linked to the core PPP are expected to increase sustainability of the programme.

g. Delegation to local departments of Partners: In only one of the five sampled FDW projects, the geographical and financial scales gave rise to issues of formal delegation to local branches of Partner organisations and identification of local 28 There are invariably ministries and other government agencies with ties to and influence on the projects

which are external to the PPP. One project also benefitted from matching funding from the World Bank and

support from an international Foundation, though seeking this funding in the project inception period caused

delays in implementation.

29 One project plan states that an Advisory Board including non-partner stakeholders was to meet

every 3 months “to review progress, discuss the more strategic aspects, options and decisions” (pg.

24). Of note is the fact, that though the Advisory Board was constituted in 2014, and 9 members

were nominated (noted in the first progress report), it had, however, not been formally inaugurated

at the time of the interviews held in June 2015. To date (early 2016) the Board has not met. When it

does meet, the intention is to have meetings twice a year.

32

coordinators. The main task of the coordinators is to avoid a situation in which beneficiaries are visited by several extension workers from different Partners separately. In other FDW Projects, where public and private sector partners are delivering PPP services through their normal channels, then issues of delegation and coordination between Partners is not an issue.

REFERENCES

Cameron, J et al (2015) FDW Establishing Impact Inception Report, Netherlands Ministry of Foreign Affairs

Castro, J.E. (2004) ‘Poverty and citizenship: sociological perspectives on water services and public-private participation’, Geoforum, 38, pp 756-771

Castro, J.E. (2007) ‘Water governance in the twenty-first century’, Ambiente & Sociedade, Vol X no. 2, pp 97-118

Castro, J.E. (2008) ‘Water struggles, citizenship and governance in Latin America’, Development, 51, pp 72-76

van Tulder, Rob (2008) Max Havelaar lecture: http://www.maxhavelaarlecture.org/downloads/Max%20Havelaar%20Lecture%20Tulder2008.pdf

van Woersem, Bert et al (2016) Mid Term Review of the FDW Programme

Zawahri,N., J.Sowers and E. Weinthal (2011) ‘The politics of assessment: water and sanitation MDGs in the Middle East’, Development and Change, Vol. 42 No. 5, pp 1153-1178

Internet sites:

Relevant PRC/PPP-LAB internet sites:

https://handshake.pppknowledgelab.org/features/using-data-to-disprove-ppp-myths/

http://www.ppplab.org/editorial-by-ppplabs-jan-ubels-the-public-p/

https://pppknowledgelab.org/

http://www.ppplab.org/

https://pppknowledgelab.org/ppp-cycle

33

34

FDW ESTABLISHING IMPACT REPORT, JUNE 2016

SECTION THREE: COLOMBIA HOUSEHOLD SURVEY

Intelligent Water Management Colombia Baseline Report on Household Survey

Niek de Jong, Luciane Lenz, Jörg Peters, Ann-Kristin Reitmann, Maximiliane Sievert

Contacts:

Maximiliane Sievert, ecol, sievert@ecol-cologne.de

Niek de Jong, ERBS, EUR, ndejong@ese.eur.nl

35

1. Introduction 37

2. Water Consumption and Contamination in Coffee Production ................................... 38 3. The Intervention ........................................................................................................ 42

3.1. Work packages ...................................................................................................... 42 3.2. Activities ................................................................................................................. 44

4. Institutional Analysis .................................................................................................. 47 4.1 Methodological and organizational aspects ............................................................. 47 4.2 PPP partners and other key actors .......................................................................... 48

5. Evaluation of impacts at the farm level ...................................................................... 57 5.1 Evaluation objective ................................................................................................ 57 5.2 Identification Strategy ......................................................................................... 59 5.3 Sampling, Sample Size power calculations ........................................................ 60 5.4 Survey tools ....................................................................................................... 61 5.5 Survey Implementation ....................................................................................... 62

6. Baseline Results ........................................................................................................ 64 6.1 River Basin profiles ............................................................................................ 64 6.2 General farm characteristics ............................................................................... 67 6.3 Water sources .................................................................................................... 69 6.4 Coffee cultivation ................................................................................................ 71 6.5 Coffee processing .............................................................................................. 75 6.6 Domestic Water Usage ....................................................................................... 77 6.7 Waste and Waste Water Disposal ...................................................................... 78 6.8 Soil protection and forestry management ........................................................... 80 6.9 Meteorological stations ....................................................................................... 81

7. Evaluation risks and general data quality assessment ............................................... 81 7.1 Assessment of data consistency and completeness ........................................... 81 7.2 Evaluation risks assessment .............................................................................. 81

8. Conclusion ................................................................................................................ 82 References ....................................................................................................................... 85 Annex 1 – Coffee Processing ........................................................................................... 86 Annex 2 - Permitted pollution level for coffee waste water ................................................ 92 Annex 3 - Evaluation Questions and Indicators ................................................................ 94

Figure 1: Coffee processing steps ........................................................................................ 39 Figure 2: IWM Colombia Intervention areas ......................................................................... 44 Figure 3: PPP for IWM Colombia ......................................................................................... 50 Figure 4: Theory of Change – effects on farm level .............................................................. 58

36

Figure 5: Harvest season in survey river basins (by department) ......................................... 64 Figure 6: Presence of Nespresso AAA or Nescafé Plan ....................................................... 66 Figure 7: Presence of other development projects ............................................................... 67 Figure 8: Sustainability initiatives ......................................................................................... 72

Table 1: Water saving potential in each coffee processing step ........................................... 40 Table 2: Organic contamination reduction potential in each coffee processing step (in BOD5)41 Table 3: IWM Colombia Work-packages .............................................................................. 43 Table 4: Work package 2: Activities on coffee farmer level .................................................. 44 Table 5: Work package 2, Activity 2: training modules ......................................................... 45 Table 6: Work package 3: Montetary Incentive for reforestation ........................................... 46 Table 8: Methodological approaches for the different beneficiary groups ............................. 60 Table 9: Coffee farmers and farms in the treatment river basins, population and sample ..... 65 Table 10: Prevalence of water shortage and soil erosion, river basin level ........................... 66 Table 11: Household’s structure variables ............................................................................ 67 Table 12: Sector of main activity of the household members, in percent .............................. 68 Table 13: Housing conditions, in percent.............................................................................. 69 Table 14: Households owning different assets, in percent .................................................... 69 Table 15: Water sources, in percent ..................................................................................... 69 Table 16: Irrigation of coffee plantation, in percent ............................................................... 70 Table 17: Water shortage and water excess, in percent ....................................................... 71 Table 18: Coffee farming ...................................................................................................... 71 Table 19: Decision making with regard to coffee cultivation, in percent ................................ 74 Table 20: Coffee processing, in percent ............................................................................... 76 Table 21: Domestic water appliances and water reduction devices, in percent .................... 77 Table 22: Domestic Solid Waste and Wastewater, in percent .............................................. 78 Table 23: Waste and Wastewater from Coffee Production, in percent .................................. 79 Table 24: Soil protection practices and reforestation ............................................................ 80 Table 25: Meteorological stations and delivered information, in percent ............................... 81

37

Introduction

Colombia is the third most important coffee producer in the world after Brazil and Vietnam, contributing with around 13.5 million bags of coffee per year around 10 percent of the world production in 2015/16 (ICO 2016). For the last decades, coffee prices on the world market have been low due to increasing supply especially from Brazil, Vietnam, and Indonesia. Production in these countries is dominated by highly concentrated, capital-intensive farms, whereas coffee production in Colombia is characterized by numerous smallholders and family businesses with labour intensive cultivation. The Colombian market share has decreased over the last decades. In order to compete on the world market, Colombia has focussed since the beginning of the century on a premium customer segment by producing high-quality coffees that stand out for their environmentally and socially sound production. The coffee sector is especially important to Colombia, since it is a major source of income for the rural population. Coffee production accounts for 33 percent of rural employment, giving work to around 2.7 million people. In terms of GDP, the coffee sector only accounts for 0.5 percent, though (FNC-Cenicafe 2013).

In Colombia, considerable amounts of water are required to process coffee after harvesting. Depending on the processing practices, the waste water can have considerably elevated organic load, a high amount of suspended solids, and low pH levels. Environmentalists frequently suspect that these effluents contribute substantially to the contamination of surface water bodies, most importantly leading to eutrophication (see for example Adams et al. 1987, Beyene et al. 2011, Chanakya et al. 2004, Haddis and Devi 2008, Mburu et al. 1994, Zuluaga and Zambrano 1993). As of January 2016, new legislation came into force, where maximum pollution levels for dumping waste water into surface water bodies are defined. Traditional water using practices lead to water contamination that transgress the standards allowed for the coffee sector by roughly factor 10.

Against this background, the Intelligent Water Management (IWM) programme in Colombia intends to contribute to improved water management among coffee farmers by information and sensitization campaigns, training, hardware investments and an improved institutional environment. The intervention has formulated a very ambitious objective, which is “to establish basic environmental, social and productive conditions to reduce poverty and increase peaceful coexistence, sustainable development and self-reliance of the rural population in Colombia by means of implementing an Intelligent Water Management System” (Appendix I to application proposal, IWM Project Plan, page 2). The programme covers 25 intervention municipalities that are evenly distributed across five of Colombia’s nineteen Departments. An ambition of the programme is to become a benchmark for other Departments of Colombia as well as other countries in the world.

IWM Colombia is implemented by the Colombian Coffee Growers Federation (Federación Nacional de Cafeteros, or FNC), in partnership with the Colombian government, the private companies Nestlé S.A. and Nestlé Nespresso S.A, as well as the research institutions Cenicafé and Wageningen University. A Public-Private Partnership (PPP) was established for the implementation of the programme, with FNC as the lead partner.

With an overall budget of €24.5 million and an RVO grant of €9.5 million, IWM Colombia is by far the largest of the 2012 projects financed by the Sustainable Water Fund (FDW) – representing a quarter of the budget of all the 13 first FDW call projects taken together and

38

over a fifth of the corresponding RVO grants. It is the only intervention covering both the themes Efficient Water Use and River Basin Management and Safe Deltas. IWM targets 40 thousand of the 526 thousand coffee growers in Colombia. It is designed as a pilot to be replicated in other parts of the Colombian coffee zone. The intervention aspires to be even replicable in other sectors in Colombia and the rest of the world.

In this report, we present results from the baseline surveys of the in-depth evaluation of the IWM programme. This evaluation encompasses two components: a quantitative and large farm-level survey-based impact evaluation and a more qualitatively oriented Public-Private Partnership (PPP)/institutional analysis. For the quantitative part of the impact evaluation, we conducted a survey among almost 700 treatment farms located in the 25 IWM intervention river basins. In addition, we visited 700 farms in 25 control river basins. A second survey wave will be conducted at the end of 2017. The main identification strategy to evaluate the intervention’s impacts is a differences-in-differences approach. The purpose of this report is to present the collected data and to portray the socio-economic living conditions and characteristics of coffee production of the surveyed farms at baseline. Furthermore, the report assesses the quality of the data and examines the comparability of the selected treatment and control river basins.

For implementing the data collection on the ground, we teamed up with the Colombian research institute CRECE that has also been contracted by IWM Colombia for conducting a mandatory external evaluation. As part of that IWM contracted evaluation, a farmer survey among IWM participants had been foreseen. This is why it had been decided during the inception phase to pool resources and design one common farmer survey that could serve both IWM’s evaluation as well as the in-depth evaluation presented in this report. In fact, during the baseline mission, IWM, CRECE and our evaluation team worked together in the compilation of a common survey questionnaire. Unfortunately, few days before starting the survey fieldwork, IWM Colombia decided to withdraw their participation in the survey implementation. One of the reasons was the alleged threat of biased interviews. IWM Colombia had planned to employ the interventions’ extension workers for conducting the interviews with the coffee farmers, who might not have been impartial enough to assure unbiased answers.

The remainder of this report is organized as follows. Section 2 provides background information on water consumption and contamination in coffee production. Section 3 introduces the IWM intervention. Section 4 presents baseline findings of the institutional/PPP analysis. Section 5 presents the methodology for the quantitative study. The results of the quantitative farm-level survey are presented in Section 6. Section 7 concludes on general data quality and evaluation risks. Section 8 provides a short overall conclusion.

Water Consumption and Contamination in Coffee Production

After harvesting the ripe coffee cherries, the majority of coffee farmers in Colombia process their coffee at the farm to produce dry parchment coffee (cps) (see Figure 1). This process requires four steps. First, during reception and classification, over-ripe and dry cherries, stones and other particles are separated from good quality cherries. Second, pulping separates the pulp from the beans. This process leaves the beans with a thin slimy layer, the mucilage. To remove this mucilage in the third step, two techniques exist: natural

39

fermentation and mechanical removal. Once the mucilage is removed, the beans are dried in a fourth step and sold.

Figure 1: Coffee processing steps

Source: Own representation, photo of washed coffee from Rodriguez et al. 2015.

About half of the water is consumed during the first two steps of reception, classification, and pulping (including transportation of pulp and pulped coffee); the other half is used during mucilage removal and washing. However, the amount of water used during coffee processing can be reduced drastically when changing to water saving devices or applying certain water reducing techniques.

Table 1 lists each coffee processing step and the respective devices and techniques that can be adopted. Moreover, the table shows the cumulative water savings potentials on a percentage basis with the conventional technology as the base case reference. For instance, if the coffee farmer switches from the washing technology “winding water channel” (canal de correteo) to washing with a Tub Tank and applying four rinsing rounds, about 40 percent of water can be saved. Pictures of all devices can be found in the annex. Many of the devices are designed for bigger coffee farms that process more than 1,000 kg of coffee cherries a day. Yet, the production scale of the majority of Colombian coffee farmers including the IWM intervention farmers is much smaller than this. The only devices that are adapted to such smaller scales are the dry hopper, and the Tub Tank. Moreover, the classification step can also be skipped completely if only good quality coffee beans are harvested and farmers pay attention to not mixing the beans with leaves or stones. This is normally also the case at smaller farms where the farm owners and their family harvest the coffee beans themselves and no external workers are contracted.

40

Table 1: Water saving potential in each coffee processing step

Coffee processing step Device/Technique Water saving

potential compared to base case (in percent)

Maximum value of

each step

Reception of coffee and classification

Base case: Traditional Wet hopper 0

0.125

Water tank 0.008

Water tank and water recirculation system 0.075 Submersible pump 0.075 Hydraulic separator with hopper and screw conveyor a 0.124

Dry hopper 0.125

Pulping Base case: With water 0

0.125 Without water 0.125

Transportation of pulp Base case: With water 0

0.125 Without water 0.125

Transportation of pulped coffee

Base case: With water 0 0.125 Without water 0.125

Removal of mucilage & coffee washing

Natural fermentation

Base case: Winding water channel 0

0.490

Submersible pump 0.306 semi-submerged channel 0.319 Technique of four rinsing rounds (in Tub Tank) 0.375-0.400

Mechanic removal

Other mucilage remover 0.418-0.463 mucilage remover “DESLIM” a 0.479 Other coffee washing devices 0.433-0.445 “Ecomill” 0.490

Total water saving potential 0.990

Source: Adapted from Rodriguez et al. 2015. Notes: a The hydraulic separator with hopper and screw conveyor and the mechanic mucilage remover DESLIM together are called BECOLSUB.

Water contamination arises above all from the pulping and mucilage removal, producing effluent water with high organic load, a high amount of suspended solids, and low pH levels. The organic load is measured in BOD5 (Biological Oxygen Demand) that specifies the amount of oxygen consumed in the degradation of the organic material. By processing 12.5 kg of cps in the conventional way, 3.59 kg of BOD5 and 3.50 kg of suspended solids are generated (Rodriguez et al. 2015). Environmentalists frequently suspect that these effluents contribute substantially to the contamination of surface water bodies, most importantly leading to eutrophication (see for example Adams et al. 1987, Beyene et al. 2011, Chanakya et al. 2004, Haddis and Devi 2008, Mburu et al. 1994, Zuluaga and Zambrano 1993). Eutrophication reduces the amount of dissolved oxygen in the surface water with negative consequences for the aquatic life (see for example Chislock et al. 2013 for a general overview on consequences of eutrophication). It may even create an anaerobic atmosphere, which is an excellent condition for health threatening bacteria (Rattan et al. 2015). According to Zuluaga and Zambrano (1993), water contamination in terms of organic load from

41

processing one kg of cps is ten times higher than contamination through feces and urin generated by one person per day. Calculating with the average coffee production of the farms in our sample that amounts to 300 kg of cps per year, contamination by coffee farming equals the contamination of eight additional persons per farm. This is why the Government of Colombia has recently defined maximum pollution levels for dumping coffee waste water into surface water bodies (Resolución 631 del 2015, see Republica de Colombia 2015). When coffee is processed in the conventional way, contamination thresholds for BOD5 and suspended solids defined by the new legislation, are exceed by roughly factor ten.

Table 2: Organic contamination reduction potential in each coffee processing step (in BOD5)

Coffee processing step Device/Technique

Contamination reduction (in percentage )

Maximum value of each

step Pulp related contamination

Reception

Base case: Wet hopper 0

0.020

Water tank 0.001 Water tank and water recirculation system 0.012 Submersible pump 0.012 Hydraulic separator with hopper and screw conveyor a 0.020 Dry hopper 0.020

Pulping

Base case: With water 0 0.15 Without water 0.150

Transportation of pulp

Base case: With water 0 0.15 Without water 0.150

Storage of pulp

Base case: Without roof 0 0.15 With roof 0.150

Decomposition of pulp

Base case: Without roof 0 0.15 With roof 0.150

Collection and treatment of drainage

Base case: No 0 0.12 Yes 0.120

Mucilage related contamination

Washing

Untreated disposal of waste water 0

0.26

Physical treatment system (e.g. separation and filtration of solid material). 0.05

Reuse of treated water. Recirculation or reuse of leachate resulting from aggregation of washing waste water to the pulp, until no leachate is left.

0.06

Aggregation of mucilage to the pulp, without recirculation of leachate. Or aggregation of water of the first two rinsing rounds to the pulp, without recirculation of leachate.

0.10

Recirculation or reuse of leachate, coming from the pulp-mucilage-mix and pulp-washing water-mix, up to their depletion. Treatment of the third and fourth rinsing rounds wastewater in treatment system and reuse of treated water.

0.16

Biological or physical-chemical treatment systems. Or aggregation of the wastewater resulting from washing the coffee to the pulp without recirculation of leachates.

0.20

Utilization of the whole mucilage for animal food. Or incorporation of all the wastewater to the pulp and reuse without discharge.

0.26

42

Total contamination reduction potential 1.00

Source: Adapted from Rodriguez et al. 2015

74 percent of the total contaminating potential measured in BOD5 is associated with the pulp, 26 percent with the mucilage. The reduction or even elimination of water usage in the different processing steps is an effective way to bring down the contamination of water, since the contact of the pulp and mucilage with water is reduced and less pollutants are washed into the waste water.

Water contamination arises above all from the pulping and mucilage removal, producing effluent

water with high organic load, a high amount of suspended solids, and low pH levels. The organic load

is measured in BOD5 (Biological Oxygen Demand) that specifies the amount of oxygen consumed in

the degradation of the organic material. By processing 12.5 kg of cps in the conventional way, 3.59

kg of BOD5 and 3.50 kg of suspended solids are generated (Rodriguez et al. 2015). Environmentalists

frequently suspect that these effluents contribute substantially to the contamination of surface

water bodies, most importantly leading to eutrophication (see for example Adams et al. 1987,

Beyene et al. 2011, Chanakya et al. 2004, Haddis and Devi 2008, Mburu et al. 1994, Zuluaga and

Zambrano 1993). Eutrophication reduces the amount of dissolved oxygen in the surface water with

negative consequences for the aquatic life (see for example Chislock et al. 2013 for a general

overview on consequences of eutrophication). It may even create an anaerobic atmosphere, which is

an excellent condition for health threatening bacteria (Rattan et al. 2015). According to Zuluaga and

Zambrano (1993), water contamination in terms of organic load from processing one kg of cps is ten

times higher than contamination through feces and urin generated by one person per day.

Calculating with the average coffee production of the farms in our sample that amounts to 300 kg of

cps per year, contamination by coffee farming equals the contamination of eight additional persons

per farm. This is why the Government of Colombia has recently defined maximum pollution levels

for dumping coffee waste water into surface water bodies (Resolución 631 del 2015, see Republica

de Colombia 2015). When coffee is processed in the conventional way, contamination thresholds for

BOD5 and suspended solids defined by the new legislation, are exceed by roughly factor ten.

Table 2 presents devices and techniques to reduce water contamination in terms of BOD5 for each processing step. Again, the contamination reduction potential is provided in percent for each device, compared to usage of the base case device or technique.

The Intervention

3.1. Work packages The activities of IWM Colombia encompass four different work-packages or results that are displayed in the Table 3:

While work-package 1) focusses on establishing a conducive institutional framework, activities of work-package 2) targets the individual coffee farmer. Work package 3) includes both activities on the river basin level and activities on farm level. Activities of work-package 4 are concentrated on the river basin level and are supposed to inform decision makers on the institutional level. Budget-wise, work-package 2) is the most important part of the intervention. Accordingly, the focus of the quantitative analysis of impacts on the beneficiary level will be on the effects of work-packages 2) and 3) on coffee farmers. The institutional analysis will focus on effects and dynamics induced through work-packages 1) and 4), as well as the institutional (PPP-related) background of all work packages.

43

Work-package 1 is also called Result 2. The preceding Result 1 was implemented in year 1 of the programme and relates to the preparation and planning of the next phases. During year 1 also a baseline study was conducted among a sample of coffee farmers, which served as a needs assessment for the design of interventions at the level of coffee farms.

Result 2 includes among others the establishment of a water platform in which some 50 institutions will participate, apart from the PPP partners. By March 2016, 13 local, regional or national organizations had joined the platform.30

Table 3: IWM Colombia Work-packages

Title of work-package

Activities Intervention targets Budget

(M. EUR)

1) “Water is everybody’s business”

Partnership building and community organization

Institutional framework 2.3

2) “Water for Sustainable Coffee farming”

Training, assistance, water solutions and financial & revenue mechanism on coffee farmer level

Domestic and productive activities of coffee farmers

11.0

3) “Water Strategic Ecosystems”

Forest, soil, and watershed management

River basin management (on river basin level); activities of individual farmers (on farm level)

2.8

4) “Water Responsible Decision Making”

Water and Weather Monitoring and data analysis

River basin management; Knowledge within sector

4.0

The activities of work-packages 2 to 4 (or Results 3 to 5) will be implemented in 25 municipalities in 5 departments. The departments are Antioquia, Caldas, Cauca, Nariño, and Valle de Cauca (see Figure 2). The 25 intervention municipalities are evenly distributed across the 5 departments, so each department has 5 intervention municipalities. Within each of the 25 municipalities, one river basin has been selected for the intervention. A river basin is defined by an area around a main river and all its tributaries. The treatment population comprises all coffee farmers in a basin who use and discharge water from/to the main river or its tributaries.

In year 2, methodologies were introduced that are contributing to improve water management in the coffee sector. Examples of this are the application of a River Basin-based planning approach, participatory knowledge management and an ICT application developed for extension work at farm level.31 The Year 2 report gives an overview of the progress achieved under Results 2 to 5.

30 The PPP partners were the first members (Annual Progress Report Results 2 to 5 – Year 2, page 2). 31 See Annual Progress Report Results 2 to 5 – Year 2.

44

Figure 2: IWM Colombia Intervention areas

Source: Own illustration based on Cenicafé (1997) Encuesta nacional cafetera and IWM (2014) River Basin Management Plan Caldas

3.2. Activities The intervention’s four work packages comprise activities on the river basin level and on the level of the coffee farmers. Work packages 2 intervenes on the coffee-farmer level and contains three activities:

Table 4: Work package 2: Activities on coffee farmer level

Activity

# of farms

directly treated

% of coffee

farmers in river basin

directly treated

Implementation Status

1 Assessment, information, awareness 11,000 43% Implemented first half of 2015

2 Training through the IWM Learning Program 8,000 31%

started by end of 2015, first round will be

completed by end of 2016

3 Access to TA and financing for implementation of water solutions

1,650 6% First round started by beginning of 2016

As the first activity, IWM Colombia assessed the needs among the target population, informed about the intervention, and worked on a first sensitization of the population regarding water relevant topics. This activity targeted 11,000 farms corresponding to around

45

half of all coffee farms in the river basin. The activities were implemented through the existing FNC network on the ground. FNC is organized at the municipality level through a municipal committee and counts on a well-established network of so-called extension workers. These extension workers are in permanent contact with the municipality’s coffee farmers and provide technical assistance to the coffee farms through regular field visits. For IWM activities, extension workers received additional training and got support from further technicians who were trained exclusively for the IWM activities. As part of Activity 1, a needs assessment was conducted among a representative sample of 1,100 farmers in all 25 treatment river basins. For this purpose, IWM Colombia implemented a detailed farmer survey on water and coffee growing behaviour.

Table 5: Work package 2, Activity 2: training modules

Training module Who participates (% of total beneficiaries)?

Business management 10%

Good agricultural practices All Activity 2 farms (80%)

Generational renewal, gender, and law 10%

Integrated water management All Activity 2 farms (80%)

Coffee processing and waste water treatment 15%

Forest and soil management 10%

Associative practices 10%

Entrepreneurship 20%

The second activity comprises further training programs for around 80% of the farmers targeted through Activity 1 (and hence around 31 % of all coffee farmers in the targeted river basins). IWM Colombia has developed several training modules that will be applied based on the needs of the farmers. All modules are displayed in Table 5. It can be seen that while some trainings will reach all 8,000 Activity 2 farmers, other training modules will only be implemented among a small subsample of Activity 2 farmers. The modules with the largest number of participants are those on “Good agricultural practices” and “Integrated water management”.

The decision on who receives which training package is taken by the extension workers on the ground few weeks before starting the training activities and based on the extension workers’ assessment of the farms’ needs. The first training activities on business management and integrated water management started in late 2015. For the remaining trainings, manuals for trainers have been developed or are under development. It is foreseen to start implementation of all trainings within the course of 2016. Many of the trainings accompany the farmers continuously over the coming years of the IWM intervention.

The third activity is technical assistance and financing for water-efficient and environmentally-compatible equipment. In the first phase, financing for equipment adoption is given as a grant and the equipment will be distributed for free among 1,650 coffee

46

farmers.32 Farmers have to make only a relatively small in-kind contribution to the investment – either by providing work or by contributing local construction materials. Coffee farmers to be targeted by this activity are supposed to be the most polluting farmers (which is determined based on the extension worker’s assessment) and those located at key positions within the river basin. Farmers at key positions are those farmers who dispose of their waste water directly into the river. They have to be located within a 100m radius around the main river to be eligible for the intervention. 719 farmers have been selected to be treated in a first phase; the remaining farms will be selected in 2017 only.

The distributed equipment address both the domestic and the productive activities of the coffee farms. In the domestic area, flow restrictors for reducing water consumption, water filters to purify drinking water, and septic tanks and grease traps to reduce water contamination will be distributed. In the productive area, the IWM Colombia intervention distributes all water and contamination reducing equipment presented in Section 3.

The distributed equipment and promoted techniques are not genuinely designed for the IWM Colombia intervention. Rather, FNC and Cenicafé have already been working on the dissemination of the equipment and knowledge for several years. Accordingly, by now some farms use these water conservation devices and they are also promoted outside the treatment river basins through the normal FNC extension service. The IWM Colombia intervention can be seen as an intensification of these ongoing interventions.

As part of work package 3 “Water Strategic Ecosystems”, GIA intervenes at the farm level and supports farmers to plant trees and renovate the coffee plantations. Farms either receive only trees or they receive trees and additionally a monetary incentive to plant and conserve trees. The value of the incentive depends on the number and type of trees planted (see Table 6). The project selects 90 farms for the reforestation component with incentive. These farms sign individual contracts with IWM Colombia where the exact number of trees or area to be reforested is specified.

Table 6: Work package 3: Montetary Incentive for reforestation

Reforestation Model

Total Incentive in COP

In EUR

Conservation of natural forests - without fence 150,000 42.90

Conservation of natural forests - with fence 500,000

143.00

Mini corridors - with fence (only protection) 2,150,000

614.90

Living fences / boundaries 915,750

261.90 Forestry plantations with native trees (commercial possibility) -- reforestation in pasturelands 915,750

261.90

Guadua plantations - enrichment of water resources 360,000

102.96 Note: Exchange rate 01.09.2015: 1,000 COP = 0.286 EUR.

32 At a later stage, IWM Colombia intends to set up a revolving fund for intelligent water management technologies. According to our interviews with the IWM Colombia team, this will not happen within the time-frame of this evaluation, though.

47

On the river basin level and part of work package 4, complementary activities take place. In year one, River Basin Management and Organization Plans were formulated, which detail all activities to be implemented within the River Basin. Moreover, activities include trainings on gender, the establishment of farmer associations to manage water resources in the river basins, a learning platform, a communication plan, as well as the installation of water and weather monitoring stations, and ecological restoration and bioengineering plans for each river basin. As of April 2016, 21 weather monitoring stations have been installed.

Institutional Analysis

Methodological and organizational aspects The PPP/institutional analysis is based on a combination of document review and interviews. Important sources of information for the analysis are:

- IWM Project Plan - IWM Colombia’s “Result 1 report”, which summarizes IWM Colombia’s activities of

the first year (1 July 2013-30 June 2014). - IWM Colombia’s “Annual Progress Report Result 2 to 5 Year 2 report”, which

summarizes IWM Colombia’s activities of the second year (1 July 2014-30 June 2015).

- Semi-structured interviews conducted with core PPP partners and with departmental and municipal-level officials involved in the implementation of the programme and other relevant actors.

A first round of semi-structured interviews was held with representatives of most of the PPP partners, both in Colombia and the Netherlands, in 2014. The questionnaire for the interviews in included in Annex A. Additional interviews were conducted in Colombia in 2015, in collaboration with researchers of CRECE. In interviews at the central level, a semi-structured interview schedule was used. In interviews at departmental and municipal level we used a combination of fully closed, semi-closed and open questions (see Annex 4).

IWM Colombia contracted the research institute CRECE for conducting a mandatory external evaluation. During the inception mission, it turned out that CRECE’s evaluation effort pursues very similar ambitions as our evaluation. In consultation with the Client, it was decided to join forces. The cooperation brings on board the extensive knowledge of CRECE on the Colombian coffee sector and water management for coffee production.

With the help of CRECE, a total of 48 persons were interviewed between September and December 2015. In a few instances, more than one person was interviewed at the same time. For this reason, the total number of interviews was 45. A partly-structured questionnaire was applied in 31 of the interviews. These were the interviews conducted with:

- 12 IWM extension workers; - 5 IWM regional coordinators (who each cover 5 micro-basins); - 4 leaders of the CDC (departmental committee of coffee growers) extension workers; - 8 municipal administrators; - 2 representatives of Corporaciones Autónomas Regionales (CARs), which are

regional autonomous public entities in charge of implementing the policies of the Ministry of the Environment.

In addition, interviews were held with representatives of Cenicafé, the FNC central office in Bogotá and a couple of other entities.

48

A second round of interviews among the PPP partners and among departmental and local-level stakeholders will be held at towards the end of the implementation of the IWM Colombia programme.

PPP partners and other key actors

PPP partners

Until recently, the PPP comprised the following six partners:

Federación Nacional de Cafeteros de Colombia (FNC), the National Federation of Colombian coffee growers, which has over 300 thousand members and represents over half a million of coffee farmers in Colombia.

Nestlé S.A. Nestlé Nespresso S.A (henceforth referred to as Nespresso). The Ministry of Agriculture and Rural Development (MADR) Wageningen University and Research Centre (WUR), represented by Alterra Cenicafé – the National Centre for Coffee Research in Colombia

FNC is the initiator of IWM Colombia and is the lead partner of the PPP. 33,34 The history of the PPP (and the programme) dates back to 2010, when the director of FNC Europe and the CEO of FNC had the first discussions on the envisaged programme. FNC’s experience of collaborating with the Netherlands embassy in Colombia and good contacts the Colombian embassy in the Netherlands with the Dutch government facilitated the identification of partners. FNC had a desire to collaborate with WUR and was already collaborating with Nestlé and Nespresso. The latter had also no pre-existing relationship with WUR.

The above-mentioned six partners agreed to form a PPP and signed a Memorandum of Understanding to develop a proposal for the programme. All partners had previous experience of collaborating with one or more of the other PPP partners, but had never worked together in a PPP set-up. The FDW call for proposals in 2012 came at the right moment. The opportunity to apply for a grant from the Sustainable Water Fund allowed for increasing the scale of the intervention.

In June 2014, five of the six PPP partners signed the agreements. The partners in the PPP signed a Partnership Agreement in which they agree to operate as partners in the implementation of the programme.35 PPP for the IWM Programme is a cooperation between the public sector (Colombian government) and the private sector (Nestlé and Nespresso), as well as a ‘third sector’ – comprising an NGO (FNC) and knowledge institutes (Cenicafé and WUR). Formally, FNC is a large Colombian NGO founded in in 1927 and represents more 33 The Ministry of Foreign Affairs defines a PPP as follows: “A form of cooperation between government and business (in many cases also involving NGOs, trade unions and/or knowledge institutions) in which they agree to work together to reach a common goal or carry out a specific task, jointly assuming the risks and responsibility and sharing their resources and competences” (see IOB, Public-Private Partnerships in Developing Countries). 34 The PPP is led by FNC. The PPPLab’s FDW portfolio scan incorrectly states that the PPP is led by Nestlé, see http://www.ppplab.org/wordpress/wp-content/uploads/2015/06/PPPLab-Explorations01-FDW-portfolio-scan.pdf 35 In Appendix I of the Project Plan, this agreement is referred to as a Framework Memorandum of Agreement (MoA).

49

than half a million of coffee farmers. It is actually one of the largest NGOs in the world. Apart from a central office in Bogotá and offices abroad (in Tokyo, New York and Amsterdam), it has representations at both Departmental and local (municipal) level in the coffee-producing Departments of Colombia. At the same time, the Colombian government is somehow represented in the top of the organisation. FNC administers a Coffee Fund, from which various activities are financed. The government sometimes contributes to this Fund.36

The partners defined a common development goal.37 In this sense, the PPP meets the first of the five criteria of developmental PPPs defined by IOB (2013:17).38

Implementation of the programme has been delegated to FNC Colombia. The FNC Head Office in Bogotá provides the Project Director and Project Coordinators and collaborates with departmental and municipal level staff members, CARs, as well as researchers of Cenicafé and WUR and representatives of the other PPP partners.

The PPP delivers as a group. Each PPP partner has specific tasks. FNC (Head Office and Europe Office) clearly plays a central and coordinating role in the PPP (see Figure 3). For some things, one partner takes the lead. For others, another partner takes the lead. Nespresso and Nestlé operate jointly in the PPP. Although Nespresso is an autonomous company, its position is fully aligned with that of Nestlé. (“Nespresso va de la mano de Nestlé.”) As knowledge institutions, WUR and Cenicafé play a complementary role in the programme.

Together with FNC and Cenicafé, WUR coordinates activities related to multi-level training and the Learning Network.39

WUR held a session at Wageningen in the International Water Learning Network Meeting, in which it presented experiences of other projects from around the world.40

WUR has also contributed to the development of modules of the multi-level training programme, to be delivered by WUR, Cenicafé, FNC Central Office, Fundación Manuel Mejía and Departmental Committees.41

36 For every exported kilogram of dry parchment coffee, 6 cents (tax) are deposited in the Coffee Fund. FNC administers this Fund and FNC in the past paid a lot of activities for support to coffee farmers and communities (schools, health facilities, roads …). In years in which coffee exports are lower, there are not enough funds and then the government contributes to the Coffee Fund. Cenicafé is also paid from this coffee fund, but now, with (relatively) low production and low market price, the government is subsidizing the Coffee Fund. 37 The goal of the IWM programme is “to establish basic environmental, social and productive conditions to reduce poverty and increase peaceful coexistence, sustainable development and self-reliance of the rural population in Colombia by means of implementing an Intelligent Water Management System” (Appendix I to application proposal, IWM Project Plan, page 2). 38 Public-Private Partnerships in Developing countries. A systematic literature review. IOB Study no. 378. Ministry of Foreign Affairs of the Netherlands. April 2013. 39 IWM Annual Progress Report Year 2, R3 MoV 4.3 Learning Networks Plan (July 2015). 40 IWM Annual Progress Report Year 2, R2 MoV 2 Learning Networks Plan (July 2015). 41 See also the Multi-level Training Plan included in the Year 1 report.

50

Figure 3: PPP for IWM Colombia

The programme builds on the existing CSR activities in Colombia of the two private PPP partners – Nestlé and Nespresso. The two companies and FNC share a philosophy in the area of sustainability. For the coffee sector, Nestlé launched its Nescafé Plan in 2010.42 In Colombia, the Nescafé Plan comprises among others the distribution of coffee plantlets, technical assistance and fertilization. Since October 2010, some 24 million new coffee plantlets have been provided to around 8,000 coffee farmers. The Nespresso AAA Sustainability Quality Program was launched in 2003 in collaboration with the Rain Forest Alliance and aims “to protect the future of the highest quality coffees and secure the livelihoods of the farmers that grow them.”43 The triple A stands for quality, sustainability and productivity. In Colombia, the AAA Program’s activities concern among others “improving farm productivity, upgrading wet milling, developing new pricing strategies and improving

42http://www.nestle.com/media/pressreleases/allpressreleases/nestle-invests-chf500-million-in-coffee-projects-doubling-direct-purchases. 43 http://www.nestle-nespresso.com/ecolaboration/sustainability/coffee.

51

business management.”44 The activities in Colombia of both Nescafé Plan and Nespresso AAA are undertaken in collaboration with the FNC. The IWM Colombia programme covers both coffee farmers that are participating in either Nescafé Plan or the Nespresso AAA Program and farmers in micro-basins outside the Nespresso AAA/Nescafé Plan areas.

Both Nespresso AAA and IWM comprise training of coffee farmers. A difference between IWM Colombia and the AAA programme is that IWM does not focus on individual coffee growers, but on the micro-basin.

MADR was recently replaced by the Agencia Presidencial de Cooperación (APC) as the representative of the Colombian government in the PPP.

For the implementation of the programme, the PPP partners set up a governance structure comprising a Supervisory Board, a Steering Committee (SC) that meets in Europe, has representation at CEO level from PPP partners and is chaired by the director of FNC Europe, as well as an Operational Committee (OC) based in Colombia, with representation from all the partners.45 In addition, a Technical-Scientific Committee (TSC) with representation from Nestlé/Nespresso, FNC/Cenicafé and WUR was set up that is ‘a key “decision-making body within the IWM’s governance structure.’46

The technical-scientific committee meets every two months (face-to-face or by telephone). The operational committee also meets regularly. The Steering Committee of the IWM Programme should meets at least every quarter. In October 2014 it met in the Netherlands. In October 2015 there was a meeting of the SC that was held in the UK.

The PPP partners agreed on how the programmes resources will be shared and made a division of labour by defining which partner is primarily responsible for particular activities. They provide monetary resources and/or contributions in kind to jointly undertake activities that are expected to have value added for the specific interventions at the farm and river basin level. Some parties provide a financial contribution, while others provide only a contribution in kind (expert man hours). Nestlé and Nespresso jointly contribute 22% of the project’s budget, FNC and Cenicafé jointly contribute 15% (€3 million, of which €0.5 million in kind), WUR contributes another 5% (€1 million) in kind, while the Colombian government is to contribute 12% (€2.5 million). The contribution of the PPP partners is matched by a grant of €9.5 million from RVO that comprises 46% of the overall budget.47 The participation and contribution of WUR decided upon in the Steering Committee meeting of 14 July 2015 comprises the following:48

1. Added Value Activities, or direct work of WUR staff appointed to IWM, accounting for approximately 200,000 Euro per year;

44 http://www.nespresso.com/ecolaboration/nl/en/article/8/2276/empowering-small-scale-coffee-farmers-in-colombia.html. 45 See also Year 1 Report, Intelligent Water Management Program Governance Structure Regulation. 46 Year 1 Report, Intelligent Water Management IWM Technical & Scientific Committee Terms of Reference (ToR). 47 Appendix I of the Project Plan, p. 33. 48 IWM Annual Progress Report Year 2, Sub-Annex 7 IWM SCO Meeting Minutes (July 2015).

52

2. Complementary Activities, or in-kind contributions representing knowledge from related projects funded by other sources, also accounting for approximately 200,000 Euro per year.

Concerning the fifth criterion of developmental PPPs – the distribution of risks between the public and the private sector - 66 risks were identified during Year 1, comprising very high, high, low and very low priority risks. In the Year 2 report the status of risks is classified into Active (high impact, high probability, but not yet occurred), Observation (medium/low impact and probability), Materialized (occurred in Year 2) and Mitigated (risks corrected through application of response plans). It is however not always clear whether the risks concerns the public or the private sector (or ‘third sector’), or whether it is a risk that is borne by the PPP as a whole.

Water & Coffee Platform It is the intention that 5 additional PPP projects will come from the Partnership Agreement, which will be carried out with the same partners or with other partners and for which separate agreements will have to be signed.49 For specific projects they sign a Project Management Agreement with the same or other partners.50 Hence, the core PPP does not include all relevant partners, but allows for an inclusive platform to involve other relevant institutions. It should be mentioned that the initial discussions on the programme took place between Cenicafé and FNC Europe, focusing on technical issues. The design of the programme was broadened to include also social aspects and the other current core PPP partners were involved in the discussions, but it was decided to limit the PPP to a few core partners with an opening to other institutions to participate in the programme. The six PPP partners are the core institutions of the Water & Coffee Platform.51 Other institutions can join the platform, but will not form part of the PPP. It is envisaged that the platform will, apart from the six PPP partners, have 50 members towards the end of the programme implementation period (i.e. June 2018). These institutions will participate with contributions in money and kind (labour hours) for implementing the so-called Complementary Projects (not outlined in the IWM Programme documents). The external institutions linked to the core PPP and their contributions to the Water & Coffee Fund are also expected to increase sustainability of the programme. The Ministerio de Ambiente y Desarrollo Sostenible (the Ministry of the Environment and Sustainable Development) is one of them. At the sub-national level, the Ministry of the Environment is represented by CARs (Comités Autónomos Regionales, or Regional Autonomous Committees), which are responsible for the implementation of environmental legislation. Some Colombian universities also expressed interested in joining the platform. GIZ is also interested and can bring in additional resources and know how. While other institutions can join, the project management is very keen on maintaining the current scope

49 See e.g. PowerPoint presentation prepared by the project. See also the Work Pan on formulation of complementary projects included in the Year 1 report. 50 Called “Specific Project Agreement” (SPA) in the Project Plan. 51 A Water & Coffee Platform Plan was made. This plan is one of the 23 Work Plans included in the Year 1 Report.

53

of the project, in order not to lose the focus of the programme. By March 2016, 13 local, regional or national organizations had joined the platform.52 More communication about the platform is required, especially at the local level.53 FNC has made a national communication plan for IWM.54 Communication of the IWM programme relates to both internal communication (between participants in the implementation of the programme, in particular the GIA extension workers and coordinators, but also regarding strategy and reporting of the operational and technical-scientific committees) and external communication, to the founding members of the PPP, educational institutions, public administration at various levels, as well as other potentially external partners. Via its website, the programme wants to communicate its results to the wider public and emphasize the importance of the strategic allies – the PPP partners and other involved organizations.55 The programme’s website refers to the Water & Coffee Platform.

Actors at departmental and local level

In each of the five departments in which the IWM is being implemented, there is an IWM coordinator. The task of that person is, in the first place, to coordinate the activities of the IWM extension worker based in each of the five micro basins and the activities of the three ‘rotating’ IWM extension workers specialized in, respectively, coffee processing, reforestation and social issues. The latter each cover all the five IWM micro basins within the department. The IWM coordinator also facilitates the coordination of the activities of the IWM extension workers with those of the regular FNC extension workers and, where relevant, those of the Nespresso extension workers and other stakeholders, such as municipal officers. The aim is to avoid a situation in which coffee farmers are visited by several extension workers separately and, where possible, to have for example the IWM extension worker present if the Nespresso extension worker organizes a group meeting to which coffee farmers (and relatives) are invited.

An important component of the IWM programme is the training of coffee farmers, which is preceded by training of the trainers (i.e. IWM extension workers). Several interviewees are of the opinion that the training component of the IWM programme will be essential to achieve sustainable results. All but one of the interviewed IWM extension workers received training in the framework of the IWM programme. Training of extension workers is done at the Fundación Manuel Mejia and by means of e-learning courses. When the interviews with IWM extension workers were held, they had all been trained in business management, while some of them had also received training on topics such as (i) good agricultural practices in coffee production, (ii) integrated water management and (iii) gender equality relations, generational renewal and rights’ focus.

52 The PPP partners were the first members (Annual Progress Report Results 2 to 5 – Year 2, page 2). 53 Interviewed Municipal officers were not familiar with the Water & Coffee Platform, which is not that

surprising because it was not yet operational. 54 See also ‘Integrated communications plan’ included in the Year 1 Report. 55 http://www.manosalagua.com/.

54

Few interviewees reported that training of coffee farmers had already started. In cases where training had started, this most often concerned Business Management, but it is not clear whether this was always provided as part of the IWM programme or as part of already existing programmes, such as Nespresso AAA. In general, implementation was supposed to start at the end of 2015. By March 2016, the extension workers had provided training in Business Management and Integrated Water Management to a substantial part of the coffee growers.

All 12 interviewed IWM extension workers reported that awareness raising of the IWM programme had taken place. Awareness-raising has been/is being done in the targeted communities/micro-basins by visits of extension workers and via radio programmes, local newspapers and local ‘coffee newspapers’.

Interviewed departmental and local actors are generally aware of environmental problems affecting the coffee sector in Colombia. Almost all (i.e. 25 out of 26) interviewed extension workers, municipal administrators, CAR representatives and leaders of the CDC extension workers who were asked about the main environmental problems in, respectively, the micro-basin, municipality, region or coffee zones of the department in which they work consider that water pollution is the principal problem; similarly, 4 out of the 5 regional IWM coordinators reported water pollution as the principal problem in the 5 micro-basins for which they are each responsible. A high level of water consumption was also frequently mentioned as an important problem.56

Other problems – such as landslides and deforestation, for instance – are perceived as factors that potentially hinder the implementation of the IWM programme. Other factors mentioned in the interviews are lack of awareness (for example because coffee farmers are not charged for the use of water), the small size of plots the relatively low price of coffee, lack of interest or lack of a ‘micro basin vision,’ or the difficulty to change attitudes and practices of coffee growers (related to the fact that their average age is relatively high because of lack of generational renewal)

In comparison, examples of factors that can facilitate the implementation of the programme mentioned by interviewed departmental and local actors are a positive role of women in the community, training, awareness raising and the presence of the FNC.

Awareness raising, training and accompaniment are at the same time considered important for arriving at a more sustainable intervention. The IWM programme’s focus on a social component in addition to traditional extension (which focus more on the technical, coffee cultivation and -processing component) is also expected to foster sustainability.

Related to this, an initiative of the programme – the formation/strengthening of so-called “Manos al Agua” groups (‘Hands to Water’ groups) at the level of the communities is also seen as a way to foster sustainability of the programme. By March 2016, 30 “Manos al Agua” groups had been established and agreements of participation had been signed.57 Various interviewees reported that such groups had been formed or strengthened. In several

56 Interview results. 57 The PPP partners were the first members (Annual Progress Report Results 2 to 5 – Year 2, page 2).

55

occasions, this concerned strengthening of already existing community groups, such as that of a group of ‘women savers.’58

Not every key actor at the local level knows the IWM programme and participated in its design. Four out of the eight interviewed municipal administrators and the two interviewed CAR representatives were familiar with the IWM programme; however, no one had participated in the design of the IWM programme.59

None of the interviewed municipal administrators was aware that a PPP was established for the IWM programme; the same holds for one of the four interviewed leaders of the Comité Departamental de Cafeteros (CDC) extension workers.60

However, two of the leaders of the CDC extension workers who were aware of the existence of a PPP, had used that information to argue for a change in the selection of micro basins for the IWM programme.

In Caldas the initial selection of micro basins was made by the central level; in first instance, the intention was to select five micro basins in the Nespresso zone. However, the Comité de Caldas suggested a different selection and proposed to choose two micro basins in the eastern part of Caldas. Nespresso initially did not want to select micro basins in that part of the Department. The CDC used the argument that not only private partners contribute resources to IWM, but also the public sector and FNC and that for that reason other areas should be chosen as well. In the end the selection was modified.

In two other Departments – Antioquia and Valle de Cauca – the selection of municipalities and micro river basins to be covered by the programme was modified later on after some pressure from the Departmental Coffee Committee. In Antioquia, instead of choosing only micro basins where Nespresso is present, some micro basins were selected in parts where Nespresso is not active. The argument that was used is similar to the one mentioned for Caldas.

These changes can be seen as one of the (so far, few) instances of the PPP influencing the implementation of IWM.

58 Interview results. Interestingly, one of the interviewed extension workers explained that the ‘Grupos Manos al Agua’ originate from de ‘Business Management’ groups. 59 Interview results. 60 Interview results.

56

Evaluation of impacts at the farm level

Evaluation objective The objective of this evaluation is to examine the effectiveness of the IWM Colombia intervention by assessing the positive and negative, intended and unintended impact of IWM activities. The intervention has formulated a very ambitious objective, which is “to establish basic environmental, social and productive conditions to reduce poverty and increase peaceful coexistence, sustainable development and self-reliance of the rural population in Colombia by means of implementing an Intelligent Water Management System” (Appendix I to application proposal, IWM Project Plan, page 2). The intervention tackles not only water and soil related activities, but tries to enhance sustainability of the coffee production also in an economic and social sense. This is why IWM addresses aspects like business management, quality of coffee produced, and gender as well as generational renewal. In order to understand impact expectations and assess impact potentials, the evaluation team dedicated some effort during the baseline mission to discuss these topics with the interventions’ staff. Together with the interventions’ director and responsible IWM staff for each impact area, key impact areas of the intervention were identified. It was decided to focus the evaluation uniquely on water and soil related activities, because these are the intervention’s core activities where changes are likely to materialize and to be measurable within the time horizon of this evaluation. FNC and Cenicafé furthermore emphasized that “gender” and “strengthening farmer associations/partnerships” are cross-cutting subjects of all their activities that should be covered by the evaluation. According to the discussions with the project staff and in contrast to what had been specified during the inception phase, sizable effects on coffee quality and quantity are not likely. Therefore, we exclude this impact dimension from the evaluation.

The following Theory of Change (ToC) illustrates possible pathways (Figure 4) on how the intervention and its activities might lead to changes in outcomes and impacts. It is based on project documentation and intensive discussions with IWM staff in Colombia.

As described in Section 4, the intervention will implement activities on the level of the coffee farmer (A), on the river basin level (B), and on the institutional level (C).

(A) For activities on the farmer level, direct outcomes will be observable. Farmers will eventually i) adopt cleaner and more efficient water usage behaviour, ii) invest in clean and efficient equipment related to water usage and water contamination and iii) start to plant trees and redesign their farms in a way that increases their resilience to droughts and floods. This is what we refer to as “forest and soil management” in the ToC. The trainings and interactions between extension workers and farmers are moreover expected to increase women’s participation in decision making and bring farmers together in associations to discuss water related behaviour and avoid conflicts.

(B) On the river basin level, IWM Colombia will support the formulation and implementation of River Basin Management and Organization Plans that include the installation of water and weather monitoring stations, and ecological restoration and

57

bioengineering plans. The latter aim at establishing two pilot lots per river basin on which activities will be implemented to improve soil resilience against excessive rainfall and decrease the concentration of sediment in water sources. One objective of these activities on the river basin level is to generate knowledge and make it accessible. The second objective is to elaborate concrete management plans. On the farmer level, the installation of water and weather monitoring stations provides the farmers with information that can enhance planning of coffee cultivation. Some farms will furthermore be targeted by bioengineering activities. For the remaining activities, effects will be measured at the river basin level.

(C) For the time being, we do not see any transmission channels in which activities on the institutional level might influence farmers’ behaviour. Most obvious impacts will be on the institutional level and will be assessed by the PPP-part of this evaluation.

Figure 4: Theory of Change – effects on farm level

Resources (human, technical, financial)

PPP is established – Activities implemented

Farmer have received training and equipment

Cost Savings

Input

Output

Outcome

Impact

Less water usage and better waste water quality

Activities

forest and soil management implemented

Better Resilience to droughts and floods

water and climate monitoring stations are

installed

Outputs on institutional level

are produced

?

less water scarcity

Farmers adopt clean and efficient behaviour

(water-related)

Farmers invest in clean and efficient equipment

(water-related)

Women participate in

decision making

Farmers are associated

ecological restoration and bioengineering plans

are elaborated

Source: Own illustration

The interventions’ outcomes influence the coffee farmers in three areas: they affect domestic activities of coffee farmers and their families, their farming activities, and the general quality of the watershed and the forest where the farm is located. For farmers receiving the Activity 3 treatment (beneficiary “equipment”), intermediate impacts on the domestic and the farming level area are less water usage and better waste water quality. This enhances resilience of the farmers to water scarcity and might furthermore result in costs savings for directly treated farmers, since (i) expenditures for water will be reduced (if applicable – many farmers do not pay for water) or (ii) farmers avoid fines for water contamination (if these are really

58

implemented). Both effects can be expected to be rather subtle. Moreover, these impacts might affect water availability and quality of farmers located downstream. This effect however will be negligible, as farms participating in the intervention are normally located just next to the main river and they dispose of their waste water directly to the main river. The water of the main river is hardly used by coffee farmers for domestic or productive uses within the river basin. Water extraction downstream the river basins differs largely across the river basins and will not be assessed quantitatively in this evaluation. IWM will measure water quality of the main river before and after the intervention as part of their monitoring activities. Thus, it will be possible to evaluate whether the water quality of the main river is eventually affected.

Farms that participate in bioengineering and reforestation activities might increase their resilience to extreme weather events like droughts and floods. Since planting trees will not produce immediate results on water availability and erosion, this evaluation will focus on monitoring outcomes, i.e. the plantation of trees and the effect of providing monetary incentives to plant trees.

It is important to mention that the intervention produces only few tangible impacts that materialize on the individual’s farmer level in the shorter run. Therefore, we will measure effects above all on the output and outcome level and make an effort to understand whether and why farmers adopt water and soil conservation behaviour. Moreover, we will try to identify sustainability indicators in order to examine whether adoption will lead to a long term change in behaviour.

According to the ToC outlined above, a set of evaluation questions can be defined for each level of the ToC. In Annex 3 evaluation questions and corresponding indicators are presented. It is furthermore specified whether these indicators correspond to the Key Performance Indicators (KPIs) defined by the IWM Colombia intervention.

Identification Strategy The key challenge of designing a proper impact evaluation is the identification of the counterfactual situation, i.e. the question what would have happened to the treated units if the intervention had not been implemented. In the present case, the question is how coffee farmers would have developed if the IWM intervention had not been present. Since this situation is obviously not observable, is has to be approximated by means of a control group. The control group serves as a reference situation and its development over time can be compared to the development of the effectively treated coffee farmers in order to draw conclusions about effects of the IWM intervention. The more similar the control group to the treatment group, the more accurate the impact measurement.

Methodologically the best way to identify a proper control group would be to randomly assign the IWM treatment to river basins and subsequently to coffee farmers within the river basin. We have contemplated this possibility during the inception mission, but since the selection of river basins had already been completed, it turned out not to be feasible. Also a controlled random phasing-in of the intervention within the 25 selected river basins is not possible, because all river basins will be treated in parallel within the first year.

The second best option is to mimic a randomized treatment assignment in a non-randomized difference-in-differences approach, which will be pursued for this impact evaluation. For this

59

purpose, the river basins to be surveyed were selected in a way that treatment and control group river basins are similar in the before-situation – which would basically be the outcome of a successful randomization. Hence, we identified control river basins that resemble the IWM river basins as much as possible. For each of the 25 IWM river basins we chose a control river basin within the same municipality in order to make sure that treatment and control river basins are subject to the same socio-economic and weather conditions. In order to identify the control river basins, we asked the FNC to propose river basins that exhibit the same characteristics as IWM river basins. More specifically, river basins should be comparable with respect to

the existence of other on-going programs that potentially pursue similar impact goals as the IWM intervention (e.g. conservation of water),

the surface area the river basin covers, the total number of farms within the river basin, the number of farms within the 100-meter-radius around the river, the size of the river (i.e. volume, length and width). Incidences of water shortage, erosion, and other factors that potentially affect

water contamination (e.g. mines). While some of the information could be retrieved from a database on all coffee farmers in Colombia that is available at FNC (SICA), other more specific or more qualitative assessments were obtained from the extension workers through an excel sheet that extension workers filled out in written. Moreover, it is important to emphasize that one criterion for selecting IWM treatment river basins is the presence of either Nespresso AAA or Nescafé Plan. Accordingly, we also chose control river basins that host activities of one of these programmes. In order to avoid spill-over effects from treatment river basins to control basins (which would bias the impact estimates), the control river basins are not located downstream of treatment river basins. Using non-treated coffee farms located within treatment river basins as a control group is not an option, since i) spill-over effects can be expected and ii) they are systematically different from treated coffee farms (located in not so densely populated areas). The comparability of treatment and control river basins is assessed in Section 6.

Since the most pronounced effects can be expected to materialize among beneficiaries of Activity 3 (who receive efficient equipment) and farms that participate in bioengineering and reforestation activities of Work-Package 3, the quantitative farm survey focusses on these beneficiary groups. To the extent the surveyed Activity 3 farmers also participate in the Activity 2 trainings, this treatment will also be evaluated using the same treatment group. As the set of Activity 3 treatments is very diverse (some address domestic water usage, some the productive water usage, some the domestic water contamination, and others the productive water contamination) it will probably be necessary to analyse subgroups of farmers within the treatment and control river basins. We will apply matching procedures to identify the most comparable farmers among control river basins to establish the most suitable control group for each subgroup of treatment farms.

Unfortunately, we are not able to include Activity 2 training participants systematically in the survey, because they will be selected by the extension workers in an ad-hoc way. However, according to IWM, it is very likely that Activity 3 farms will also be the focus of training activities. Some simple questions on awareness and knowledge on intelligent water

60

management are also integrated in the quantitative survey which enables us to assess the effect of the training on those farmers that participate both in Activity 2 and Activity 3. For the remaining Activity 2 farmers, we rely on focus group discussions in the follow- up both in treatment and control river basins in order to test awareness on intelligent water management. This rather qualitative approach is also suitable, since most impacts of the trainings will anyhow be too subtle and diverse to be measureable by a survey-based tool. We will make an attempt to conduct these focus group discussions in all treatment river basins and additionally in approximately 25 control river basins. This will enable us to compare parts of the findings also quantitatively (for example by transferring qualitative answers into multinomial codes or by including very simple contingent valuation components into the focus group discussion).

Table 7: Methodological approaches for the different beneficiary groups

Beneficiary group Identification strategy Main survey tool

Work-Package 2-Activity 3 farmers (beneficiary equipment)

difference-in-differences structured questionnaire

Work-Package 3-Reforestation farmers (beneficiary reforestation)

difference-in-differences structured questionnaire

Work-Package 2-Activity 2 farmers that participate at the same time in activity 3

difference-in-differences structured questionnaire

All Work-Package 2-Activity 2 farmers (beneficiary training)

Cross-sectional analysis at follow-up, retrospective analysis

Focus group discussions

Interviews conducted as part of the Institutional Analysis (see Section 4) are used as a source for triangulation and contextualisation of results.

For the quantitative farmer survey, we conduct a survey wave before and after the intervention, i.e. in 2015 and 2017. Focus group discussions will be held in 2017 when trainings will have been implemented.

Sampling, Sample Size power calculations In treatment river basins, we applied a random sampling approach among the pre-selected beneficiary equipment and reforestation farmers. We received a list from IWM with all 817 farms to be treated in the first wave. All of these farms are located within a 100m radius of the main river. The remaining approximately 900 farms will be treated in the coming years and had not been selected so far. The sampling unit is the coffee farm. In some cases, coffee farmers own more than one farm and therefore several farms in our sample might have the same owner. This is especially the case in the department Nariño, where each

61

farmer has on average 1.7 farms. We will account for this fact during the final analysis and cluster standard errors at the farmer level.

For identifying comparable farms among control river basins, we received a complete list of farms within the river basin from FNC with details on the area in coffee cultivation. From this list, we excluded farms that exceeded or fell below the minimum and maximum farm size of treatment farms. Since it was not clear from the list which farm is located within the 100m radius along the main river, the ultimate selection of farms was done in the field. The sample size per river basin was set according to the number of farms interviewed in the treatment basin of the same municipality. We interviewed a total of 1,399 farms, 699 treatment farms and 700 control farms.

We designed the questionnaires in a way that the person who is most acquainted with the coffee production at the farm has to be present. The questionnaires were answered in most cases by the owner of the farm (62 percent), a near family member such as the spouse or the brother of the farm owner (29 percent), or by an administrator (9 percent).

The sample size of almost 1400 farms gives us statistical power to detect changes in water conservation behaviour of a magnitude around 6 to 12 percentage points depending on the indicator. We calculated minimum detectable effect sizes using the baseline data for three exemplary indicators: usage of tub tank and application of four rinsing rounds, pouring of production waste water directly into surface water, and participation of women in economic household decisions. We assume a power of 80%, and alpha 0.05. For these indictors, the study’s power seems sufficient, because effect sizes smaller than the minimum detectable ones can probably be considered as failures of the program. Given the large number of other impact indicators, though, we cannot be sure to have enough power to detect each and every true programme impact. This is why in the final impact analysis we will analyse non-significant results on sensitivity to the study’s statistical power in order to assess whether in fact the true programme impact exists but is only too small to be detected given the available sample size (so-called “false negative” findings).

Survey tools The main survey tool is a structured questionnaire that has been elaborated in cooperation with IWM staff. Since it had been originally foreseen to pool resources and make use of a monitoring survey conducted by the project itself, we worked together with IWM in the compilation of a questionnaire during the baseline mission in May 2015 and the following month. All modules have been discussed in detail with IWM staff. The questionnaire elicits basic socio-demographic characteristics of the farms and details on coffee cultivation (area in cultivation, type and age of coffee plants, production levels, plagues, participation in sustainability initiatives and organizational details). Water usage is elicited in detail for domestic and productive activities. Special attention is dedicated to ownership, usage, and maintenance of water saving equipment. The same applies to equipment to reduce waste water contamination. It turned out not to be feasible to test waste water quality, since most farms do not have a single waste water disposal but rather dispose of waste water at different places of the farm. Water quality will be inferred from the adopted and installed equipment and the effective day-to-day usage behaviour. Moreover, the questionnaire tests attitudes to and knowledge on water saving and water contamination.

62

In addition, the questionnaire also has a special module on forest and soil management and on whether famers receive and use data from water and weather monitoring stations. Moreover, the questionnaire covers gender topics, most importantly decision making in different areas. Finally, participation in trainings and associations is addressed.

On the river basin level, a semi-structured questionnaire elicits information among regional extension workers in order to scrutinize the comparability of treatment and control river basins. Moreover, the questionnaire was designed to obtain contextual information on water pollution factors and presence of other interventions in treatment and control river basins.

Survey Implementation Initially it had been planned to jointly implement the survey with IWM Colombia. IWM had planned to collect baseline information for monitoring and evaluation purposes among their treatment farms. This information had been foreseen to be used for an external evaluation for which they contracted CRECE, a Manizales-based research institute with many years of experience in the Colombian coffee sector. In order to maximize synergies, we decided to cooperate with CRECE for the implementation of the survey, too. IWM had intended to survey all of their beneficiaries and our evaluation team had planned to survey a representative sample of control river basin farms as well as a small subsample of treatment farms. During the baseline mission in May 2015 and until the end of August 2015, IWM, CRECE and our evaluation team worked together in the compilation of a common survey questionnaire. It was foreseen to merge the data sets and to mutually exploit the collected information.

Unfortunately, few days before we started our survey fieldwork, IWM Colombia had decided to withdraw their participation in the survey implementation. One of the reasons was the alleged threat of biased interviews. IWM Colombia had planned to use the interventions’ extension workers for conducting the interviews with the coffee farmers, who might not have been impartial enough to assure unbiased answers. Since the decision was taken only few days before sending our survey team to the field, in the first instance, we stick to the original logistics and only redistributed our sample to 375 treatment and 375 control farms. In parallel, we discussed with the Dutch Ministry of Foreign Affairs to increase our sample for reaching the originally foreseen sample size. We received the final approval only around three weeks after the beginning of field work, which meant some river basins had to be visited twice.

CRECE managed the logistical organization of our survey including the recruitment of the enumerators. All enumerators had worked with CRECE in other projects before, and hence, were experienced in doing coffee related fieldwork. All enumerators additionally attended a two-day training workshop. CRECE conducted three field tests in order to guarantee the appropriateness and the feasibility of the survey questions in the field. After the field tests, some improvements were made to the questionnaire. Moreover, CRECE was responsible for the quality assurance of the survey, i.e. that the households were sampled properly, the questionnaires were completed consistently, and the data entry was done accurately.

The fieldwork started the 29th of August 2015 and by end of September we received the Ministry’s formal agreement for increasing the sample size to 1,400 farms. The increase in sample size involved some logistical challenges because the survey team had to visit certain river basins a second time. In two control river basins there were not enough farms within

63

the 100-meter-radius around the river (our sampling criterion) to conduct the intended number of interviews. In these two basins we therefore extended the sampling area to a 200-meter-radius. The field work ended the 9th of December 2015. Most of the data entry took place during the time of the survey.

CRECE submitted the entered data to the evaluation team by mid-December 2015. The accuracy of the data was checked and necessary revisions were reported back to CRECE. After two rounds of revisions, the final version of the data was submitted to the evaluation team at the beginning of February 2016. Throughout the whole process, CRECE turned out to be a competent and reliable partner and they supported us with any inquiry we had.

Baseline Results

River Basin profiles The impact evaluation covers 50 river basins in all five intervention departments of Antioquia, Caldas, Cauca, Nariño, and Valle del Cauca. In each department, all five treatment river basins and additional five control river basins were surveyed. This section portraits the 50 river basins and presents the balancing between treatment and control basins.

The five departments, and hence also the surveyed river basins, differ with respect to climatic conditions. One indicator for this is the difference in harvest seasons (see Figure 5). While in Cauca and Nariño there is only one main harvest in the whole department (from March to June), certain regions of the other three departments have two harvest periods. In Valle del Cauca, all river basins are located in a region with two harvest periods: one from September to December, and the other from April to May. In Antioquia and Caldas, all regions have a main harvest from September to December. A second harvest from April to May exists in the three treatment river basins (and their three control counterparts) located in the South of Antioquia and the three river basins (and their three control counterparts) located in the North of Caldas.

Figure 5: Harvest season in survey river basins (by department)

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

Antioquia Second harvest (South)

Main harvest (all regions)

Caldas Second harvest (North)

Main harvest (all regions)

Cauca Main harvest

Nariño Main harvest

Valle del Cauca Second harvest Main harvest

Source: Own illustration based on a harvest season map provided by Cenicafé (Encuesta Nacional Cafetera – 1997)

According to IWM data on their beneficiaries (see Table 8), Nariño has the largest river basins with on average 2,100 coffee farmers and 3,115 farms per river basin. The smallest river basins are located in Valle del Cauca, with an average of 550 coffee farmers and 603

64

farms per river basin. The difference in the number of coffee farmers and farms indicates that in some cases one coffee farmer operates more than one farm.

The number of coffee farmers and farms treated in the course of IWM is around 440 coffee farmers per river basin. In all departments but Nariño, the number of treated farmers is the same as the number of treated farms. In Nariño, treated farmers often have more than one farm, which is why the number of treated farms is almost twice as high as the number of farmers. The share of coffee farmers treated within a river basin is highest in Valle del Cauca and Caldas with a percentage of 80 and 74, respectively. The lowest coverage is achieved in Nariño where on average only 21 percent of the total coffee farmers within the river basin are reached by the IWM intervention.

The IWM intervention activities will be launched gradually over the time frame of five years. For the implementation of equipment distribution and reforestation activities, 817 coffee farms were selected for at least one of the treatments in the first year. 63 of these farms receive both treatments. Within the selection of first round beneficiaries, farms in Caldas are overrepresented. It is not yet clear whether further treatment rounds will treat the other departments more intensively or whether Caldas will generally be overrepresented among beneficiary equipment and reforestation farmers. For our baseline survey, we succeeded in interviewing around 85 percent of all beneficiaries selected for first year treatment (in total 699). Furthermore, we interviewed 700 additional farms that are located in the control river basins.

Table 8: Coffee farmers and farms in the treatment river basins, population and sample

Antioquia Caldas Cauca Nariño Valle del Cauca

Population Population of coffee farmers per river basin 785 603 1,116 2,101 550

Population of farms per river basin 1,003 740 1,439 3,115 603 Total number of IWM treatment coffee farmers per river basin 439 443 439 441 438

Total number of IWM treatment farms per river basin

439 444 440 762 439

Share of coffee farmers treated by IWM (in %)

56 74 39 21 80

Share of coffee farms treated by IWM (in %) 44 60 31 24 73

Total number of coffee farmers selected by IWM for first year treatment a

111 325 118 152 111

Beneficiary equipment 91 301 103 136 88 Reforestation Activity 49 44 20 19 29

Sample Total number of coffee farms interviewed 186 592 188 259 174

Number of treatment coffee farms interviewed 93 296 94 129 87

Number of control coffee farms interviewed 93 296 94 130 87

Source: Data provided by IWM (only information on the IWM implementation river basins). Note: a Some farms participate receive both equipment and reforestation activities.

65

For the selection of appropriate control river basins, we solicited information on the prevalence of soil erosion and water shortage problems from the extension workers of the FNC (Table 9). According to FNC extension workers, soil erosion is more problematic in the river basins than water shortage. All but five river basins report problems with soil erosion while only 32 out of 50 river basins report problems with water shortage. No differences between treatment and control river basins can be observed. Note that this is based on the extension workers’ qualitative assessments of the overall situation in the river basins. The coffee farmer data analyzed in the next section sheds more detail on evidence of soil erosion and water shortage at the farm level.

Table 9: Prevalence of water shortage and soil erosion, river basin level

Control (N=25)

Treatment (N=25)

Number of river basins with Water shortage 16 16

Soil Erosion 23 22

Source: Assessment of regional extension workers

We furthermore asked the extension workers to identify particularities of the (location of the) river basin that might affect water contamination in the river basins. Most frequently mentioned contamination factors additional to those tackled by IWM are animal husbandry and water contamination from urban or larger rural settlements. In some river basins, water is also contaminated by upstream mining activities or other productive activities such as a recycling stations and large scale commercial tree plantations. In some regions of Valle del Cauca the guerilla movement FARC is still active. Here, farmers had been restricted in accessing their land and partly forced to abandon it completely.

As can be seen in Figure 6, in almost all river basins either the Nespresso AAA Program or Nescafé Plan is active. The latter is active in Valle del Cauca, while the former is present in Antioquia, Caldas, Cauca and Nariño. In Antioquia and Caldas, the Nespresso AAA program is not present in six and four river basins, respectively (equally distributed over treatment and control river basins).

Figure 6: Presence of Nespresso AAA or Nescafé Plan

66

Source: Assessment of regional extension workers

In 38 out of our 50 river basins, other development projects are active, which (in parts) also pursue water conservation and reforestation activities (see Figure 7). These are sustainability initiatives like the Fairtrade initiative, 4C Association, Rainforest Alliance and UTZ Certified, or activities of the coffee farmer cooperatives and the public regional autonomous corporations (CAR). The governmental activities are partly financed by royalties from natural resource extraction. In Cauca and Caldas, different reforestation programs exist. One of them is run by the German KfW Development Bank.

Figure 7: Presence of other development projects

Source: Assessment of regional extension workers

General farm characteristics To illustrate the living conditions in the IWM project areas and the control river basins, this section presents descriptive statistics on the socio-economic structure of the surveyed farms. The results of this and the following subsections will also provide information on the quality of the collected data and whether the control group represents a proper counterfactual to the treatment farms. A summarizing assessment of the quality of the collected data follows in Section 0.

Table 10: Household’s structure variables

Control Treatment p-value N

Household size 3.53 3.60 0.477 1,399 Share children 0-5 years, in % 4 5 0.620 1,399 Female household heads, in % 11 14 0.088 * 1,399 Age of the household head 53.74 54.03 0.690 1,399 Household head received formal education, in %

91 92 0.638 1,399

Household head received secondary education, in %

15 15 0.873 1,399

67

Notes: Asterisks represent statistical significance of the difference between control and treatment communities. * indicates that the difference is statistically significant at the 10, ** at the 5, *** at the 1 percent level. Source: FDW Colombia baseline survey 2015

For the purpose of the survey and the analysis, we adopt a common working definition of the household as a group of individuals who live regularly at the farm. Under the authority of a single person, who is called the ‘head of the household’, these individuals pool their resources together to meet their basic food needs. In our sample, we have 1,399 households with a total of 4,986 members. The average household size is 3.6 (Table 10), with five percent of them being children younger than six years. Household size and the share of children younger than six are very similar for treatment and control farms, which indicates that the two groups are comparable along that demographic dimension. The same holds for most of the characteristics of the household head, with the exception of the share of female household heads. The latter variable exhibits a statistically significant difference between the two groups. Considering that the difference is very small in size, though, it does not give reasons for further concerns in terms of comparability of the treatment and control group. Nonetheless, we will include the socio-economic variables as control variables using multivariate techniques in order to account for such differences in the final analysis.

Among both the treatment and control farms, the majority of household heads work at the coffee farm and has on average 37 to 39 years of experience in coffee farming (Table 11). Considering the average age of the household head of 54 years, it becomes clear that most farmers have been working for their whole life in coffee farming. Although 21 percent in the control and 17 percent in the treatment group cultivate other crops than coffee, the cultivation area for other crops is relatively small compared to the area cultivated with coffee (only 0.3 ha vs. 2.5 ha for coffee cultivation). At only around 24 percent of the farms a household member works outside the farm. This illustrates that the dominating income source for the majority of farms is coffee production. Some of the variables exhibit differences that are borderline statistically significant, but again only small in size.

Table 11: Sector of main activity of the household members, in percent

Control Treatment p-value N Household head...

… works on coffee farm 81 80 0.674 1,399 … works outside of farm 8 6 0.137 1,399

Years of experience in coffee farming (not in %)

37.73 39.10 0.122 1,395

Farm cultivates other crops than coffee 21 17 0.121 1,399 At least one HH member works outside of farm 23 24 0.741 1,399

Notes: Asterisks represent statistical significance of the difference between control and treatment communities. * indicates that the difference is statistically significant at the 10, ** at the 5, *** at the 1 percent level.

Source: FDW Colombia baseline survey 2015

In the treatment area, only 77 percent (Table 12) of the households have their residence on the farm and the remaining 23 percent reside somewhere close to the farm on another property, with other family members in the same micro basin or even further away in urban areas. In the control area, a higher share of farms has a dwelling house on the farm and the

68

difference is statistically significant. This difference is very important for analyzing domestic water usage, since farmers who do not live on the farm do not have much domestic water consumption. In the final analysis we will restrict the sample for this analysis to farmers that live on their farm and we will maybe use matching approaches to identify the most comparable control farms.

With respect to housing conditions, no substantial differences exist between the treatment and control households can be observed. The main flooring material is cement or gravel. Around one third of the farms has even floors made of better materials. The main wall material is plastered bricks or blocks. The average number of rooms is three. Treatment farms have slightly more rooms, but the size of the difference is rather negligible. Nearly all residences have a kitchen, whereof around 70 percent are equipped with a chimney.

Table 12: Housing conditions, in percent

Control Treatment p-value N Farm has dwelling 83 77 0.005 *** 1,399 Floor material a 0.950 1,399

Cement or gravel 62 61 0.645 1,399 Higher material than cement 34 35 0.807 1,399

Wall material a 0.293 1,399 Plastered brick or blocks 34 35 0.895 1,399

Lower material than plastered bricks or blocks 38 38 0.939 1,399 Number of rooms (not in %) 3.25 3.38 0.063 * 1,399 Residence has a kitchen 100 99 0.704 1,399 Kitchen has chimney 68 67 0.719 1,399

Notes: Asterisks represent statistical significance of the difference between control and treatment communities. * indicates that the difference is statistically significant at the 10, ** at the 5, *** at the 1 percent level. a The options of floor material and wall material are ranked according to the Colombian National Administrative Department of Statistics (DANE). Source: FDW Colombia baseline survey 2015 With regard to ownership of different assets treatment and control households are also very well balanced (Table 13). Most of them own a TV (more than 90 percent), a refrigerator (more than 80 percent) and a radio (88 percent). Virtually everybody has a mobile phone. About 40 percent of the households own a motorcycle, while bicycles and cars are rare. The difference in car ownership is statistically different between the two groups. This difference, however, is again small in absolute numbers and does not point at a systematic difference between the control and treatment households. Table 13: Households owning different assets, in percent

Control Treatment p-value N Refrigerator 84 83 0.423 1,399 TV 94 92 0.165 1,399 Radio 88 88 0.877 1,399 Landline phone 0.43 0.14 0.318 1,399 Mobile phone 98 99 0.125 1,399 Motorcycle 40 40 0.896 1,399 Car 8 11 0.086 * 1,399 Bicycle 9 10 0.708 1,399

69

Notes: Asterisks represent statistical significance of the difference between control and treatment communities. * indicates that the difference is statistically significant at the 10, ** at the 5, *** at the 1 percent level. Source: FDW Colombia baseline survey 2015

Water sources The coffee farmers extract water from different on- and off-farm sources (Table 14). Almost 70 percent have an own on-farm water source. The main on-farm sources are rivers, streams, and springs. More than 50 percent share these sources with other farms. The most common off-farm water source are aqueducts that normally provide the whole neighbourhood with water. Most of these aqueducts were installed by the coffee federation and are operated by the communities themselves.

Table 14: Water sources, in percent

Control Treatment p-value N Farm has own water source 67 67 0.940 1,399

River or stream 31 29 0.534 1,399 Spring 47 51 0.229 1,399

Lagoon 1 1 0.783 1,399 Wetland 1 1 0.998 1,399

Farm shares own water source with other farms 53 55 0.739 934 Farm receives water from aqueduct 51 57 0.044** 1,399 Farm pays for water 49 51 0.438 1,399 Monthly payment for water, in pesos b 5198.85 5168.91 0.944 697 Farmer knows monthly water consumption 4 8 0.007 *** 1,399 Farm has water concession 9 8 0.401 1,399 Main water source for human consumption a 0.448 1,398

Aqueduct 45 48 Water source of other farm 30 30 Water source of own farm 21 18

Farm does not purify water before drinking 9 9 0.785 1,399

Farm uses water for animals 56 48 0.004*** 1,399 Main water source for animals a 0.144 731

Aqueduct 34 41 Water source of other farm 37 30 Water source of own farm 25 27

Farm uses water for crop irrigation 80 84 0.064* 1,399 Main water source for crop irrigation a 0.140 1,142

Rain water reservoir 71 69 Water source of other farm 14 13

Aqueduct 6 9

Farm uses water for coffee irrigation 20 20 0.936 1,399 Main water source for coffee irrigation a

Water source of other farm 54 49 0.440 285 Water source of own farm 26 43 0.131 285

70

Aqueduct 20 18 0.632 285 Rain water reservoir 1 0 0.154 285

Main water source for coffee processing a 0.133 1,398 Aqueduct 40 43

Water source of other farm 33 33 Water source of own farm 23 21

Notes: Asterisks represent statistical significance of the difference between control and treatment communities. * indicates that the difference is statistically significant at the 10, ** at the 5, *** at the 1 percent level. a The three most frequent options of water sources are represented in the table. b The coffee farmers were asked about the amount they paid last month. Exchange rate 01.09.2016: 1,000 COP = 0.286 EUR. Source: FDW Colombia baseline survey 2015 Farms receiving water from an aqueduct have to pay for this service. Some few families are exempted from paying, for example because they administer the aqueduct’s operation or helped constructing it. In both control and treatment areas, about 50 percent pay around 5,000 Colombian pesos (~1.50 EUR) per month for their water. Normally the payment is a lump-sum fee, since water meters are very rare. Accordingly, only few farmers know their monthly water consumption with this share being somewhat higher among treatment farms than among control farms. Even though this and few other differences in Table 14 are statistically significant they are rather small in magnitude.

Depending on the purpose for which water is used, water sources vary (see Table 14). Most farmers purify the water before drinking it. The most common form of purification is boiling the water. Only very few farms have water filter. Note that the coffee farmers potentially extract water from different sources for the same purpose; the water sources presented here only represent the main source for the respective purpose.

Table 15: Irrigation of coffee plantation, in percent

Control Treatment p-value N Manual irrigation with water hose etc. 78 87 0.027** 285

Sprinkler irrigation system 22 11 0.009*** 285 Reason for not irrigating coffee cultivation 0.011 1,114

Coffee doesn’t need irrigation 65 71 Farm doesn’t have water to irrigate 31 23

Farn doesn’t have the money to irrigate 3 4

Source: FDW Colombia baseline survey 2015 What is interesting to see is that 80 percent of farms use water to irrigate their crops. For this purpose, most farms collect rain water in buckets and irrigate their crops with rain water. With regard to coffee farming, only 20 percent of the farms use any artificial irrigation and collected rain water is hardly used. Most farms that irrigate their coffee plantation do so manually using a water hose, a mobile water pump, or simply a watering can. Some few farmers also have proper sprinkler irrigation systems. The share of farms with these systems is substantially larger among control basin farmers. The most important reason why the majority of farmers does not irrigate their coffee plantation is that they deem their coffee plantations not to need irrigation. Some farmers exist, though, that state not to have enough water or money to irrigate the coffee.

71

Table 16: Water shortage and water excess, in percent

Control Treatment p-value N Farm experienced shortage of water in 2014 24 25 0.535 1,398 Farms with evidence of...

... erosion 29 34 0.055 * 1,399 ... mudslides, landslides and/or avalanches

affecting the residence 8 8 0.994 1,399

... mudslides, landslides and/or avalanches affecting the crops 28 32 0.099 * 1,399

Source: FDW Colombia baseline survey 2015

The IWM Colombia intervention intends to increase resilience of the coffee farmers against extreme weather phenomena, most importantly droughts and flooding. Among the treatment and control farms, around one quarter have experienced water shortages in 2014 (Table 16). Around one third of all farms furthermore report some incidences of erosion caused by heavy rainfall. The share is slightly higher among treatment farms. Erosion affects primarily the farmers’ crops; the residences are rarely affected.

Coffee cultivation Most coffee farmers own the farm they are living on (hereafter: residential owners - Table 17), the other part of farmers does not live on the farm. Some of this latter group commute to their farm on a daily basis or contract administrator to operate the daily business. The share of these absentee owners is significantly higher among treatment farms and we have to control for this during the final analysis.

Table 17: Coffee farming

Control Treatment p-value N Land tenure, in % a 0.039 ** 1,399

Residential Owner 63 54 Absentee Owner 31 39

Lease-to-own agreement 3 4 Total area of farm, in ha 4 5 0.365 1,399 Area with coffee cultivation, in ha 2 3 0.013 ** 1,399 Number of coffee trees 10,650 13,621 0.005 *** 1,399 Number of trees in production 8,564 11,156 0.003 *** 1,399 Average age of coffee plants, in years 4.21 4.36 0.259 1,362 Coffee production in 2014, in kg 245.96 348.51 0.024 ** 1,388 Share of coffee that was sold as…, in % b

Dry parchment coffee (cps) 86 88 0.026 ** 1,399 “Pasilla”c 8 8 0.900 1,399

before drying 2 2 0.753 1,399 Coffee cherry 1 0 0.011** 1,399

Share of farms that participates in the following sustainability initiatives, in %

Nespresso AAA 48 48 0.978 1,399 Fairtrade – FLO 26 28 0.393 1,399

Rainforest Alliance 12 15 0.084 * 1,399 4C Association 5 6 0.470 1,399

72

Notes: Asterisks represent statistical significance of the difference between control and treatment communities. * indicates that the difference is statistically significant at the 10, ** at the 5, *** at the 1 percent level. a The three most frequent options of land tenure are represented in the table. b The three most frequent selling forms are represented in the table. c Defective coffee beans, affected by plagues or drought. Source: FDW Colombia baseline survey 2015

The average farm size in our sample is 4 to 5 hectare and the area cultivated with coffee trees varies between 2 and 3 hectare. The farm size and coffee production is slightly higher among treatment farms. This can also be seen in the number of coffee trees (10,659 trees in the control and 13,621 in the treatment area) and the number of trees in production (8,564 trees in the control and 11, 156 in the treatment area). It is not surprising that the coffee production in 2014 with around 354 kg is about 100 kg higher for the treatment than for the control farms. These differences are statistically significant. Since the amount of coffee produced is an important factor for decisions on coffee processing equipment and behaviour, we have to deal with this difference carefully in the impact assessment. We will control for coffee production in the final analysis and in case this characteristic turns out to be driving impacts substantially, we will consider using matching approaches that allow us to use those control farms as counterfactuals for the treatment farms that are most comparable with regard to coffee production.

Almost 90 percent of the coffee produced in 2014 was sold as dry parchment coffee (café pergamino seco, cps), which are the processed and dried coffee beans. A very small share was sold without drying or even without peeling (coffee cherry) and around eight percent both among treatment and control river basins was of low quality and sold as so-called “pasilla”. These defective coffee beans surge because of plagues or because they dry out while growing. The share of coffee sold as cps is slightly higher among treatment farms, but the size of the difference is negligible.

About half of all coffee farms participate in at least one sustainability initiative; the most prevalent initiatives among our control and treatment farms is Nespresso AAA (48 percent in both areas), followed by Fairtrade – FLO (26 percent and 28 percent respectively) and the Rainforest Alliance (12 percent and 15 percent, respectively).

Figure 8: Sustainability initiatives

Sustainability Initiatives

Nespresso AAA61 is a private sustainability initiative with the following three pillars: quality, sustainability and productivity. Among other things, the farmers are helped to adopt best practices in coffee production (in order to guarantee high quality coffee), they are taught the protection of biodiversity, water management, assurance of farm workers’ rights and improvement of working conditions. Also they are assisted to improve their productivity and implement cost reduction initiatives.

The Fairtrade62 standards are designed to strengthen the position of small-scale producers. They promote an equitable world trade and the protection of the environment. Moreover, they demand

61 https://www.nestle-nespresso.com/sustainability/the-positive-cup/coffee 62 http://www.fairtrade.net/standards/aims-of-fairtrade-standards.html

73

the prevention of child labor, the establishment of decent salaries and living conditions, and the implementation of safety and health standards in production.

The Rainforest Alliance63 pursues mainly the protection of the environment in coffee cultivation regions, but also promotes social standards for the coffee farmer, his family and workers. Among the basic ecological principles are reforestation and the preservation of ecosystems, the protection of wildlife, the conservation of soils and water sources, as well as the prevention and correct disposal of waste, and recycling.

The 4C Association64 maintains the 4C Code of Conduct, which establishes social, economic and environmental principles for sustainable coffee production. The purpose is to improve the economic viability for the coffee producers (i.e. productivity, efficiency and market access) and the working and living conditions of their families and workers. Coffee farmers are furthermore encouraged and taught to preserve natural resources such as primary forest, water, soil, biodiversity and energy.

Table 18: Decision making with regard to coffee cultivation, in percent

Control Treatment p-value N

Coffee cultivation Share of HHs where decision maker is… a 0.163 1,399

Owner 81 81 Spouse 9 7

Children 5 6 Share of female decision maker 11 13 0.098 * 1,399 Share of HHs where the implementer is… b 0.658 1,399

Owner 61 58 Children 8 11 Worker 9 10

Share of female implementer 4 6 0.055 * 1,399 Coffee production Share of HHs where decision maker is… a 0.051 * 1,399

Owner 80 78 Spouse 9 6

Children 5 6 Share of female decision maker 11 14 0.053 1,399 Share of HHs where the implementer is… b 0.455 1,399

Owner 65 61 Administrator 9 11

Spouse 10 9 Share of female implementer 6 8 0.110 1,399 Coffee commercialization Share of HHs where decision maker is… a 0.211 1,399

Owner 83 82 Spouse 8 6

Children 4 6 Share of female decision maker 14 16 0.132 1,399 Share of HHs where the implementer is… b 0.664 1,399

Owner 77 74 Spouse 8 8

63 http://www.rainforest-alliance.org/de/work/agriculture/coffee 64 http://www.4c-coffeeassociation.org/the-standard/about

74

Children 6 8 Share of female implementer 12 11 0.746 1,399 Forest activities Share of HHs where decision maker is… a 0.378 807

Owner 86 81 Spouse 6 6

Children 4 6 Share of female decision maker 12 17 0.041 ** 807 Share of HHs where the implementer is… b 0.657 807

Owner 69 63 Children 7 9

Administrator 7 8 Share of female implementer 5 8 0.077 * 807

Notes: Asterisks represent statistical significance of the difference between control and treatment communities. * indicates that the difference is statistically significant at the 10, ** at the 5, *** at the 1 percent level. a The three most frequent decision maker are represented in the table. b The three most frequent implementer are represented in the table. Source: FDW Colombia baseline survey 2015

For around 80 percent of all the coffee farms, the owner is the main decision maker with respect to coffee cultivation, coffee production, coffee commercialization, and forest activities

Table 18On fewer farms the main decision maker is either the owner´s spouse or their children. The differences with respect to the decision maker are very small between the control and treatment farms, with only few of the differences being statistically significant. However, the share of female decision makers is generally slightly higher among treatment farms than among control farms. In most cases, the owner also implements most of the activities.

Coffee processing After the coffee cherries are harvested, they are transported from the plantation to the processing infrastructure. Not all coffee farmers have the processing infrastructure on the surveyed farm; some of them process the coffee beans on another farm of their possession, a family member´s farm or the neighbor´s farm. Since these places are normally also located within the same river basin, the potential impact on waste water contamination remains unaffected. In the control area, a share of 84 percent processes the coffee on the surveyed farm. In the treatment area, this share is with 81 percent slightly smaller with the difference being borderline significant.

As described in detail in Section 0, the processing consists of three basic steps: reception and classification of the coffee, pulping, removal of the mucilage and washing.65 For each of these steps, IWM Colombia distributes water saving equipment and teaches water conservation practices. As can be seen in Table 19 some of these devices and techniques are already used by the farms, both in control and treatment river basins.

For coffee reception, IWM Colombia recommends the usage of a dry hopper in order to reduce water consumption. Among the control and treatment farms, 35 percent and 46 percent already have a dry hopper, 14 percent and 13 percent have a wet hopper and 51

65 After washing, there is another step, which is the drying process. However, due to the fact that no water is used during this, we will not consider it in our analysis.

75

percent and 41 percent do not have a hopper at all. The difference in the distribution of hoppers between the control and treatment group is highly significant with more treatment farms having the IWM promoted dry hopper. Only around 9 percent of all farmers classify their coffee cherries to separate the good quality coffee from over-ripe and dry cherries as well as to remove small stones and small branches. Virtually all of them use cans or tanks with water to separate the cherries by flotation. Only a small share of farms reuses the water from the classification process as recommended by IWM. The share of farmers classifying cherries is higher among control river basins. Only two treatment farms use a hydraulic separator that IWM Colombia promotes for saving water in the classification process. This does not come as a surprise as this equipment is designed for farms that process more than 1,000 kg of coffee cherries a day. Only few of these farms exist.

For pulping, IWM recommends not using water in the process and not to use tubes with water to transport the pulped coffee from one processing step to the next. 99 percent of all coffee farms have a pulping machine. Less than half of them use water in the pulping process. The share is higher among control farms (49 percent) than among treatment farms (44 percent). Yet, the share of farms that transport their pulped coffee with water is higher in the treatment than in the control group (10 percent and 6 percent, respectively). The differences are again significant.

Table 19: Coffee processing, in percent

Control Treatment p-value N Coffee farmer processes coffee on surveyed farm 84 81 0.102 1,399 Farm knows their water consumption for coffee processing

2 2 0.824 1,389

Reception of coffee Fincas with the following type of hopper 0.000 *** 1,389

Dry hopper 35 46 Wet hopper 14 13

Does not have a hopper 51 41 Classification Farmer classifies coffee cherries 11 8 0.051 * 1,389 Farmer reuses water from classification process 28 20 0.331 126 Farmer uses hydraulic separator 0 0.29 0.155 1,389 Pulping Farmer has pulping machine 99 99 0.776 1,389 Farmer pulps with water 49 44 0.042 ** 1,389 Farmer transports the pulped coffee with water 6 10 0.003 *** 1,389 Removal of mucilage & Washing Natural fermentation Farmers that apply natural fermentation 99 96 0.000 *** 1,389 Share of those that…

... use the “winding water channel” 13 16 0.061 * 1,358 ... use the traditional fermentation tank 70 72 0.359 1,358

... use another device 8 5 0.058 * 1,358 ... use the Tub Tank 22 23 0.572 1,358

Farmer applies technique of rinsing 98 98 0.761 1,357 Number of rinsing rounds… 0.002 1,328

Less than four rinsing rounds 68 75 Four rinsing rounds 24 20

76

More than four rinsing rounds 9 5 Farm uses the Tub Tank and applies four rinsing rounds

6 5 0.676 1,358

Mechanic removal Farmer applies mechanic removal 2 4 0.005 *** 1,389 Share of those that use...

... Ecomill 15 0 0.025 ** 44 ... BECOLSUB 85 100 0.025 ** 44

Notes: Asterisks represent statistical significance of the difference between control and treatment communities. * indicates that the difference is statistically significant at the 10, ** at the 5, *** at the 1 percent level. Source: FDW Colombia baseline survey 2015

The last step, the mucilage removal and the washing, can be conducted in two ways: either via natural fermentation or mechanically. Some farmers have the equipment for both techniques and consequently apply both. In general, only bigger farms apply mechanical removal, since the corresponding devices require a certain amount of coffee for operating correctly and economically. The IWM intervention recommends mechanical removal for farms that process more than 1000 kg of coffee cherries a day. For smaller farms they recommend natural fermentation in a Tub Tank applying four rinsing rounds. Less rinsing rounds does not remove the mucilage sufficiently and produces lower quality coffee. More than four rinsing rounds do not have any additional effect on the coffee bean and are thus not necessary. Our data shows that at baseline, only slightly more than 20 percent of the farmers have a Tub Tank and only five percent also apply four rinsing rounds using the Tub Tank. Most farmers apply less than four rinsing rounds. Only around three percent apply mechanical removal.

For all processing steps, the proper operation and maintenance of equipment is crucial for water consumption and contamination. Our baseline data set contains all the necessary information in order to analyse the correct usage of the devices. This information will be important for the impact analysis after the follow-up.

Domestic Water Usage

IWM distributes water saving and contamination reduction equipment also at the domestic level. The sanitary infrastructure among the control and treatment households is very similar (Table 20); nearly all houses are equipped with a bathroom and a shower. The two groups only differ slightly with regard to the possession of a kitchen sink and laundry tub; these differences are significantly different from zero, but small in absolute numbers. Furthermore, there are no significant differences with respect to water reduction devices in the sanitary infrastructure. 10 percent of both treatment and control households installed a low water consumption toilet. Few farms also installed flow restrictors in their bathroom sinks, showers and kitchen sinks (4 percent of the control and 3 percent of the treatment farms).

Table 20: Domestic water appliances and water reduction devices, in percent

Household has… Control Treatment p-value N …bathroom 98 99 0.251 1,398 Share of those with… 0.625 1,379

… flow restrictors 1 1 … low water consumption toilet 10 10

77

… another water saving device 2 3 ….bathroom sink 40 43 0.259 1,398

Share of those with flow restrictors 8 5 0.287 580 ….shower 97 97 0.890 1,398

Share of those with flow restrictors 4 3 0.442 1,351 ….kitchen sink 79 84 0.029 ** 1,398

Share of those with flow restrictors 4 3 0.742 1,142 ….laundry tub 98 97 0.059 * 1,398

Share of those with flow restrictors 2 1 0.684 1,363

Notes: Asterisks represent statistical significance of the difference between control and treatment communities. * indicates that the difference is statistically significant at the 10, ** at the 5, *** at the 1 percent level. Source: FDW Colombia baseline survey 2015

Waste and Waste Water Disposal About 28 percent of the coffee farmers in both areas have a domestic wastewater treatment system and almost all of them effectively also use it (Table 21). The system consists of three parts that are necessary for a proper water treatment: a grease trap, a septic tank and an anaerobic filter. Not all of the treatment systems have all three components: Only 98 percent have a septic tank, almost 87 percent have a grease trap and about 82 percent have an anaerobic filter. Those households that do not have a water treatment system, pour their waste water somewhere on the farm (almost 50 percent) or into a specific place they use only for waste water (around 25 percent). The latter are normally holes in the ground, some equipped with a grease trap, some without. Around 13 percent pour the domestic waste water directly into a surface water body which is particularly harmful because it directly contaminates the water without being filtered by soil layers. In general, the way domestic waste water is disposed of is slightly more environment-friendly among treatment farms.

Virtually all farmers separate organic from the inorganic waste (Table 21); although there is a significant difference between control and treatment farms, the difference is very small in size.

Table 21: Domestic Solid Waste and Wastewater, in percent

Control Treatment p-value N Domestic solid waste HH separates the organic and inorganic waste 96 97 0.085 * 1,399 Domestic wastewater Farm has a domestic wastewater treatment system 27 28 0.798 1,399

Farms effectively uses the domestic wastewater treatment system

26 27 0.749 1,399

Share of treatment systems that … … have a grease trap 85 87 0.708 384 … have a septic tank 98 97 0.761 384

… have an anaerobic filter 80 82 0.626 384 Disposal of domestic wastewater without treatment a

Somewhere on the farm (unprotected) 52 44 0.015 ** 1,034 Hole in the ground 23 26 0.295 1,034

Directly to the water source 14 11 0.180 1,034

78

Notes: Asterisks represent statistical significance of the difference between control and treatment communities. * indicates that the difference is statistically significant at the 10, ** at the 5, *** at the 1 percent level. a The three most frequent disposal places are represented in the table. b The five most frequent treatment systems are represented in the table. C SMTA: Sistema Modular de Tratamiento Anaerobio; Modular Anaerobic Treatent System Source: FDW Colombia baseline survey 2015 The coffee processing procedure generates two types of waste that are potentially harmful to the environment. The first type of waste is solid and consists of the pulp that is left from the pulping process. If this pulp comes into contact with water (e.g. rainwater), toxic substances such as tannins, caffeine, chlorogenic acid, and potassium are washed out of the pulp and thereby enter the ground and surface water. These components hamper the process of degrading the organic matter even more. This is why the IWM intervention advocates for the use of roofed pits to protect the pulp from rainwater and prevent effluents to trickle away. In these pits, the pulp can be decomposed into organic fertilizer. A proper pit requires a roof and walls as well as an impermeable floor. Of those farms that have a pit (42 percent in the control and 61 percent in the treatment area), only 56 percent of control farms and 61 percent of treatment farms have a functional pit with proper walls, roofs, and flooring (see Table 22). While the overall share of coffee farms with a pit is significantly higher among treatment farms, no significant difference exists regarding the share of farms that owns a functional roofed pit.

In order to collect drainage water from the pit, IWM promotes the installation of tanks. Among the farms with a pit, 13 percent of the control farms and 16 percent of the treatment farms have a tank for drainage water – the difference is not significant. Around 88 percent of both the treatment and control farms make sure that the pulp decomposes completely. There are three methods that can be applied to accelerate the decomposition: (i) the rotation of the pulp, (ii) the addition of additives or (iii) a worm culture. 59 percent of the control farmers and 56 of the treatment farmers rotate the pulp in the pit on average every 29 and 32 days, respectively. These differences are not statistically significant, nor are the differences among those that add additives or have a worm culture.

Table 22: Waste and Wastewater from Coffee Production, in percent

Control Treatment p-value N Solid waste of the production process Farm has a pit to compost the pulp 42 60 0.000 *** 1,399

Pits is installed properly (i.e. with floor, roof and walls) 56 61 0.131 717 Pits has a tank for collecting drainage water 13 16 0.179 717

Farmer ensures that pulp decomposes completely 88 86 0.664 717 Farmer rotates the pulp in the pit 59 56 0.484 717

Average rotating period (days) 29 32 0.390 409 Farmer add additives 53 46 0.108 717

Farmer uses worm culture pit 11 10 0.633 717 Farm uses the decomposed pulp as organic fertilizer 95 91 0.036 ** 717 Application of organic fertilizer to…

… garden 17 15 0.468 666 … coffee plantation 82 82 0.906 666 … seedling nursery 28 27 0.983 665

… other crops 47 40 0.093 * 666 Disposal of pulp (without pit) a

79

At the coffee crops 52 54 0.534 733 At the ground or a heap 38 33 0.193 733

At the water source 0 0 0.844 733 Production waste water Farm has any production wastewater treatment system 14 14 0.652 1,399

… actually realizes production wastewater treatment 14 13 0.643 1,399 Farm has… b

… a skimmer 8 7 0.543 1,399 … a hydrolytic reactor 2 3 0.198 1,399

… a biological filter 3 3 0.623 1,399 … a SMTAc 2 3 0.202 1,399

… another treatment tank 3 0.4 0.001 *** 1,399 Disposal of production wastewater without treatment a

Somewhere on the farm (unprotected) 70 67 0.253 1,212 Directly to the water source 15 12 0.090 1,212

Hole in the ground 5 8 0.019 1,212 Share that treats leachates 2 5 0.001 *** 1,399 Share that has a system to treat the water used for washing the agrochemical supplies

7 8 0.510 1,076

Farmer paid fine for contamination of a water source 0.29 0.29 0.994 1,385

Notes: Asterisks represent statistical significance of the difference between control and treatment communities. * indicates that the difference is statistically significant at the 10, ** at the 5, *** at the 1 percent level. a The three most frequent disposal places are represented in the table. b The five most frequent treatment systems are represented in the table. C SMTA: Sistema Modular de Tratamiento Anaerobio; Modular Anaerobic Treatent System Source: FDW Colombia baseline survey 2015

Of those farmers who have a pit, the majority (about 95 percent) uses the decomposed pulp as organic fertilizer. About 82 percent in both areas applies it to the coffee plantation. Other places of application are other crops (47 percent of the control and 40 percent of the treatment farms) or the seedling nursery (about 28 percent). Those farmers who do not decompose their pulp in a pit, dispose the pulp mainly among their coffee crops, at the ground or at a heap. Virtually nobody disposes of the pulp directly into surface water bodies.

The second type of waste is liquid and consists of the wastewater produced during the fermentation and washing stage. It contains the dissolved mucilage consisting of sugars that by fermentation make the waste water very acid (pH of less than 4). Moreover, the digested mucilage builds a thick crust on the surface of the waste water. Only 14 percent of farms in both areas have a sewage system of whatever sort. Among the most common sewage systems are the skimmer, the hydrolytic reactor, the biological filter, and the modular anaerobic treatment system (SMTA). The main disposal places for untreated production water are similar to those of domestic wastewater. Again, around 13 percent of the farms dispose of the waste water directly to a surface water body. This share is slightly higher among control farms. Another by-product of the coffee production process is the leachate that stems from the mechanical mucilage removal. Since the share that applies mechanical removal is small, the share of those that properly treats the leachates is small as well (2 percent of the control and 5 percent of the treatment farms) and differs significantly between the two groups.

80

So far, hardly any farmer has ever been obliged to pay fines for contaminating water sources. Even though the new legislation on water contamination entered into force only at the beginning of 2016, the contamination level of most farms had already been too high according to the old legislation. However, the old legislation had apparently rarely been applied to sanction the farmers with fines. Whether the new legislation will be implemented more strictly remains to be seen.

Soil protection and forestry management In order to protect the soil, the IWM intervention promotes different conservation practices. They advise farms to conduct plant contouring, which is already done by more than 90 percent of farms. The coverage of soil with plants is exercised by around 90 percent of farms. Other practices are the establishment of protection areas, living fences (using trees and bushes as hedges) and soil coverage with weeds that do not compete with coffee trees for nutrients, space, water or light (so called “noble weeds”) .

In 2014, 8 percent of the control farms and 5 percent of treatment farms had already participated in a reforestation program. The difference between the two groups is significant and hence, needs to be controlled for in the final analysis. Among the control farms that participated in a reforestation program, an average number of 176 trees were reforested in 2014. The respective number is significantly lower in treatment river basins and amounts to 77 trees. In an open question we asked the farms for the reason they are reforesting. The most important reasons are the prevention of mud- and landslides, the protection of water sources and the conservation of forest relicts.

Table 23: Soil protection practices and reforestation

Control Treatment p-value N Share of farms that realize burnings 7 4 0.050 ** 1,399 The five most applied conservation practices, in %

Plant contouring 92 90 0.220 1,399 Soil coverage (plants) 89 92 0.065 * 1,399

Protection areas 76 74 0.346 1,399 Living fences 61 64 0.233 1,399

coverage with noble weeds a 57 57 0.847 1,399 Farms that participated 2014 in a reforestation program, in %

8 5 0.020 ** 1,398

Average number of trees reforested in 2014 176,19 77,02 0.033 ** 287 Main reasons for reforestation, in %

Prevention of mud-/landslides 26 28 0.738 289 Protection of water sources 46 47 0.926 289

Conservation of forest relicts 19 19 0.915 289

Notes: Asterisks represent statistical significance of the difference between control and treatment communities. * indicates that the difference is statistically significant at the 10, ** at the 5, *** at the 1 percent level. a noble weeds are those weeds that do not compete with the coffee trees for nutrients, space, water or light. Source: FDW Colombia baseline survey 2015

81

Meteorological stations The IWM intervention is going to install meteorological stations in each department. In certain areas, these stations already exist and around 4 percent of the farms are aware of their existence (Table 24).

Table 24: Meteorological stations and delivered information, in percent

Control Treatment p-value N

Farm is aware of a meteorological station in river basin 3 5 0.214 1,399 Farm has received information on...

... weather forecast 4 6 0.118 1,398 ... early weather warnings 5 6 0.486 1,397

Notes: Asterisks represent statistical significance of the difference between control and treatment communities. * indicates that the difference is statistically significant at the 10, ** at the 5, *** at the 1 percent level. Source: FDW Colombia baseline survey 2015

The meteorological stations produce information on the climatic and weather conditions in the respective zone. This information is supposed to reach the coffee farms through extension workers, the radio or even the internet. The share of farms that received zonal information on weather inquiries or early weather warnings is very small in both areas.

Evaluation risks and general data quality assessment

Assessment of data consistency and completeness The envisaged 1,400 farm interviews were administered successfully. Logistics worked out properly and no interviewee refused participation in the survey. The only shortcoming occurred in two control river basins in Caldas (Quebrada Pore and Quebrada San Pablo) where not enough control farms could be found within the 100m radius around the river. This is why additional farms were interviewed within a 200m radius. The total number of farms interviewed outside the 100m radius is with around 10 farms very small, though. Missing responses were minimized to a large extent by carefully selecting and pre-testing the questions that were included in the survey. The detailed discussion of the data collection instruments with IWM staff turned out to be very helpful. Verifications of the collected data were routinely conducted in the field. In particular at survey start, consistency checks were performed in order to ensure that all questions were well administered by the interviewers. The enumerators confirmed that respondents understood the questions well. Evaluation risks assessment

Will the differences between control and treatment group compromise the identification strategy?

82

For several indicators, we found statistically significant differences between treatment and control river basins. Most of these differences are small in magnitude, though, and do not hint at any structural difference that would confound the identification strategy. Moreover, the difference-in-differences approach that will be applied in this evaluation allows the groups to differ in terms of the levels of outcome variables (e.g. waste water treatment). The only underlying assumption is that the changes in variables would have been the same in the two groups in the absence of the intervention (the so-called parallel-trends assumption). The difference-in-differences model also allows us to include demographic and other socio-economic characteristics as control variables and assess the specific impact of the intervention while accounting for these other factors. Finally, farm-level matching may help to underpin the parallel trends assumption by putting similar treatment and control farms in one group for analysis.

Not least, at the time of the follow-up survey we will make an efforts to obtain information on general socio-economic changes in the river basins and whether they occur only in parts of the sample or in all river basins. This will further underpin the identification assumption. If we find that for certain river basins specific events might compromise the parallel trends assumption, we will make an attempt to account for this or, in the worst case, remove individual river basins from the sample.

All remaining differences will be analysed thoroughly in the final report to assess in how far results might be biased due to these differences. We will carefully analyse size and direction of possible biases and test sensitivity of the results as suggested by Rosenbaum 2002.

Will the impacts analysed among first round beneficiaries be representative for the whole treatment population?

The survey participants are all recruited from the first round of beneficiaries. In year 4 and 5 of the intervention, additional farms will be treated. In theory it is possible that the first round beneficiaries differ to some extend from those treated at a later stage. For example, it is possible to imagine that farmers that see most individual benefit for their farm decide to participate immediately. Other farms may take longer to be convinced. Since FNC has a data base with basic data on all farms, we will be able to scrutinize in how far beneficiaries of the different treatment rounds differ with respect to these characteristics, like production volume, location, age of coffee plants, property status, or other crops cultivated. In case differences can be observed, these differences will guide us to analyse in how far measured impacts can be expected to materialize also among second round beneficiaries. Different ways to do so are possible: First, qualitative approaches like focus group discussions with beneficiaries can be done to understand possible heterogeneous impact potentials. Second, we can identify subgroups in our survey sample that better represent the whole treatment population and estimate impacts only among this subgroup. These impacts will then be extrapolated to the whole treatment population.

Conclusion

This report presented the results of a baseline survey and contextual interviews as the basis for the in-depth impact evaluation of the “Intelligent Water Management Colombia” (IWM) intervention financed through the Sustainable Water Fund of the Netherlands Ministry of Foreign Affairs. The intervention is designed to improve water management among coffee

83

farmers by information and sensitization campaigns, training, hardware investments and an improved institutional environment.

The report provided background information on water consumption and contamination in coffee production and described the implementation of the IWM intervention. It presents insights from the Institutional Analysis. Furthermore, it outlined the evaluation strategy to assess the impacts of the project on the coffee farmer level. The baseline data that we collected among coffee farmers in all 25 treatment municipalities between August and December 2015 will serve as a yardstick for an end-line survey to be conducted in 2017. We will then use the two data waves to carry out robust difference-in-differences or before-after estimations to identify the impacts of the intervention. Additionally, we will assess further impacts of training activities using qualitative approaches at the time of the follow-up survey. Since implementation of the majority of IWM activities started at the beginning of 2016, it seems realistic that the follow-up survey can be implemented as scheduled at the end of 2017, possibly exactly two years after the baseline survey. We will monitor the implementation of the intervention over the course of 2016 and 2017 in order to take a well informed decision on the exact timing of the follow-up survey.

One major objective of this report was to verify the quality of the collected data. In general, the quality appears to be good: the data are internally coherent (e.g. larger farms exhibit higher coffee output), non-response rates are very low and with regard to our identification strategy we could underpin the assumption of sufficiently similar treatment and control river-basins. While a few statistically significant differences could be observed, the size of these differences (i.e. the economic significance) is mostly very small. We observe some differences regarding coffee cultivation that have to be analysed thoroughly in the final impact assessment to assess in how far results might be biased due to these differences. Yet, the differences-in-differences approach and multivariate regression analysis, potentially combined with matching procedures, will probably be able to control for the majority of confounding factors and we are confident that the identification assumptions will hold.

From a different perspective, the differences in coffee cultivation are informative, as they give insights into the targeting of the IWM Colombia intervention. In general, farms in the IWM Colombia treatment river basins seem to cultivate and process coffee in a more environment-friendly way as the average Colombian coffee farmer already at baseline. Accordingly, the intervention commences from a favourable starting point. This might not come as a surprise, as river basins have been purposefully selected among river basins where the sustainability initiatives of Nestlé and Nescafe had already done some preliminary work. It is important, though, when thinking about the upscaling of the intervention to other Colombian river basins. Impacts may materialize not as rapidly if the activities target less prepared coffee farmers. We will dedicate special emphasize to this aspect during the follow-up study.

84

References

Adams, M.R. and J. Dougan (1987), Green Coffee Processing. In: Clarke, R..J. and R. Macrae, ed., Coffee. Volume 2: Technology. New York, NY: pp. 257 – 291 Beyene A, Kassahun Y, Addis T, Assefa F, Amsalu A, Legesse W, Kloos H, Triest L (2011), The impacts of traditional coffee processing on river water quality in Ethiopia and the urgency of adopting sound environmental practices. Journal of Environmental Monitoring Assessment 184:7053–7063. Chanakya, H.N. et al (2004), Environmental Issues and Management, Primary Coffee Processing Process Safety and Environmental Protection, 82 (4): 291 – 300. Chislock, M. F., E. Doster, R.A. Zitomer & A. E. Wilson (2013), Eutrophication: Causes, Consequences, and Controls in Aquatic Ecosystems. Nature Education Knowledge 4(4):10. FNC-Cenicafe (2013), Manual del cafetero colombiano. Investigacion y tecnología para la sostenibilidad de la caficultura. Tomo 1. Chinchiná, Cenicafé. Haddis, A. And R. Devi, (2008), Effect of effluent generated from coffee processing plant on the water bodies and human health in its vicinity. Journal of Hazardous Materials 152(1):259-62. International Coffee Organization (ICO) (2016), Total coffee production by all exporting countries. Available at: http://www.ico.org/trade_statistics.asp [accessed 17.06.2016]. Mburu, J.K., Thuo, J.T., R.C. Marder (1994), The characterization of coffee waste water from coffee processing factories in Kenya. In: Kenya Coffee. Vol. 59, No. 690. 1757-1761. Rattan, S., A. K. Parande, V. D. Nagaraju, and Girish K. Ghiwari. (2015), A comprehensive review on utilization of wastewater from coffee processing. Environmental Science and Pollution Research, 22:6461–6472. REPÚBLICA DE COLOMBIA. MINISTERIO DE AMBIENTE Y DESARROLLO SOSTENIBLE (2015). Resolución 631 del 2015. Parámetros y valores límites máximos permisibles en vertimientos puntuales a cuerpos de agua superficiales y al alcantarillado público. Available at: https://www.minambiente.gov.co/images/normativa/app/resoluciones/d1-res_631_marz_2015.pdf Rodríguez Valencia, N., J. R. Sanz Uribe, C. E. Oliveros Tascon, C. A. Ramírez Gómez et al. (2015). “Beneficio del café en Colombia. Prácticas y estrategias para el ahorro, uso eficiente del agua y el control de la contaminación hídrica en el proceso de beneficio húmedo del café.” Chinchiná, Cenicafé. Rosenbaum, P.R. (2002) Observational Studies. 2nd edition. New York: Springer. Zuluaga, V. and F. ZAMBRANO. 1993. Manejo del agua en el proceso de beneficio húmedo del café para el control de la contaminación. Avances Técnicos Cenicafé No. 187. Chinchiná, Cenicafé.

85

Annex 1 – Coffee Processing

Reception of coffee and classification Wet hopper1 Dry hopper1

Water tank (tanque sifón)1 2

Submersible pump1

Hydraulic separator with hopper and screw conveyor:

86

1

2

Pulping2

With water Without water

Removal of mucilage & Washing

Winding Water Channel (canal de correteo)1

Semi-submerged channel (canal semisumergido)1

87

Tub Tank1

Mechanic mucilage remover1 BECOLSUB (screw conveyor & DESLIM)2

88

Ecomill2

Treatment systems

Septic System (domestic waste water)

89

Modular Anaerobic Treatment System (SMTA)

90

Roofed pit

Sources: 1) IWM – Manual for Interviewers of Needs Assessment. 2) Rodríguez Valencia, N., J. R. Sanz Uribe, C. E. Oliveros Tascon, C. A. Ramírez Gómez et al.

(2015). “Beneficio del café en Colombia. Prácticas y estrategias para el ahorro, uso eficiente del agua y el control de la contaminación hídrica en el proceso de beneficio húmedo del café.” FNC –

91

Cenicafé. Chinchiná.

92

Annex 2 - Permitted pollution level for coffee waste water

Resolucion 631 del 2015

https://www.minambiente.gov.co/images/normativa/app/resoluciones/d1-res_631_marz_2015.pdf

93

Annex 3 - Evaluation Questions and Indicators

Level Evaluation Question Indicator IWM – KPI?

Outcome

On farmer level Do farmers adopt clean and

efficient behaviour (water-related)?

Participation in relevant training activities Elicit details on water usage for processing coffee

7, 9

Do farmers adopt clean and efficient equipment (water-related)?

Participation in relevant training activities For each of the appliances promoted through Activity 3

Ask about: 1) ownership 2) financing 3) usage 4) maintenance

5) satisfaction

8, 10, 11, 18

Do farmers adopt forest and soil management activities?

Number of trees planted Reason of planting trees Incentives received Implementation of water source protection practices

3

Do women participate in decision making?

Who decides on household expenditures?

Who decides on economic decisions? Who decides on family planning? Who decides on children’s education? Opinion on role of women in different areas of life

Do farmers participate in associations?

Membership of head of household and spouse in different associations Frequency of attending associations

On river basin level Have River Basin Management

and Organization Plans been implemented?

Verify existence of plans and steps towards implementation 24

Do these plans include planning for ecological restoration and bioengineering?

Verify content of plans Analyse influence of other programs in the area like KfW reforestation programs

No

Have forest and soil management activities been implemented?

Verify implementation of forest and soil management activities on river basin level

No

Have water and weather monitoring stations been installed?

Verify existence of monitoring stations and produced data Have farms received information based on the weather data? Has this information been used for planning of coffee cultivation?

No

Impacts Do farmers use less water for

domestic uses? Total water used (if metered)

Induce water usage from equipment used and elicited usage

No

94

Level Evaluation Question Indicator IWM – KPI?

behaviour Do farmers use less water for

farming activities? Total water used (if metered) Induce water usage from equipment used and elicited usage behaviour

No

Do farmers produce less polluted waste water?

Induce water contamination from equipment used, usage and maintenance behaviour Make use of data collected by Cenicafé on river basin level

21

Do farmers experience less water shortages?

Incidence of water shortage 19

Are farmers more resilient to extreme weather events?

Incidence of landslides Incidence of floods, incidence of water shortage

19, 20

Do farmers save money?

Money paid for water usage (if applicable) Incidence of pollution fines

14

On institutional level Does institutional arrangement

have influence on coffee farmers’ behaviour?

We do not see any transmission channel of how institutional arrangements could influence coffee farmers’ behaviour within the monitoring horizon of this evaluation.

No

95

FDW ESTABLISHING IMPACT REPORT – JUNE 2016

SECTION FOUR: HOUSEHOLD SURVEYS FOR IMPACT

EVALUATION OF SUSTAINABLE WATER SERVICES

(SWSH) IN HARAR, ETHIOPIA

Getnet Alemu and John Cameron

96

Table of Contents

1. Intervention Context 97 2. Evaluation Approach 99

3. Data Sources and Sampling Frame 99 3.1 Survey design, sampling frame and consideration for urban and peri urban population .... ............................................................................... 100

3.2 Survey design, sampling frame and considerations for rural households 102

4. Baseline Survey Reports 103 4.1 Urban and peri urban .............................................................................. 103

4.1.1 General characteristics of households .............................................. 103

4.1.2 Water supply and quality................................................................... 105

4.1.3 Health, hygiene and sanitation .......................................................... 108

4.1.4 Finance and governance .................................................................. 110

4.2 Rural Household ..................................................................................... 111

4.2.1 General characteristics of households .............................................. 111

4.2.2 Water supply and quality................................................................... 113

4.2.3 Health, Hygiene and Sanitation ........................................................ 116

4.2.4 Finance and governance .................................................................. 118

97

1. Intervention Context

The Harari People Regional State (HPRS) including the Harar city is characterised by extreme water scarcity. In order to alleviate this problem, African Development Bank financed a project that brought water from near Dire Dawa over a distance of 71 km to Harar city which has been operational since 2012. The problem, however, is that Harar Water Supply and Sewerage Authority (HWSSA) does not have the financial and technical capacity to deal with the high calcium content of the water that disrupts the water supply system.66 The other problem is that in the Harar city Non Revenue Water (NRW) loss is estimated to be is close to 45%. This has created serious financial problem for the HWSSA in operating efficiency. HWSSA lacks technical expertise and capacity to adequately operate the water supply system physically and financially.

The rural water supply around Harar city is also characterised by scarcity and high failure rate of current systems. Only 46% of the population has access to safe water in acceptable quantities, leaving about 40,000 rural inhabitants poorly served. To address both rural and urban water supply shortage, the Sustainable Water Service in Harar (SWSH) project FDW Project was designed. The SWSH is an integrated water resources conservation and allocation approach that combines water resource conservation with capacity development, with reduction of NRW, and with construction of a decalcification reactor to ensure long term integrity of the urban water supply system.

The Project has a mix of software and hardware activities. These include improving the internal management processes of the HWSSA combined with improving the interface between supplier and customers in terms of NRW reduction which is currently estimated at about 45%, developing and implementing water resources conservation, building a decalcification plant, and providing improved access to water supply for 50,000 people. Provision of access to safe water is planned by constructing public water points and household connections (shared yard) for about 25,000 urban and peri urban dwellers and constructing rural water supply schemes for 25,000 rural people who are identified as poor and lacking affordable access to safe water. The project also aims to increase revenue generating water supply for HWSSA by reducing the existing high level of NRW.

This project is implemented by the HWSSA in partnership with other organisations: Vitens Evides International BV (the lead partner), Heineken Brewery SC, Harari People Regional State (HPRS), Ethiopian Catholic Church-Social and Development Coordinating Office of Harar, MS Consultants, Acacia Water, and DHV BV.

66 Recent water quality measurements of the Dire Dawa wells (o.a. measured by VEI during a due diligence research of June 2012) showed that CaCO3 concentrations are in the range of 300-450 g/m3 which pose a very high risk of pipe clogging by calcium precipitates and system failure within a few years of system operation.

98

Therefore this partnership includes public, private and NGO organisations as envisaged in the FDW programme, though the NGO component is weak in terms of integration into local civil society.

The Establishing Impact Evaluation Team is questionnaire survey is focussed on the NRW intervention on water supply and revenue of HWSSA67, and the impact of improved water supply for 50,000 people on their wellbeing. Our power calculations suggested samples of around 150 (treatment or control) would be necessary and sufficient to assess significance of the FDW interventions for key variables.

2. Evaluation Approach The first pivotal step of any successful evaluation exercise is a common understanding of the intervention logic or Theory of Change (ToC) of SWSH project. This will help to assess the mechanisms behind observed changes and interventions to identify all possible impacts, including unintended ones.

The basic evaluation questions are:

i) Does improved access to water supply (due to SWSH) bring reductions in school absenteeism and time spent in fetching water and hence improve livelihood of the community?

ii) Does access to safer water supply bring changes in health?

As described in the intervention context, the intervention has a package context that includes capacity building. But the focus here is on the activities that provide improved access to water supply for 50,000 people: urban, peri urban and rural. By February 2015, about three thousand urban and peri-urban households had been identified by Harar local government (kebele) representatives/officials as being ‘poor’ and without direct access to piped water. In addition, the water utility authority (HWASSA) identified ‘urban’ households capable of being supplied through ‘yard’ connections or ‘peri-urban’ households to whom piped water supply could be extended through collective water points (one criterion for receiving funding support from a specific fund was not being ‘in-house’ connections). Based on this, the urban people are categorised into different groups based on types of intervention and areas where beneficiaries are found. The way in which households were identified by local government representatives/officials shows respect for local participative processes, but makes identifying an independent control group impossible. Firstly, the criteria used by local government to identify households are unclear and probably vary from kebele to kebele. In addition, each kebele was asked to identify all eligible households, and all are going to be treated but at different time. We take advantage of implementation sequencing and adopt a ‘pipeline’ sampling approach. Categories of households that are going to be treated later can then act as controls for categories of households that are going to be treated earlier in the implementation process which also has desirable ethical qualities. We took a three months gap between categories receiving 67 The NRW intervention will be evaluated using FDW Harar Project and HWSSA records and key informant interviews.

99

earlier interventions and later interventions as acceptable to assess immediate impact. We assume that drinking water interventions produce rapid adjustment at household level in response to access time, cost, and health status changes. For this a period of three months is considered a sufficient, if minimal, time difference between baseline and follow up surveys. Based on this we undertook baseline survey in Feb 2015. The follow up survey was designed to be conducted after three months. But the original plan was not fully followed and categories to be treated in early April were not treated and follow up survey could not be conducted after three months. Based on the advice from the project implementing agency we revised our plan to conduct the follow up survey in July 2015. When we visit Harar in July 2015, we found out that the implementation gap was not only short, but the revised plan was not fully followed. We did a further survey of the same households only for the urban with aim of obtaining some impact insights, assuming that response to change in drinking water availability/quality/costs would evoke a rapid response. This situation forced us to adjust the household survey design which implies that the evaluation will be taken care of with no control groups. Households surveyed before the intervention will serve as control for themselves to be compared with impact survey that will be conducted in February/March 2018.

3. Data Sources and Sampling Frame

The urban water supply has targeted 25,000 people with five different types of implementation intervention; most funded by funds from the FDW Harar town water project:

i) Rehabilitation of water tankers to deliver water to a number of points where piped water is in short supply. This is linked to the rehabilitation of ROTO storage/distribution facilities. Both funded from general FDW Harar project funds. These interventions were regarded as very similar in probable impact to provision of collective ‘waterpoints’ and impact could be established from the ‘waterpoint’ sample.

ii) Yardpoint metered provision funded from a specific funding source aimed at small group collective provision where two to five households lived in close physical proximity.

iii) Water point metered provision funded by the same fund but available to wider local populations in ‘peri-urban’ areas judged to have deficient piped water supplies at present.

iv) ‘In-house’ metered provision funded by a fund without collective conditionality. These households are located between households with existing ‘in-house’ piped water provision and therefore not suitable for ‘yardpoint’ provision.

v) Institutional improved water provision to three institutions.

In what follows are discussed the distribution of targeted households for these different forms of provision and the decisions on data collection of the Establishing Impact Evaluation team.

100

Sampling frame and general sampling considerations for urban population shared yard connection (see Category 1, 2, and 4 in Table 1): HWSSA categorises urban and peri urban population into different groups based on types of intervention and areas where beneficiaries are found. The urban households for ‘shared-yardpoint’ interventions were identified by local government authorities on the general criteria of lack of direct access to piped water and local perceptions of socio-economic deprivation (very poor to pay for the connection). Households from each category are selected using a proportionate simple random sample (proportionate to number of households targeted for ‘shared-yard points’ in each category). We take 10% from each category to generate a suitable total sample size.

Sampling and data collection in practice: The target population was initially identified from a mixture of HWSSA records of ‘shared yardpoint’ target households in the early and later intervention woredas, mainly soft copies in English (but also some hard copies in Amharic). A random sample of households, proportionate to number of target households in each category, was drawn (‘yardpoints’ with single households and multiple households in the same ‘yardpoint’ were not included). The sample was geographically divided and teams of 2/3 interviewers (who had been trained for one and a half days) were given manageable numbers of potential interviewees for each day’s work and fully debriefed at the end of each working day.

Where cases in the sample were not found (urban households’ numbering is somewhat imprecise and names can be ambiguous), then substitutes were found through a new random selection, again broadly proportional to number of cases missing in each kebele.

Sampling frame and general sampling considerations for urban population individual connection (See category 3 and 5 in Table 1): population in category 5 are identified as the poorest of the poor and people in category 3 as people with HIV. These categories were not included in the original selection of households as these households were seen as unsuitable for collective water provision for reasons of location or risk of social discrimination. Lists of these households were finalised for interventions in February 2016. Sample surveys were undertaken with households in both categories in early March 2016 with a view to integrating these results with the February 2015 results and re-interviewing in February/March 2018 for a baseline/impact comparison.

The sampling approach used for households receiving individual connection (household level) will remain the same; 10% from the total households using simple random sampling. A group of 81 households were selected randomly from the list of category 3. Then we take the first name from the list of each group. But we could not find house no. for 11 households. Six of them were possible to replace from the

101

respective kebele just by looking the list of the group which has house number while the rest 5 could not be replaced as we could not find names with house number from the respective kebele. So we drop 5. We followed the same procedure to select 43 households from category 5.

Sampling frame and general sampling considerations for peri-urban households water points (Category 6, 7, 8, and 9 in Table 1): HWSSA categorises peri urban population in to different groups based on areas where beneficiaries are found. Some categories do not have access at all to safe water and some have limited access. The four selected ‘peri-urban’ categories proved different in ethnic and socio-economic characteristics. Category 6 and 7 included locations that were villages based on agricultural activity and ‘suburban dormitories’, where virtually the whole working age population went to Harar town for economic activity. In addition, most people were overwhelmingly Oromifa speakers while most of the interviewers did not have this language and local interpreters had to be found.

For each category we identify the proposed water point (with help of HWSSA). Starting from the site of the proposed ‘waterpoint’, interviewees were identified through a random ‘paces’ method (10 to 200 paces) moving away from the ‘waterpoint’ site in four directions with an arbitrary starting direction.

Table 1: Intervention categories and sample distribution for urban and peri urban68

Category Woreda Kebele Village Sample

Category 1 Abadir 4 and 5 Amir Nur/Abadir 20

Amir nur 1/2/6/7 14

Category 2 Hakim 18/19 Jinila/Hakim 50

Jinela 14/15/16 7

Category 3 76

Category 4 Aboker 11/12/13 Shenkor/Aboker 30

Category 5 43

Category 6 Erere 1 Aw-umer Gende Bube 36

Category 7 Dire Tayara Aboker Muti Gende Qejela 25

Category 8 Dire Tayara Miyay Gende Ahmed 20

Category 9 Sofi Sofi Deker 62

68 Category 5 and rural water supply are not covered in this baseline survey.

102

Source: HWSSA and HH survey, February 2015 and March 2016

3.2 Survey design, sampling frame and considerations for rural households

The FDW Harar Sustainable Water Project had a goal of improving water accessibility for 25,000 rural people. Originally, the Establishing Impact team planned to conduct all baseline surveys in February/March 2015, but the FDW Project was not in a position to identify intervention target areas or households at that time and so it was decided to postpone the baseline survey until February/March 2016.

The plan is not yet clearly finalised even by February 2016, but there were sufficient indications of interventions and locations to design a survey of 300 households which will permit possible ex post bifurcation into treatment and control groups after the 2018 re-survey and/or a baseline/impact comparison. Postponing again for February/March 2017 would be too late for a baseline survey. It is with this assumption that the baseline survey was conducted in March 2016.

There are three rural districts in the region. Enhancing rural people’s access to drinking water is envisaged as taking place through three forms of intervention:

i) Spring development in the south/south west of the Region at the foot of a watershed (the control here is households in villages at the foot of the other side of the watershed);

ii) Sand dam development in the north east of the region (the control group is households in villages on the north east side of the region close to rivers where sand dams might be an option). This intervention connects directly with another FDW Project objective of experiments in innovatory methods of local water retention; and

iii) Rehabilitation of pre-existing, local ‘non-functioning’ wells and associated pumping mechanisms across the whole region with no clear target groups at this time (the household survey is designed to include large numbers of households close to wells and this data can be bifurcated into treatment and control groups at the impact evaluation stage.

The survey aims at interviewing about 300 households in 25 to 28 villages/settlements in all districts. The villages fall into five broadly geographical groups in 3 districts:

i) North east (sand dam intervention) ii) South east (spring development) iii) South west (control group for spring development target group) iv) North east (control group for sand dam intervention)

103

v) Erere valley (concentration of pre-existing water supply interventions, some of which could benefit from rehabilitation)

The FDW Harar Sustainable Water Project proposal aimed to improve quantity/quality of drinking water for 25,000 rural people. The FDW Establishing Impact proposal stated it would conduct household questionnaire sample surveys (to a power of 0.90) to establish impact of the FDW project. A randomised sample of about 150 households (about 750 people) from the intervention target groups and the same number from control groups given the relative homogeneity of the whole Harar rural population in key variables was calculated as likely to meet the power requirement.

Table 2: Rural sample frame and distribution District Kebele Village Possible intervention

Erere Erere Dodot kebele G/Mudir settlement potential sand dam treatment sample

Erere Hawaye kebele Halele Same

Erere Hareye same

G/Ayub Treatment and control groups for pump and water point rehabilitation

G/Mudi Jiramu same

G/Roqa Gema same

G/Qefera same

Sofi Sofi G/Jelie Treatment group for spring development

G/Kilitu Same

G/Fersa Same

G/Kola Same

Sofi settlement Same

Dire Teyara

Aw Berkhel G/Harosomaya Control group for spring development

G/Beyareba Same

Gende Hassen Same

G/Kalya Same

Aboker_Mut kebele G/Tulle Same

Dire Teyara kebele G/Jalle Same

G/Halele Same

Sigicha kebele Shake Ibrahim Same

Source: HWSSA and HH survey, March 2016

4. Baseline Survey Reports 4.1 Urban and peri urban

4.1.1 General characteristics of households Understanding important household level characteristics is crucial to understanding how programs like providing access to safe water impact household health, poverty and related issues. This section describes the key characteristics of the households. The analysis of such household level characteristics is based on data collected at the household level: demographics; income; water supply and quality; health, hygiene and sanitation; finance and governance.

104

General demography: The quantitative data gathered for this baseline came from a survey of 383 households with 1799 total household members of which 53.4% are female. From total household members about 59.3% are in the productive age group (16-59 years) and those aged 60 and over accounted for 6.3%. For urban households this is 66.9% and 8.4% respectively. The productive age group for peri urban household is relatively small, 49.7% and 3.7% are aged 60+ years. Total average household size 4.7 ranging between 1 and 16 members. Peri urban households are a little higher (5.6) than the urban household (4.2).

Table 3: General demographics (urban and peri urban households)

Age group Peri urban Urban Total survey

Female Male Total Female Male Total Female Male Total

Below 5 69 72 141 36 27 63 105 99 204

6 to 15 119 112 231 107 77 184 226 189 415

16 to 59 206 192 398 356 312 668 562 504 1066

60 plus 9 21 30 59 25 84 68 46 114

Total 403 397 800 558 441 999 961 838 1799

Source: HH survey 2015/16

Housing condition: The housing condition for most households is poor. Most live in one room (45.2%) and the situation is worse in urban (54.95) rather than peri urban (29.5%). For details see Table 4. Table 4: Housing condition Source: HH survey 2015/16

Source and level of cash income: There are different source of livelihoods in Harar city and its surroundings. The survey result revealed that about 27% of the households reported that their main source of cash income in the last three months

Variable Urban Households Peri Urban Households Sample Households

Frequency % Frequency % Frequency %

Number of rooms

One room 124 54.87 41 29.50 165 45.21

Two rooms 85 37.61 52 37.41 137 37.53

Three rooms 14 6.19 33 23.74 47 12.88

Four rooms 2 0.88 9 6.47 11 3.01

Five rooms 0 0 4 2.88 4 1.10

Total 226 100 139 100 365 100

Roofs Type

Thatch 2 0.84 1 0.70 3 0.79

Zinc or corrugated metals 232 97.07 142 99.30 374 97.91

Tiles/slates 1 0.42 0 0 1 0.26

Other 4 1.67 0 0 4 1.05

Total 239 100 143 100 382 100

105

is agriculture including livestock (see Table 5). Others (26%) stood second most important followed by daily labour (22%) and trade (16%). As expected, this pattern does not remain the same for urban and peri urban households. The most important source of cash income for peri urban households is agriculture including livestock (66.9%) followed by others (12.7%), while for urban households the main source is others (33%) followed by daily labour (31%).

The amount of cash income generated by most households in the last one month is small (see Table 6). About 42% of the households cash income is between 500 and 999 birr which is between 23 and 45 USD and 32% between 0 and 499 birr. In general, 74% of the household monthly income is less than 1000 birr.

Disaggregating this between urban and peri urban portrays an interesting story. Most urban households are poorer than peri urban households. While 39.1% of urban households income is less than 500 birr, it is only 28.9% for peri urban.

Table 5: Main sources and level of cash income

Source: HH survey 2015/16

4.1.2 Water supply and quality The main consideration in obtaining drinking water is cleanliness. Close to 73.8% of the total interviewed households reported that their main consideration in obtaining drinking water is cleanliness. The same pattern holds true for urban and peri urban (see Table 7). Both urban and peri urban households use different sources of water (see Table 7). The main source of water for urban households is neighbor’s private drinking (83.7%) which is followed by tanker (4.6%) and river/pond/lake (4.2%). As expected this is different for peri urban households. River/pond/lake (42.3%) is the major

Variable Urban Households Peri Urban Households Sample Households

Frequency % Frequency % Frequency %

Sources of Income

Agriculture 12 4.71 95 66.90 107 26.95

Trade 49 19.22 13 9.15 62 15.62

Crafts man 10 3.92 4 2.82 14 3.53

Daily labour 78 30.59 11 7.75 89 22.42

Remittances 22 8.63 1 0.70 23 5.79

Other 84 32.94 18 12.68 102 25.69

Total 255 100 142 100 397 100

Incomes

Birr 0-499 93 39.08 41 28.87 134 35.26

Birr 500-999 94 39.50 65 45.77 159 41.84

Birr 1000-4999 47 19.75 33 23.24 80 21.05

>Birr 5000 4 1.68 3 2.11 7 1.84

Total 238 100 142 100 380 100

106

source for peri urban households. The second most important is neighbor’s private drinking (29.1%) followed by collective hand pump/borehole (18.4%). Worth noting is that there is no own piped water tap and own hand pump/borehole in both urban and peri urban surveyed households. Table 6: Water sources

Source: HH survey 2015/16

Water is not available every day. Discontinuity is one of the basic characteristics. Among the 381 households surveyed, more than 24% reported that they could not get water for more than 20 days in the last one month (four weeks before the survey). This is almost the same for both urban and peri urban areas. The main reason for this is the failure of the piped sources (44%) followed by distance (20.6%). While failure of piped sources is the main reason for urban (42.6%), distance is major reason for peri urban households (41.8%). The second most important reason for urban households is distance (26.6%). For peri urban households distance is the major reason (42%) which is followed by failure of piped sources (37.3%) constitute the second most important reason (for details see Table 7).

Table 7: Discontinuity of water supply and main reasons

Variable Urban Households Peri Urban Households Sample Households

Frequency % Frequency % Frequency %

Main Consideration

Cleanliness 185 76.76 100 68.97 285 73.83

Time 5 2.07 4 2.76 9 2.33

Distance 31 12.86 33 22.76 64 16.58

Cost 20 8.30 8 5.52 28 7.25

Total 241 100 145 100 386 100

Water Sources

Piped 10 4.18 0 0 10 2.63

Hand pump 0 0 0 0 0 0

Unprotected well 0 0 1 0.71 1 0.26

Protected well 3 1.26 8 5.67 11 2.89

Collective hand pump 4 1.67 26 18.44 30 7.89

River/pond/lake 10 4.18 60 42.55 70 18.42

Rainwater 0 0 0 0 0 0

Neighbours private 199 83.26 41 29.08 240 63.16

Private run water point 1 0.42 3 2.13 4 1.05

Bottled 1 0.42 0 0 1 0.26

Tanker 11 4.60 2 1.42 13 3.42

Total 239 100 141 100 380 100

Variable Urban Households Peri Urban Households Sample Households

Frequency % Frequency % Frequency %

Zero 27 11.25 34 24.11 61 16.01

107

Source: HH survey 2015/16

Most households assess the quality of drinking water by its cloudiness, color, and taste. Most households use cloudiness as the major yardstick to assess the quality of drinking water; urban (67.6%) and per urban (57%). The second most important is taste (urban 62.9% for the urban and 54.3% for the peri urban (see Table 8). Most households do not treat the water before drinking. Some households add bleach/chlorine; 36% for Urban and 21% peri urban (see Table 9 for details). Table 8: Quality of drinking water

Source: HH survey 2015/16

Table 9: Treatment of water

1-4 35 14.58 21 14.89 56 14.70

5-9 55 22.92 29 20.57 84 22.05

10-19 66 27.50 22 15.60 88 23.10

20 or more 57 23.75 35 24.82 92 24.15

Total 240 100 141 100 381 100

Why you did not get enough water

Distance 21 9.72 46 41.82 67 20.55

Waiting 13 6.02 11 10.00 24 7.36

Piped source failed 95 43.98 41 37.27 136 41.72

Other source failed 11 5.09 2 1.82 13 3.99

Cost 6 2.78 2 1.82 8 2.45

Storage 22 10.19 0 0.00 22 6.75

Other 48 22.22 8 7.27 56 17.18

Total 216 100 110 100 326 100

Variable Urban Households Peri Urban Households Sample Households

Choice Frequency % Choice Frequency % Choice Frequency %

Colour Yes 148 61.92 Yes 69 48.94 Yes 217 57.11

No 91 38.08 No 72 51.06 No 163 42.89

Total 239 100 Total 141 100 Total 380 100

Cloudiness Yes 164 68.62 Yes 80 57.14 Yes 244 64.38

No 75 31.38 No 60 42.86 No 135 35.62

Total 239 100 Total 140 100 Total 379 100

Smell Yes 136 57.38 Yes 55 39.01 Yes 191 50.53

No 101 42.62 No 86 60.99 No 187 49.47

Total 237 100 Total 141 100 Total 378 100

Taste Yes 149 62.87 Yes 76 54.29 Yes 225 59.68

No 88 37.13 No 64 45.71 No 152 40.32

Total 237 100 Total 140 100 Total 377 100

Variable Urban Households Peri Urban Households Sample Households

Choice Frequency % Choice Frequency % Choice Frequency %

Boil Yes 40 16.9 Yes 8 6.7 Yes 48 12.4

108

Source: HH survey 2015/16

4.1.3 Health, hygiene and sanitation The health condition in the town and surrounding areas is not something serious. As presented in Table 10, among the total respondents, more than 65% has somehow an episode of diarrhea. Table 10: The household last have an episode of diarrhea needing medical treatment

Source: HH survey 2015/16

The main reasons for the indicated episode of diarrhea is bad water followed by bad sanitation. This is the same for both urban and peri urban households (see table 11). This implies that access to safe water will have significant impact on health.

Table 11: Reasons to have diarrhea episodes

No 197 83.1 No 132 94.3 No 329 87.4

Total 237 100 Total 140 100 Total 377 100

Bleach Yes 86 36.3 Yes 30 21.1 Yes 116 30.6

No 151 63.7 No 112 78.9 No 263 69.4

Total 237 100 Total 142 100 Total 379 100

Strain Yes 31 13.1 Yes 30 21.4 Yes 61 16.2

No 206 86.9 No 110 78.6 No 316 83.8

Total 237 100 Total 140 100 Total 377 100

Filter Yes 12 5.1 Yes 4 2.9 Yes 16 4.2

No 225 94.9 No 136 97.1 No 361 95.8

Total 237 100 Total 140 100 Total 377 100

Solar Yes 5 2.1 Yes 2 1.5 Yes 7 1.9

No 231 97.9 No 136 98.6 No 367 98.1

Total 236 100 Total 138 100 Total 374 100

Let Yes 61 25.6 Yes 30 21.3 Yes 91 24.0

No 177 74.4 No 111 78.7 No 288 76

Total 238 100 Total 141 100 Total 379 100

Variable Urban Households Peri Urban Households Sample Households

Freq % Freq % Freq %

Two weeks or less 27 16.8 11 31.4 38 19.4

Two weeks to a month 9 5.6 13 37.1 22 11.2

A month to three months 21 13.0 8 22.9 29 14.8

More than three months 36 22.4 3 8.6 39 19.9

Nobody is sick 68 42.2 0 0 68 34.7

Total 161 100 35 100 196 100

Variable Urban Households Peri Urban Households Sample Households

Frequency % Frequency % Frequency %

No idea 24 10.2 30 21.3 54 14.3

Bad water 139 58.9 82 58.2 221 58.6

109

Source: HH survey 2015/16

There were some parts of the family members that were prevented to perform their main activity due to these episodes of diarrhea. For instance, out of the total 118 households about 22% reported that there is an episode of diarrhea that prevents children to go to school. The situation in urban areas is worse; 29% while this is just 7.7% in peri urban (see Table 12).

Table 12: Episodes diarrhea and attending education?

Source: HH survey 2015/16

From our survey we found out that it is only 4.9% of peri urban households and 5% of urban households that practiced open defecation. About 58.7% adult member of households in peri urban areas use household simple pit latrine, which is not shared with other households and 22.4% are using shared simple pit latrine. The majority of urban households (48.7%) are using household simple pit latrine while 35.8% are using shared simple pit latrine (see Table 13). Most households dispose children faeces into toilet (64.4%). In peri urban areas this is 68% while for urban areas it is only 58.7%. In urban areas significant part is also thrown into ditch (20.6%) while this is as low as 1% in peri urban areas.

Table 13: Toilet facility

Bad sanitation 35 14.8 11 7.8 46 12.2

Bad food 18 7.6 7 5.0 25 6.6

Poor hygiene 19 8.1 10 7.1 29 7.7

Chance or fate/cultural reasons 1 0.4 1 0.7 2 0.5

Total 236 100 141 100 377 100

Urban Households Peri Urban Households Sample Households

Frequency % Frequency % Frequency %

Yes 23 29.1 3 7.7 26 22.0

No 56 70.9 36 92.3 92 78.0

Total 79 100 39 100 118 100

Urban households Peri urban households Sample households

Frequency % Frequency % Frequency %

Toilet facility

Flush/pour flush to septic tank 2 0.8 0 0 2 0.5

Modern Community flush toilet 8 3.3 1 0.7 9 2.3

VIP 6 2.5 1 0.7 7 1.8

Simple pit latrine (household ) 117 48.7 84 58.7 201 52.5

110

Source: HH survey 2015/16

Hygiene education is provided for children and communities. Parents reported that they themselves have learned about better hygiene as a public education. About 64.3% of peri urban and 86% of urban households reported that they are educated on better hygiene. In addition to this, children have also discussed at home about the education they got for better hygiene and sanitation from school. This is being practiced in the 64.1% households of peri urban and 75.7% urban households (see Table 14). This seems to have an impact on the sanitation practice of households.

Table 14: Hygiene education at school

Source: HH survey 2015/16

4.1.4 Finance and governance As described in Table 6, the main source of water for urban households is neighbor’s private drinking (83.7%) and for peri urban households is river/pond/lake (42.3%). Because of these most households do not receive bills from HWSSA. About 93.5% of urban households and all respondent from peri urban households do not receive bills from HWSSA. The basis of payment for drinking water is clear. As most get it from neighbors, the payment is per unit and at the moment of water collection. Nearly all households are under this category (see Table 15).

Simple pit latrine (shared) 86 35.8 32 22.4 118 30.8

Composting toilet 7 2.9 17 11.9 24 6.3

Bucket emptied in a ditch 2 0.8 1 0.7 3 0.78

Open area 12 5.0 7 4.9 19 4.96

Total 240 100 143 100 383 100

Disposal of children’s faeces

Baby was assisted to use toilet/latrine

10 15.9 6 6.2 16 10.0

Put/rinsed in to toilet or latrine 37 58.7 66 68.0 103 64.4

Put/rinsed in to drain or ditch 13 20.6 1 1.0 1 0.6

Thrown in to garbage 3 4.8 13 13.4 26 16.3

Buried 0 0 11 11.3 14 8.8

Total 63 100 97 100 160 100

Urban Households Peri Urban Households Sample Households

Frequency % Frequency % Frequency %

Hygiene education at school

Yes 159 75.7 84 64.1 243 71.3

No 51 24.3 47 35.9 98 28.7

Total 210 100 131 100 341 100

Hygiene from public education

Yes 197 86.0 92 64.3 289 77.7

No 32 14.0 51 35.7 83 22.3

Total 229 100 143 100 372 100

111

Table 15: Basis of payment for drinking water

Source: HH survey 2015/16

Some households observed that the cost of drinking water increased in the last three months more than the cost of food. About 25.5% of the sampled urban households and 38.9% of peri urban households confirmed this. Most households (64.5%) did nothing for the effect of the cost. Close to 35.5% sampled households (45% urban and 18.5% of peri urban) involved in some kind of adjustment for the effect of higher costs of drinking water. The most preferred adjustment was reducing drinking water use (see Table 16).

Table 16: Response to increasing costs of drinking water

Source: HH survey 2015/16

Despite the overwhelming majority of sampled households do not have their own water system and do not have any contact with water bureau in the last three months, most feels that the regional government is quite important for their supply of drinking water (43.7% urban and 67.8% peri urban).

4.2 Rural Household

4.2.1 General characteristics of households General demography: The quantitative data gathered for this baseline came from a survey of 304 households with 1730 total household members of which 52.1% are

Basis of payment Urban households

Peri urban households

Sample households

Freq % Freq % Freq %

Per unit of water used at moment of water collection

238 94.0 115 99.1 333 95.7

Per unit of water used some time after water collection

5 2.2 0 0 5 1.4

Fixed amount per day/week/month/year 8 3.5 1 0.9 9 2.6

Irregular share of payment for maintenance/repairs

1 0.4 0 0 1 0.3

Total 232 100 116 100 348 100

Adjustment for drinking water costs

Urban households Peri urban households

Sample households

Frequency % Frequency % Frequency %

I did not took any action 132 55.0

110 81.5 242 64.5

Reduce drinking water use 77 32.1

12 8.9 89 23.7

Economise on food 14 5.8 6 4.4 20 5.3

Economise on other items 7 2.9 3 2.2 10 2.7

Reduce savings 6 2.5 4 2.9 10 2.7

Other 4 1.7 0 0 4 1.1

Total 240 100 135 100 375 100

112

female. From total household members about 39.2% are in the productive age group (16-59 years) and those aged 60 and over accounted for 3.1%. Total average household size is 5.61 ranging between 1 and 13 members.

Table 17: General demographics

Age group Rural

Female Male Total

Below 5 200 176 376

6 to 15 320 303 623

16 to 59 351 327 678

60 plus 30 23 53

Total 901 829 1730

Source: HH survey 2015/16

Housing condition: The housing condition for most households is poor. Most walls are from

wood and mud (63.9%), a typical rural house.

Table 18: Rural housing condition by district

Source: HH survey 2015/16

Source and level of cash income: There are different source of livelihoods in rural areas. The survey result revealed that about 75.4% of the households reported that their main source of cash income in the last three months is farming. This is almost the same for all districts (see Table 19). The amount of cash income generated by most households in the last one month is small. About 58% of the households cash income in the last one month preceding the survey is less than 500 birr which is less

Variable Sample Dire Tayara Sofi Erere

Frequency % Frequency

% Frequency

% Frequency

%

Roofs type

Thatch 16 5.2 0 0 6 4.3 10 8.9

Zinc or corrugated metals 247 88.9 57 100 115 82.7 102 91.1

Other 18 5.8 0 0 18 12.9 0 0

Total 308 100 57 100 139 100 112 100

Walls

Wood and Mud 197 63.9 48 84.2 42 30.2 107 95.5

Wood and Cement 15 4.9 6 10.5 4 2.9 5 4.5

Hollow Blocks 1 0.3 1 1.7 0 0 0 0

Other 95 30.8 2 3.5 93 66.9 0 0

Total 308 100 57 100 139 100 112 100

113

than 23 USD. Given 5.61 average household size, this translates to less than 89 birr or 4 USD per head for one month. This is less than the poverty line even by Ethiopian standard.

Table 19: Source and level of rural households’ income

Source: HH survey 2015/16

4.2.2 Water supply and quality The main consideration in obtaining drinking water for the rural community is cleanliness. Close to 87.2% of the total interviewed households reported that their main consideration in obtaining drinking water is cleanliness. The same pattern holds true for all the three rural districts (see Table 20). Rural households use different sources of water (see Table 20). The main source of drinking water is river/pond/lake (36.1%) followed by collective hand pump (34.7%). This pattern does not hold for all rural districts. For instance, major source for households in Dire Tayara district is collective hand pump (40.7%) followed by protected well (22.2%). Collective hand pump is also the major source of for sampled households from Erere district (72.8%). River/pond/lake is second important source (11.6%). Access to water supply is worse in Sofi district. River/pond/lake is still the major source (62.8%).

Water is not available all the time including river/pond/lake. Among 304 rural households surveyed, 16.4% have no access to water for more than 5 days in the last four weeks before the survey. This is much worse in Sofi district; 20.4% have no access at all for more than five days. One of the main reason for this is distance. 37.7% of the sampled households could not get enough water due to distance. The

Total Dire Tayara Sofi Erere

Frequency % Frequency % Frequency % Frequency

%

Sources of Income

Farming 239 75.4 47 79.7 105 75.5 87 73.1

Livestock 17 5.4 0 0 4 2.9 13 10.9

Farming and livestock 30 9.5 4 6.8 14 10.1 12 10.1

Trader 12 3.8 0 0 6 4.3 6 5.0

Crafts man 0 0 0 0 0 0 0 0

Labourer 3 0.9 1 1.7 2 1.4 0 0

Remittances 4 1.3 1 1.7 3 2.2 0 0

Other 12 3.8 6 10.2 5 3.6 1 0.8

Total 317 100 59 100 139 100 119 100

Incomes

Birr 0-499 170 58.0 30 54.5 88 64.7 52 51.0

Birr 500-999 70 23.9 9 16.4 30 22.1 31 30.4

Birr 1000-4999 49 16.7 13 23.6 18 13.2 18 17.7

>Birr 5000 4 1.4 3 5.4 0 0 1 1.0

Total 293 100 55 100 136 100 100 100

114

second and third most important reason is waiting time and failure of piped sources (for details see Table 20). Reasons across woredas differ. While distance is the major reason in Sofi woreda (64.1%), failure of piped source is the most important reason in Erere district (47.1%). For Dire Tayara woreda waiting time is the main reason for not getting enough water (43.5%). Table 20: Water sources

Source: HH survey 2015/16

Table 21: Availability of water

Source: HH survey 2015/16

Sources of water supply

Total sample Dire Tayara Sofi Erere

Frequency % Frequency % Frequency % Frequency %

Main consideration

Cleanliness 265 87.2 52 91.2 118 87.4 95 84.8

Time 4 1.3 0 0 1 0.7 3 2.7

Distance 34 11.2 5 8.8 15 11.1 14 12.5

Cost 1 0.3 0 0 1 0.7 0 0

Total 304 100 57 100 135 100 112 100

Water sources

Own hand pump 4 1.4 2 3.70 2 1.5 0 0

Unprotected well 30 10.2 10 18.5 14 10.2 6 5.8

Protected well 24 8.2 12 22.2 5 3.6 7 6.8

Collective hand pump 102 34.7 8 40.7 5 3.6 75 72.8

River/pond/lake 106 36.1 0 14.8 86 62.8 12 11.6

Rainwater storage 2 0.7 0 0 0 0 2 1.9

Tanker 26 8.8 0 0 25 18.2 1 0.97

Total 294 100 54 100 137 100 103 100

Sample Dire Tayara Sofi Erere

Frequency

% Frequency

% Frequency

% Frequency

%

Availability of water

Zero 161 52.9 34 59.7 7 56.2 50 45.5

1-4 55 18.1 16 28.1 24 17.5 15 13.6

5-9 50 16.4 7 12.3 28 20.4 15 13.6

10-19 25 8.2 0 0 8 5.84 17 15.6

20 or more 13 4.3 0 0 0 0 13 11.8

Total 304 100 57 100 137 100 110 100

Reason for not getting enough water

Distance 52 37.7 4 17.4 41 64.1 7 13.7

Waiting 35 25.4 10 43.5 16 25.0 9 17.6

Piped source failed 27 19.6 2 8.7 1 1.6 24 47.1

Other source failed 13 9.4 1 4.3 3 4.7 9 17.6

Cost 0 0 0 0 0 0.0 0 0

Storage capacity 4 2.9 2 8.7 1 1.6 1 1.96

Other 7 5.1 4 17.4 2 3.1 1 1.96

Total 138 100 23 100 64 100 51 100

115

Like that of urban households, most rural households use cloudiness as the major yardstick to assess the quality of drinking water. Households in Dire Tayara and Sofi district used cloudiness as major yardstick. The only exception is Erere district where the major yardstick is color. Most households do not treat water before drinking in all districts (see Table 22).

Table 22: Water quality

Source: HH survey 2015/16

Sample Dire Tayara Sofi Erere

Frequency % Frequency % Frequency % Frequency %

Quality of drinking water

Colour Yes 233 76.1 46 80.7 105 75.5 82 74.5

No 73 23.9 11 19.3 34 24.5 28 25.4

Total 306 100 57 100 139 100 110 100

Cloudiness Yes 247 80.5 51 89.5 118 84.9 78 70.3

No 60 19.5 6 10.5 21 15.1 33 29.7

Total 307 100 57 100 139 100 111 100

Smell Yes 216 70.4 45 78.95 105 75.5 66 59.5

No 91 29.6 12 21.05 34 24.5 45 40.5

Total 307 100 57 100 139 100 111 100

Taste Yes 185 60.3 38 66.67 90 64.7 57 51.3

No 122 39.7 19 33.33 49 35.2 54 48.6

Total 307 100 57 100 139 100 111 100

Do you treat your drinking water to make it safer to drink

Boil Yes 13 4.3 1 1.7 3 2.2 9 8.3

No 291 95.7 56 98.2 136 97.8 99 91.7

Total 304 100 57 100 139 100 108 100

Bleach Yes 31 10.2 8 14.0 11 7.9 12 11.0

No 274 89.8 49 85.9 128 92.1 97 89.0

Total 305 100 57 100 139 100 109 100

Strain Yes 32 10.6 9 16.1 13 9.3 10 9.3

No 271 89.4 47 83.93 126 90.6 98 90.7

Total 303 100 56 100 139 100 108 100

Filter Yes 3 1.0 1 1.8 1 0.7 1 0.9

No 300 99.0 55 98.2 138 99.3 107 99.1

Total 303 100 56 100 139 100 108 100

Solar Yes 0 0 0 0 0 0 0 0

No 303 100 56 100 139 100 108 100

Total 303 100 56 100 139 100 108 100

Let Yes 56 18.4 12 21.4 25 18.0 19 17.3

No 249 81.6 44 78.6 114 82.0 91 82.7

Total 305 100 56 100 139 100 110 100

Other Yes 6 2.0 2 3.6 1 0.7 3 2.9

No 292 98.0 54 96.4 137 99.3 101 97.1

Total 298 100 56 100 138 100 104 100

116

4.2.3 Health, Hygiene and Sanitation The health condition in the rural district can be stated as good. About 73% of total sample households has reported no diarrhea that needs medical treatment. The main reason for an episode of diarrhea is bad water (see table 23).

Table 23: Episode of diarrhea needing medical treatment

Source: HH survey 2015/16

There were some parts of the family members that were prevented to perform their main activity due to episodes of diarrhea. For instance, out of the total 129 households about 8.5% reported that there is an episode of diarrhea that prevents children to go to school. The situation in Dire Tayara is worse; 24% while this is just 1.8% in Sofi and 8.2% in Erere (see Table 24).

Table 24: Episodes of diarrhea and attending education

Sample Dire Tayara Sofi Erere

Frequency % Frequency % Frequency % Frequency %

Episode of diarrhea needing medical treatment

Two weeks or less 40 13.2 10 18.2 18 13.1 12 10.9

Two weeks to a month 15 4.9 3 5.5 6 4.4 6 5.5

A month to three months

14 4.6 3 5.5 6 4.4 5 4.5

More than three months 17 5.6 2 3.4 8 5.8 7 6.4

Nobody is sick 216 71.5 37 67.3 99 72.3 80 72.7

Total 302 100 55 100 137 100 110 100

Reasons to have episodes

No idea 88 29.1 15 28.3 42 30.4 31 27.9

Bad water 147 48.7 31 58.5 68 49.3 48 43.2

Bad sanitation 29 9.6 4 7.5 8 5.8 17 15.3

Bad food 11 3.6 0 0 7 5.1 4 3.6

Poor hygiene 18 5.9 2 3.8 7 5.1 9 8.1

Chance or fate/cultural reasons

9 2.9 1 1.9 6 4.3 2 1.8

Total 302 100 53 100 138 100 111 100

Diarrhea preventing from attending education

Sample Dire Tayara Sofi Erer

Frequency % Frequency % Frequency % Frequency %

117

Source: HH survey 2015/16

From our survey we found out that about 40.8% of sampled population practiced open defecation. Relatively the figure for Dire Tayara district is much better (27.4%). About 45.7% adult member of households in rural areas use household simple pit latrine, which is not shared with other households. Most households dispose children faeces into toilet (39.7%). In most indicators of hygiene and sanitation, Dire Tayara district is better than the other two rural district.

Table 25: Toilet facility and disposal of children faeces

Source: HH survey 2015/16

Hygiene education is provided for children and communities. Parents reported that they themselves have learned about better hygiene as a public education. About 61.8% of sampled rural households reported that they are educated on better hygiene. In addition to this, children have also discussed at home about the education they got for better hygiene and sanitation from school. This is being practiced in the 56.6% households (see Table 26). Most households in Dire Tayara district are exposed to hygiene and sanitation education than the other two districts. This has an impact on using toilet facilities and disposal of children faeces (see Table 25 above). This is one evidence that suggests education can bring changes.

Yes 11 8.5 6 24.0 1 1.8 4 8.2

No 118 91.5 19 76.0 54 98.2 45 91.8

Total 129 100 25 100 55 100 49 100

Total sample Dire Tayara Sofi Erere

Freq % Freq % Freq % Freq %

Toilet facility

VIP 1 0.3 0 0 0 0 1 0.9

Simple pit latrine (hh ) 132 45.7 35 68.6 55 41.7 42 39.6

Simple pit latrine (shared) 38 13.1 2 3.9 23 17.4 13 12.3

Open area 118 40.8 14 27.4 54 40.9 50 47.2

Total 289 100 51 100 132 100 106 100

Disposal of children’s faeces Baby was assisted to use toilet/latrine

7 3.2 1 2.4 5 5.1 1 1.3

Put/rinsed in to toilet or latrine 87 39.7 19 46.3 40 40.4 28 35.4

Thrown in to garbage 69 31.5 11 26.8 30 30.3 28 35.4

Buried 5 2.3 1 2.4 3 3.0 1 1.3

Left in open 44 20.1 7 17.1 17 17.2 20 25.3

Other 7 3.2 2 4.9 4 4.0 1 1.3

Total 219 100 41 100 99 100 79 100

118

Table 26: Hygiene and education

Source: HH survey 2015/16

4.2.4 Finance and governance

As described in Table 20, the main source of drinking water is river/pond/lake which is followed by collective hand pump. Because of these most households do not pay for water. About 88.7 sampled rural households do not pay for water. It is an open access (see Table 27). Table 27: Basis of payment for drinking water

Source: HH survey 2015/16

In the rural areas collective hand pumps (second most important source of drinking water for rural districts) is managed by WASH committees (WASHCOs). This is supported by the significant size of respondents. Close to 90% of sampled households reported that WASHCOs is very important for their water supply (see Table 28).

Total sample Dire Tayara Sofi Erere

Freq % Freq % Freq % Freq %

Have your older children talked about hygiene lessons at school

Yes 155 56.6 38 73.1 61 49.2 56 57.1

No 119 43.4 14 26.9 63 50.8 42 42.9

Total 274 100 52 100 124 100 98 100

Have you yourself learned about better hygiene from public education?

Yes 189 61.8 46 80.7 86 61.9 57 51.8

No 117 38.2 11 19.3 53 38.1 53 48.2

Total 306 100 57 100 139 100 110 100

Main reason you did not pay for drinking water

Total sample Dire Tayara Sofi Erere

Freq % Freq % Freq % Freq %

No charge made 181 88.7 7 46.7 79 95.2 95 89.6

Contribute monthly for water committee

1 0.5 0 0 0 0 1 0.94

Contribute when the scheme is broken

3 1.5 1 6.7 0 0 2 1.9

Own water source 10 4.9 6 40.0 4 4.8 0 0

Could not afford to pay 2 0.98 0 0 0 0 2 1.9

Other 7 3.4 1 6.7 0 0 6 5.7

Total 204 100 15 100 83 100 106 100

119

Table 28: Importance of WASHCOs

Source: HH survey 2015/16

Surprisingly, close to half of the sampled households reported that there is no WASHCOs in their community. In those community where there is water committee, some households reported that the role of WASHCOs is to repair water points and equally important size of household reported that they have no idea about the role of WASHCOs (see Table 29). This is surprising as all rural districts are close to the Harar town.

Table 29: Roles of WASHCOs

Source: HH survey 2015/16

Water points are often broken down and disrupts the continuity of water supply. In the last six months only (preceding the survey) water points in Erere district have been broken down. About 62% of the sampled households in this district reported that water points have been frequently broken down.

Total sample Dire Tayara Sofi Erere

Freq % Freq % Freq % Freq %

Not important 23 7.74 0 0 7 5.30 16 14.41

Quite important 176 59.26 41 75.93 79 59.85 56 50.45

Vital 98 33.00 13 34.07 46 34.85 39 35.14

Total 297 100 54 100 132 100 111 100

Sample Dire Tayara Sofi Erere

Freq % Freq % Freq % Freq %

No local water committee 150 49.18 33 57.89 82 59.42 35 31.82

Maintains water supply 32 10.49 8 14.04 19 13.77 5 4.55

Repairs water supply 66 21.64 13 22.81 17 12.32 36 32.73

Improved water supply 4 1.31 0 0 3 2.17 1 0.91

Do not know 53 17.38 3 5.26 17 12.32 33 30.00

Total 305 100 57 100 138 100 110 100

120

Table 30: Status of water points

Variable Total sample Dire Tayara Sofi Erere

Freq % Freq % Freq % Freq %

Yes 113 37.67 22 40.74 23 16.79 68 62.39

No 187 62.33 32 59.26 114 83.21 41 37.61

Total 300 100 54 100 137 100 109 100

121

FDW ESTABLISHING IMPACT REPORT – JUNE 2016

SECTION FIVE: INTEGRATED WATER MANAGEMENT AND

KNOWLEDGE TRANSFER IN SISILI KULPAWN BASIN IN THE

NORTHERN REGION OF GHANA (FDW/12/GH/02) BASELINE

REPORT

Elena Gross69 and Carley Pennink70 71

69 Corresponding author: Elena Gross, University of Bayreuth, Chair of Development Economics and Bayreuth International Graduate School of African Studies (BIGSAS), Germany. Phone: +49 921 55 6107 Email: elena.gross@uni-bayreuth.de 70 Institute for Housing and Urban Development Studies, Erasmus University Rotterdam Acknowledgement: We would like to thank Dr. Joseph Amikuzuno from the University of Development Studies in Tamale for support and supervision of data collection. Further we would like to thank Yvette Schutgens, Raymond Frempong, Johannes Markert and Lukas Wellner for helpful research assistance.

122

Contents

1 Introduction ........................................................................................................................................ 12123

2 The Institutional Context of the PPP ................................................................................................... 12124

3 The Intervention in its regional context .............................................................................................. 12125

The study area ................................................................................................................................ 12125

Agriculture and irrigation in Ghana ............................................................................................... 12127

Description of the IWAD/SADA intervention ..................................................................................... 130

4 Evaluation Approach ............................................................................................................................... 132

Evaluation objective ............................................................................................................................... 132

Identification Strategy........................................................................................................................ 135

Sampling, sample size power calculations ......................................................................................... 136

Survey tools ........................................................................................................................................ 138

Survey Implementation...................................................................................................................... 139

5 Baseline results ....................................................................................................................................... 140

Qualitative results of Focus Group Discussions ................................................................................. 140

Quantitative Results of baseline survey ............................................................................................. 143

6 Risks for impact measurement ............................................................................................................... 152

Self-selection of participants in groups .................................................................................................. 152

Parallel programs ................................................................................................................................... 153

Time horizon ........................................................................................................................................... 153

References ................................................................................................................................................. 154

Annex ......................................................................................................................................................... 155

Annex A1 List of evaluation indicators ............................................................................................... 155

123

1 Introduction

This baseline report gives an overview of the data and information collected on the Sisili Kulpawn

irrigation project during the baseline phase since January 2015.

The intention of the project is to foster smallholder farmers and private sector led growth through

the promotion of integrated water management practices and the development of irrigation

agriculture in the Savannah agro-ecological zone in the Northern Region of Ghana. The project is a

pilot project which is planned to go to scale after 2016. Therefore, a robust impact evaluation of the

content of the project and the PPP is important to evaluate whether a sustainable impact of the

project is achieved.

Based on difficult savannah agro-ecological conditions such as annual flooding, erratic rains, drought

(dry spells) and poor soils, the Sisili Kulpawn (SK) Basin was chosen as the project area. Despite the

challenges identified, the region provides opportunities for the development of large scale

commercially irrigated agriculture with the availability of land, water and human resources. The

project area is relatively peaceful despite some land conflicts among chiefs in the area since 2010.

Since the 1950s there have been efforts to develop irrigation agriculture with a peak in the 1970s

and 1980s. Over the past three decades, a number of irrigation schemes have been implemented in

developing countries to strengthen agricultural productivity. As noted by the ODI (Overseas

Development Institute, 1976): “The potential for the production of food and other crops on existing

irrigation projects in many developing countries appeared enormous”. Evidence, however, shows

that the sustainability of these schemes aiming at long term impacts on household welfare have

been minimal (Namara et al, 2011).

Most farmer families in the project area obtain their land from the local chief who is the land holder,

official and spiritual leader. Land is acquired mainly through patrilineal inheritance or distribution by

the chief. The crops most often cultivated are cereals (maize, rice, and millet), root plants (cassava,

yam), cowpea, groundnuts and soya beans. Most farmers use fertilizer but overuse or misuse often

happens. Pest control is not wide spread.

The report is structured as follows: Section 2 gives a short overview of the structure and partners of

the Public Private Partnership (PPP) implementing the project, but the institutional analysis is not a

focus of this report and is discussed in detail elsewhere the FDW evaluation. Section 3 describes the

demographic and agricultural situation in the study area. Section 4 presents the evaluation approach

and the impact evaluation design of the study. Section 5 discusses the qualitative and quantitative

baseline survey results without assessing any impact measurement but reporting on the to-be

124

measured and evaluated outcomes. Section 6 reports risk which potentially can threaten or bias the

impact measurement.

2 The Institutional context of the PPP By mutual consensus between all partners, Wienco Ghana Ltd. took the initiative as the lead party to

plan and implement a 400 ha irrigation scheme and Farmer Field Schools (FFS) on the topic of

conservation farming in the Mamprugu Moaduri District in the Northern Region of Ghana.

According to the Memorandum of understanding, Wienco Ltd. has established a legal entity for the

purpose of the project, the IWAD Ltd., to which Wienco will transfer part of its right and obligations

of the project. IWAD Ltd. will act as the project coordinator and is responsible for operation and

management, communication between parties, administration, reporting and supervising the

progress of the project. IWAD is also responsible for the construction of the irrigation block scheme

and is coordinating the work of different contractors and consultants on the ground. IWAD Ghana

Ltd. joined the PPP as a separate company for the coordination and operational aspects at field and

local level.

The other partners in the project include the Government of Ghana represented by the Savannah

Accelerated Development Authority (SADA) to coordinate the government’s development plans for

the agricultural sector in Northern Ghana. SADA has been supportive in the regional promotion of

the project in the context of the local economic and agricultural conditions.

Wageningen University and Research Centre (WUR), represented by the Alterra research institute, is

responsible for the capacity component of the project. Alterra has set up strategic partnerships and

works in cooperation with PhD students of the University for Development Studies (UDS) and the

Savannah Research Institute (SARI). This supports the PPP in the knowledge transfer component of

the project.

RebelGroup International RV supports the PPP with its expertise in contracting and legal

implementation of the PPP.

The composition of the PPP is presented in Table 1.

Table 25 SK Project PPP structure

Partner Sector Strategic role

Wienco Ghana Ltd. Private Coordinator

125

Integrated Water and Agricultural Development Ltd. (IWAD)

Private Coordinator and Implementation

Savannah Accelerated Development Authority (SADA)

Public Governmental Representation

Wageningen University and Research Centre – Alterra

Research Capacity training and Research

RebelGroup International BV Private Contracting

The duration of the PPP agreement is from April 2013 to December 2017 and includes the flagship-

phase from April 2013 onwards and the start of the up-scaling phase from 2015 onwards.

The total project costs are EURO 11.7 million and the subsidy contributes 60% of the total budget or

EURO 6.9 million. The remaining 40% of the budget will be covered by Wienco (27.66%), SADA

(6.9%), Alterra (5.2%) and the RebelGroup (0.3%). IWAD will receive the subsidy in a separate

account and distribute the budget between parties. A detailed analysis about the allocation of the

subsidy will be conducted throughout the study.

3 The intervention in its regional context The study area

Ghana has a population of 26.4 Million which was growing at 2.05% in 2014. About 46% of the

population lives in rural areas. The World Bank classified Ghana as a Lower Middle Income country in

2010 ($1,046 to $4,125 GDP per capita). GDP per capita was $1,460 in 2014, annual GDP growth was

declining from 14% in 2011 to 4.2% in 2014. The most important sector contributing to GDP is the

service sector with 50% in 2014, followed by industry with 29% and agriculture with 21%, see Error!

Reference source not found.. The latest available data of 2010 estimates that 41% of the population

is working in the agricultural sector (45% of men and 37% of women), so probably most of the

people living in rural areas are engaged in agriculture.

126

Figure 9 Composition of GDP in Ghana since 2000

Source: World Development Indicators, 2015

Ghana has seen many improvements in its overall poverty situation since 1991 and has met the

MDG Goal 1 by 2010, 5 years ahead of target. However, this decline in poverty incidence faces great

variations across regions. Ghana’s three Northern Regions (Northern Region, Upper-West and

Upper-East) are facing the highest burden of extreme poverty. In 2012, the national poverty

headcount ratio was 24.2 percent (WDI, 2015). Poverty is not only highest in the three Northern

Regions but also most severe: the poverty gap ratio is 38 percent in the Northern, 39 percent in the

Upper East and 47 percent in the Upper West region. A key challenge are the working poor, i.e.

working fulltime but earning below the poverty line. A key challenge here remains education: only

48 percent of the working population has basic education, only 19 percent have completed

secondary education (UN Ghana, 2015).

The SK project is implemented in the Mamprugu Moaduri district, one of the 25 districts in the

Northern Region. The baseline survey was conducted in four different districts: West Mamprusi,

Mamprugu-Moaduri, and Builsa South. All districts except the latter belong to the Northern Region

of Ghana and share boarders with the project district. Builsa South is a district of the Upper East

Region, however it shares boarders in the south with Mamprugu Moaduri and in the west with West

Mamprusi. North Gonja is the fourth district included in the baseline survey as the project intends to

expand to this district in the near future. The Mamprugu Moaduri District with its capital Yagaba was

part of West Mamprusi (capital Walewale) until decentralization reforms in 2012, the North

Gonja District with its capital Daboya was segregated from West Gonja. Builsa South (capital

127

Fumbisi) was created out of the former district Builsa (capital Sandema) in Upper East (see

www.ghanadistricts.com ).

Besides the official political structure, traditional structures are still prevalent, powerful and

important in Ghana. These traditional systems are also referred to as traditional authority (TA) and

differ across the country due to every locality having its own culture, tradition and structure

(Mahama, 2009; Tonah, 2012). The TA are also widely known as chieftaincies. Overall the

composition of the TA is similar with the Chief as the traditional leader (both official and spiritual) of

the hierarchy and around him the Elders who provide advice and guidance. The government

acknowledged the importance of the traditional governance, however it defined clear roles for them

in order to enhance accountability and governance (Mahama, 2009). The chiefs draw their prestige

from the social and economic (landownership) role they play and have relatively strong judicial

power, mainly in civil affairs such as inheritance, family law and land tenure matters. Every

community has its own TA to ensure peace and tranquility. In all four districts a system is in place

whereby an overall chief is appointed who is the leader of all the communities within his area. In the

Mamprugu Moaduri and West Mamprusi district all the TAs are obliged to the king of Mamprugu,

the Nayiri. In Builsa South, all chiefs have to oblige to the paramount chief, the Sandem-Naba, who

leads the entire Builsa traditional area. For North Gonja, the leader is the Wasipe-Wura, who rules

the Wasipe traditional area which is one of the five areas in the Gonja Kingdom (GSS, 2014).

Agriculture and irrigation in Ghana

Agriculture is one of the most important income sources for people living in rural areas of low-

income countries. In 2010, about 40 percent of the Ghanaian workforce was employed in

agriculture, almost equally distributed between men and women. The contribution of agriculture to

GDP has been declining during recent years. A sharp decline of agriculture as share in GDP since

2009 can be noted. In 2009, the share was 31% and has since decreased to 22% in 2013. Recent

research found that this is not the result of a structural transformation process but rather resulting

from a decline in agricultural productivity. The decrease stems mainly from a declining contribution

of the crop sector from 24% to 17% in agriculture (GSS, 2015). Productivity, measured as output per

unit of arable land, was estimated to be declining by 8.5% during the period 1999 to 2009 due to

slow increases in the output of staple crops (3.7% per year) but even faster growing cultivated land

area (5.7% per year) (Akudugu et al, 2013).

Rain-fed agriculture has no future in dry areas. Besides conservation agriculture (CA), irrigation is

identified as a coping strategy for climate change (IPCC, 2014). Two types of irrigation systems exist

in Ghana. The conventional systems are mainly initiated and developed by the Ghanaian

128

government (in cooperation with the Ghana Irrigation and Development Authority (GIDA)) or

nongovernmental organizations (NGOs). The emerging or informal systems are initiated by private

entrepreneurs and farmers. It is estimated that less than a third of the total irrigated land in Ghana

lies within the public irrigation schemes. Very little is known about the location, development and

management of the informal irrigation schemes that account for the remaining two-thirds of total

irrigated land. Of the total 6.9 million ha of arable land only 33,800 ha are irrigated which represents

less than 0.5 percent of the total area. Potentially 1.9 million ha could be irrigable (Namara et al.

2011).

FAO (2014) estimates that less than 2 percent of the irrigable land in Ghana is actually irrigated.

About half of the total arable land of 15 Mio ha is estimated to have the potential for irrigation. The

problems of the mainly public irrigation schemes are lacking maintenance, difficult or insufficient

cost recovery because of under-utilization and insufficient capabilities of workers and farmers. The

infrastructure has deteriorated and many public irrigation projects are not able to fully develop their

potential or not functioning at all (Namara 2011; FAO 2014). Landownership conflicts are another

reason for difficulties arising from public irrigation schemes (Alhassan et al, 2013).

Besides the big public irrigation schemes several small schemes exist having different property

structures. Financial problems exist at irrigation-scheme level and at farmer-household level. The

irrigation scheme needs money for maintenance, rehabilitation and operation. However, the user

pays principle is not applicable because the skill level of workers is too low (sometimes dealing with

illiteracy), especially when it comes to reading water meters and writing water bills to achieve

financial sustainability. Farmers have lacking or far too low willingness to pay for maintenance and

investment costs. Given an in-field irrigation system costs about USD 2 500 per hectare (FAO, 2014),

the gap to the farmer’s willingness to pay (about USD 8.5 per hectare per year, see Braimah and

Agodzo, 2014) is huge. Paying for the irrigation scheme is only one thing, however, for achieving

higher yields, irrigation is only effective when combined with better crop management methods

(application of agrochemicals). Farmers have to accept massive investment, both time and money

wise, to benefit from irrigation. However, most of the farmers are smallholders practicing

subsistence farming and often own or rent less than a hectare. A study of 40 farmers using the

Botanga irrigation scheme in 2012 by Alhassan et al. (2013) shows that farmers are usually

smallholders72 with an average farm size of 0.8 ha. 6 percent of farmers were leasing their lands

while the majority gets authority via a family balloting system. The average lease price per hectare

72 The definition of smallholders differs between countries, agro-ecological zones, size of land cultivate and number of livestock. (FAO, 2004) http://www.fao.org/docrep/007/y5784e/y5784e02.htm#TopOfPage

129

was GHC 24.17. The average income of the farmer was GHC 1760 while the median was only GHC

500.

Besides irrigation, conservation agriculture (CA) is a promising strategy to improve yields and income

of farmers in developing countries. CA is a set of soil management practices, which promote

sustainable and profitable agricultural development by reducing the disturbance of the soil’s

structure as well as its composition and biodiversity to a minimum level. Introduced by the FAO as a

“concept for resource-efficient agricultural crop production based on integrated management of

soil, water and biological resources” (Giller et al. 2009, 24), it calls into question the cleaning of the

seedbed through conventional tillage based on the intensive use of a heavy plough. CA includes

three major, simultaneously applied principles: 1) Continuous minimum mechanical soil disturbance

or minimum tillage, 2) Permanent organic soil cover or mulching and 3) Diversification of crop

species grown in sequences or associations, i.e. crop rotation instead of mono-cropping (FAO, 2015).

Beyond that, CA promotes additional management practices favoring the implementation and

execution of conservation farming methods. Examples are integrated disease and pest management,

avoidance of crop residue burning and limitation of human as well as mechanical traffic upon the

topsoil. In the last decades, the development community - from political and institutional decision-

makers to academic researchers and extension workers - has announced the urgent need for a more

sustainable agricultural production and the practice of CA (Knowler and Bradshaw 2007; Wall 2007;

FAO 2015). South America has the largest area under CA application. Within the Mercosur countries,

70 percent of the total cultivated area is permanently under CA cultivation. Especially in Brazil, CA

has been a success story: today, about 32 million hectare are planted under CA (FAO 2015). Since

first trials in 1972 were mainly held by wealthy farmers, CA practices have spread widely after

herbicides and adequate direct tillage equipment became available. In Africa, CA methods are

mainly practiced in the continent’s Eastern and Southern regions. About 40 percent of the total area

under CA is located in South Africa, where it is mainly practiced by large-scale commercial farmers. It

is estimated that more than 400,000 smallholder farmers working on a total area of one million

hectares practice CA methods in SSA (Kassam et al. 2012). Farmer field schools for smallholders

strengthen the understanding of CA methods and their adaptation. At the same time, supply chains

are fostered in order to develop the necessary machinery for CA adoption in smallholder farming

systems. Building on a mixture of traditional, mostly indigenous knowledge and experience of

successful CA adoption in Asia and South America, conservation farming methods are seen as a

promising alternative to promote sustainable agriculture in Africa. With a special regard to the

challenges of African smallholder farmers, CA can address the challenges of climate change, high

energy cost, environmental degradation, and labor shortages (Kassam et al. 2009).

130

Description of the IWAD/SADA intervention

The intervention will introduce effective water management practices (WMP) and conservation

agriculture (CA) or conservation farming methods (CFM) into an area which is traditionally

characterized by dry-land farming. Through providing a reliable source for irrigation water,

introducing water conservation measures through improved crop and land management and

offering knowledge transfer to smallholder farmers, an improvement of living conditions (see theory

of change below) is envisaged. Commercial irrigation is a new technology in the agricultural

production system in the area and will require properly trained farmers and willingness to pay for

water supply and inputs. The project wants to include committed farmers and therefore

subsequently includes farmers from the Farmer Field Schools (FFS) who passed the first round of

training on CA and show their interest for inputs and irrigation system (IS) farming later on. In total

3000 out-grower smallholder farmers will benefit from the program by receiving training and

assistance on conservation rain-fed farming in the project villages. In four villages, approximately

150 farmers will get access to commercial irrigation farming (block irrigation) and benefit from the

new infrastructure. The farmers are supposed to organize themselves in groups and form farm

cooperatives for knowledge transfer and establishing an agriculture business. The FFS will be held in

22 villages (four irrigation villages and 18 villages only CA).

The technical component of the project contains the construction of four different systems in the

irrigation block scheme: Pivot irrigation, overhead sprinkler, furrow irrigation and drip irrigation.

The irrigation systems can be used for an additional harvest in the dry season and as supplement in

case of a drought or low rainfall in the rainy season.

The two different interventions (IS and FFS) will take place in multiple dimensions: development of

irrigation infrastructure through the building of an irrigation dam, bulk water infrastructure, micro-

credit provision, market access, supplies and knowledge transfer over a long-term period for farmers

in communities of the project area. A season-long training attended by farmers- in the FFS is a

requirement before receiving inputs (rain-fed/conservation farming) or/and access to the irrigation

scheme. The two treatments are structured as follows:

1) Irrigation block scheme: The intervention will introduce effective WMP into an area which is

traditionally characterized by dry-land farming by providing a reliable source for irrigation, introduce

water conservation measures through improved crop and land management and offer knowledge

transfer to smallholder farmers. During the flagship project, villages in this group receive training on

conservation farming techniques (obligatory FFS) to qualify for farming on the irrigation farm. The

total irrigated area covers 400 ha, out of which 250 ha will be allocated to nucleus farms

131

(commercial farming by IWAD) and 150 ha to smallholder farmers in 4 villages (Yagaba, Loagri,

Gbima and Kuuba). An assignment strategy of the program set by IWAD as a requirement, is that

farmers who participate in the irrigation farming get a proportionate amount of plot with irrigation

in relation to their current land holding, experience, education level and yield. This assignment

assures that farmers have some experience in farming on larger plots where they sell part of the

harvest and not only do subsistence farming and that farmers are trained in using CFM.

2) FFS: The other part of the intervention is the knowledge transfer that goes directly to farmers

participating in FFS on CFM. Farmers receive training on the planting of five different crops: cotton,

rice, soybean, maize and sorghum in the course of 14 training sessions during the farming season

(rainy season). The content of the session is structured for a whole planting period, starting with

sowing, followed by chemical use and ending with harvest and clearing the fields. Farmers can

qualify to receive inputs on credit only if they follow the training completely. The credit will be given

per crop and in-kind as seeds, fertilizer, herbicides and pesticides. Farmers pay back the credit in

cash or in-kind in form of produce. Farmers can also sell their harvest to IWAD. In 2014, 13

communities received the first sessions of the conservation farming training i.e. farmers had the

possibility to qualify themselves to receive inputs for credit in 2015. An additional 9 villages will

receive the possibility to set up a demonstration field and receive the first session of the training

sequences from 2015 onwards. This design gives us a new dimension for the impact evaluation as

we can observe differences in outcomes one and two years after the intervention took place in the

same setting.

Figure 1 shows the project area. Treatment villages are marked in red, control villages are marked in

black. The location of the demonstration field for the FFS are highlighted in yellow. See Annex XX for

a list of treatment and control villages and respective population size.

132

Figure 10 Project area

© OpenStreetMap (and) contributors, CC-BY-SA; Sources: Esri,DeLorme, USGS, NPS

Source: GPS data SK Basin baseline dataset 2015. GPS data of demonstration fields provided by IWAD.

4 Evaluation approach Evaluation objective

Based on the Monitoring and Evaluation system of the project and relevant literature (Waddington

et al, 2014), a set of possible output, outcome and impact indicators was defined. According to the

ToC, a set of evaluation questions is defined for each level of the ToC. The table in Annex A1 displays

evaluation questions and proposes corresponding indicators of how to approach these questions.

Using this available set of evaluation indicators and complementing it with existing findings of recent

research as well as the view of local experts, it was possible to develop a Theory of Change, see

Figure 2, and a structured questionnaire (see Appendix A2 and A3) for conducting a baseline survey

among the treatment and control villages and households in March 2015.

As this project is a flagship project which will be brought to scale after 2016, it is of utmost

importance to make a robust evaluation of the activities of the project to guarantee sustainability

and a (positive) impact of the project on small holder farmers. This impact evaluation will focus on

several output and outcome indicators which are deviated in the Theory of Change (ToC) below.

As described in the former section, the intervention will implement activities on two different levels:

the irrigation scheme with four different technologies and access to irrigated land for farmers in four

villages and FFS in villages without irrigation.

Legend Control villages

Treatment Villages

Demo Fields IWAD

133

Figure 11 Theory of Change of farmer Field Schools

Source: own illustration, also Waddington et al. (2014)

The intervention’s outcomes will influence farmers in three dimensions: farming, household and

generally in living conditions, especially if the household is located close to the irrigation block

scheme.

For activities on the farmer level of farmers participating in the FFS, direct outcomes will be

observable. Farmers will eventually i) adopt new farming behavior and technologies ii) use improved

seeds and chemical inputs and iii) practice conservation farming methods. It is expected that farmers

understand the importance of land clearing methods without bushfires, of the application of

chemicals (timing and method) and of proper harvesting methods (especially cotton which has to be

picked carefully to maintain its quality) during the whole cultivation process. Accordingly, the

outcome “adoption of good agricultural practices” in the ToC represents the set of potential changes

in behavior that are intended through the FFS as minimum tillage, crop rotation, mulching and

precision fertilizer application. Exact indicators for this study have been designed based on theory

and reviews of agricultural studies, survey material and expert interviews. During an inception

134

mission from January 7th to January 14th 2015 the research team conducted interviews with trainers

of farmers, supervisors of trainers on the FFS content and received documents on the content of

FFS.

On the irrigation scheme level, IWAD will supervise farmer groups closely to implement the new

technologies and the application of CFM. For activities on the farmers’ level who participated in the

FFS, direct outcomes will be observable similar as to the case of conservation farming. Farmers will

additionally i) adopt new irrigation farming technologies ii) invest own time and money to work on

the irrigation farm (in contrast conservation farming farmers work on their own land) and pay for

water.

For farmers participating in the irrigation block scheme, the impact of improved agricultural

practices and technologies is expected to result in a second harvest per year. This means not only

100% higher yields and income increases but also implies more effort demanded from every farmer

in terms of labor input. Farmers participating in a FFS on CFM can get access to improved inputs in

the form of seeds and chemicals but only work on the land they provide themselves. Farmers should

be organized in farmer groups and have to show effort because of frequent visits by field agents.

Also IWAD will offer to buy the harvest and act as trader between the producer and markets. The

common procedure in selling harvest in the project area is via trade agents who visit the villages

after harvest. IWAD will offer high quality products and also likely better sales opportunities.

The influences on household level can be a direct result from the intervention or a mid to long term

result. If the households achieve higher yields and eventually crop variety, the nutritional status of

household members improves because of a higher quantity and variety. However, the effect will

occur with a time lag of one to two years. Farmers applying conservation farming practices are

expected to have decent yield increases in the first years and improving over time with more

experience. The effect of water management and reduced flooding risk potentially also reduces the

incidence of water related disease.

As a focus of the FFS lies especially in motivating women to participate, this can also result in the

empowerment of women concerning decision making on farming, nutrition, education and

expenditure. Improved anthropometric measures of children, the intra-household decision making

processes and expenditure patterns can be affected by women’s engagement in income generating

activities or higher contributions to household consumption.

The influence of the intervention on schooling outcomes is unclear and there has not yet been much

research done on the effect of improved farming methods on education. More intensive agricultural

135

practices and higher yields can either result in higher or lower school enrolment and school

attendance. School enrolment is usually higher than actual school attendance. Because of higher

income (more) children can enrol in school and pay for fees and books (also longer, e.g. proceed to

secondary school). But as more labour effort has to be invested when conservation farming methods

are applied, children might also be needed to conduct other household activities as watching their

siblings when the mother is working or collecting firewood and water. Therefore, this dimension has

been included in this impact evaluation.

Identification strategy

The major challenge in conducting this evaluation and measuring changes in outcomes is that

neither the irrigation block nor the FFS are allocated randomly. For the irrigation block, a random

allocation is not feasible due to technical and natural requirements of irrigation (i.e. having an

accessible source of water (river) nearby). Therefore, households participating in the program might

differ systematically from households not participating in the program. For the FFS treatment a pure

random assignment was also not possible because 13 treatment communities in the program area

have already been chosen for the first phase in 2014. These villages have already been selected

during the previous year and received demonstration fields for FFS on conservation farming.

Another 9 villages are selected as potential treatment villages for 2015 and onwards. These

communities are phased in communities and represent a first set of control communities because

they will receive the training between 2015 and 2017, so that we can see an impact over time on

adoption and diffusion. The phased in communities represent eligible areas still waiting to receive

the program. All treatment villages will be treated in 2015 and supervised until 2017. The nine new

villages will start step by step until 2017.

The matching indicator for selecting the control communities is population size. The four districts are

generally comparable concerning savannah agro-ecological conditions as soil or climate, lie in a quite

inaccessible area between Wa and Walewale called the ‘Overseas’ area, and mainly focus on

subsistence farming as main income activity. The whole area where the control group is sampled

from is eligible for the scale-up of the program after 2017.

A common method used to support internal validity is to make observations over time using the

difference-in-difference (DD) approach, as it has two advantages: the use of two time periods,

namely before and after an observed treatment took place, and the comparison of a treatment

group to a (similar) control group or counterfactual. Any correlation between treatment status and

observed or unobserved time-invariant village characteristics is neutralized by applying the DD

136

approach. The source of bias influencing the treatment outcomes thus will be reduced. We will

observe the impact indicators over time with a baseline in 2015 and a follow-up study in 2017.

The Treatment Group (T) is represented by a sample of farmer groups using irrigation (beneficiaries:

150-250 farmers) and the rain-fed farming scheme (beneficiaries: 3000 farmers). The Control Group

(C) is not part of the program and will maybe never benefit. However, the communities lie in an area

which has been identified to have the potential to be selected for the scaling-up of the program.

During the investigation and planning of the project, Wienco and IWAD have identified areas where

it would be possible to scale up the program. These areas serve as the control group for the

sampling because they face similar savannah agro-ecological conditions. Based on this, we selected

villages in the four districts: West Mamprusi, Mamprugu Moaduri, Builsa South and North Gonja. A

second condition for the sampling is population size of communities which we used to match control

villages to treatment villages.

So the treatment and control communities will firstly be similar regarding savannah agro-ecological

conditions and secondly be similar regarding population size and the households examined are all

households with farming as main source of income.

Sampling, sample size power calculations

In the ideal case, the rollout of the project, i.e. conservation farming training and access to irrigation,

would be randomized among all potential beneficiaries and treatment would be assigned according

to the random choice, not any other selection strategy. Any other selection strategy than

randomness might cause biased results, but being aware of this bias ex-ante enables us to control

for several factors. Therefore, we applied the following sampling techniques:

A two-stage clustered random sampling was applied with the first stage being a cluster on the village

level and the second stage on the household level. To make the sample population more

homogenous and decrease variance, the sample was stratified on agriculture as the main source of

income in the second stage, i.e. excluding households which are not engaged in farming as main

source of income (e.g. traders, fishers, and craftsmen).

For the first stage all villages in the four survey districts (West Mamprusi, Mamprugu Moaduri, North

Gonja and Builsa South) were listed and respective population size data was collected from the

Ghana Statistical Service in Tamale and in district offices. By using information on population size of

the treatment villages, the control villages were matched using propensity score matching, however

only on the one indicator population size. The primary sampling units were 266 villages out of which

137

192 could be matched to the 23 treatment villages based on population size. All 23 treatment

villages (4 irrigation and 19 FFS) were eligible for the survey and out of the 192 potential control

villages 27 were selected randomly as control villages. The number of control villages exceeds the

number of treatment villages to increase statistical power73.

The second stage sampling was based on full household listings in all treatment and control

communities. The full household listing in 50 villages was conducted in February 2015. During the

listing the following indicators were collected: Firstly, the name of the household head and secondly

whether farming is the main source of income of the household in order to make stratification

possible. After the listing process, one village was excluded from the list of control villages because

all households’ main source of income was fishing and not agriculture. Therefore, only 26 control

villages remained.

The study sample therefore comprises 49 communities in four districts with a total population of

approximately 30.000 individuals. Based on power calculations74 we derived a necessary sample size

of 1500 farmers interviewed in 49 communities to be able to detect an effect. In the end 1400

households were interviewed because some villages were very small with less than 30 households

and one control village was excluded. Nevertheless, the sample size should be sufficient as we

expect even larger effects of the programs than assumed in the power calculations. Larger effects

allow for a smaller sample.

Survey tools

In order to measure the potential evaluation outcomes and impacts outlined above several survey

tools were applied: Focus group discussion (FDG), expert interviews, and structured interviews

(household and village questionnaires).

In January 2015, FGDs were conducted in four villages, two already participating in the project in

2014 and two potential control villages. The principal tool for measuring impact is the structured

household questionnaires. This questionnaire included questions on socioeconomic household

characteristics (size, education, health, nutrition, financial situation) and agricultural practices (use

73 The power (or statistical power) of an impact evaluation is the likelihood that the study will detect a difference between the treatment and comparison groups, when in fact a difference exists. Power calculations indicate the smallest sample size necessary for an evaluation to detect a meaningful difference in outcomes between the treatment and comparison groups. By increasing the sample size, statistical power can be improved. 74 For the power calculation effect sizes for FFS on conservation farming we used. We used figures of studies investigating Integrated Crop Management (ICM) practices. These studies find that ICM practices increased from 18% to 31% after the FFS. We assumed a power of 90%, alpha 0.05 and intra‐cluster correlation of 0.40 (assumptions based on David and Asamoah, 2011, for Ghana and Erbaugh, 2010 for Uganda and Waddington et al. 2014). For the comparison of the irrigation farmers we found no adequate study but as an increase in yields by 100% is expected due to a second harvest (instead of only one before) we decided to oversample the villages with access to irrigation to increase statistical power.

138

of chemicals, planting techniques…), land holding, harvest and sales of harvest. As the project has a

special gender component to foster the role of women in agriculture, the questionnaire also

included a module on intra-household decision making with regards to daily expenditure, agriculture

and children’s schooling. The idea of these questions has been taken from the Ghanaian

Demographic and Health Survey Questionnaire 2008 but has been made more concrete by allowing

decisions to be done by more than one household member and being more detailed on decisions

related to agriculture. The other modules of the questionnaire were mainly taken from the Ghana

Living Standard and Measurement surveys conducted by the Ghanaian Statistical office and the

World Bank, after they were adjusted for the purpose of this study. This ensures that questions are

standardized to international questionnaires. For all parts of the questionnaire, answering these

questions was voluntary and households were informed that the data will be treated anonymously.

Interviews were conducted by male and female enumerators but there were more men than women

because women fear to work in remote rural areas. The survey was conducted in March 2015

because this is a month when farmers are not time constrained. The last harvest (usually beans) is

finished in January and February and the next farming season only starts in May or June, depending

on the rains. Farmers appreciated this procedure.

With the village questionnaire (see Appendix A2), information about regional and village specific

characteristics was collected. The questionnaire was usually asked to the village chief and a group of

elders who represent the local authority. It included questions on infrastructure access and quality,

local economic and agricultural conditions such as crops, cooperatives, aid projects, employment

opportunities, migration and prices.

Survey implementation

For the implementation of the survey, a team of Ghanaian, Dutch and German researchers

collaborated. For the organization of the field work, the research team worked closely with the

University of Tamale, Faculty of Agribusiness, Department of Climate Change and Food Security.

Prof. Dr. Joseph Amikuzuno was responsible for the recruitment of the interviewers which was a

major challenge because interviewers needed to know the local languages Dagbani, Buli and

Mampruli. Ms. Yvette Schutgens, a Dutch researcher was responsible for the survey logistics (hire

cars and renting accommodation for interviewers). Dr. Elena Gross and her PhD student Raymond

Frempong were responsible for designing the data entry tool and managing data entry and data

quality assessment. Each of the five survey teams with a total of 25 enumerators met daily with one

of the three responsible supervisors (Amikuzuno, Schutgens, Gross, Markert) to ensure the proper

139

implementation of the questionnaire. Random field visits were conducted by the supervision to

ensure that the enumerators interviewed households properly and followed all instructions.

The preparation of the baseline survey started during a preparatory field visit in January 2015. The

listing of villages and households started right after the field visit and lasted 4 weeks. The training of

enumerators started on February 23rd, 2015 and took one week including questionnaire training and

questionnaire pre-tests in the field (in villages not included in the sample). These pre-test served to

practice the questionnaire, verify the feasibility of the questions and to make further improvements

of content (e.g. including all possible answers for pre-coding to avoid open ended questions). The

data entrants participated in the training of enumerators in the morning and had special data entry

tool training in the afternoon while the enumerators conducted role plays and did translations of the

questionnaire modules into the local languages. The role of the interviewer was discussed including

how they had to present themselves when visiting a household.

The role out of the survey was planned by the supervisors and the other team members. IWAD

supported the research team in providing information on the geographical location of villages and

helped in finding accommodation for team members.

The baseline survey then took place from March 2nd until March 27th 2015. Each enumerator was

able to conduct two to four interviews per day. In the beginning, the survey advanced more slowly

with only two interviews per enumerator per day but after a few days enumerators were more

experienced and could do up to four interviews per day, depending on household size and the

number of crops and fields a household possessed.

The data entry took place in the town of Fumbisi as electricity was most stable there. The accuracy

of the entered data was checked and final revisions were made by the research team. The final

version of the data was handed over to Dr. Elena Gross on April 15th 2015 to start the data cleaning

process to reduce any possible measurement error (check data for essentials, consistency checks

(e.g. only women can have children), identifiers of households and persons, skip patterns, flag

outlier and implausible codes, re-open affected questionnaires).

140

5 Baseline results

Qualitative results of Focus Group Discussions

The following paragraphs give an overview of the results of the qualitative FGDs discussed with

chiefs and elders in four villages of the Mamprugu Moaduri in January 2015. The discussions in each

village lasted 2 hours with topics on land acquisition and tenure, land use, input use, farming

methods, irrigation and credit and market access.

Land acquisition:

Land ownership is vested in village chiefs with sub-chiefs acting as custodians of land in their areas

of jurisdiction on behalf of the village chief. Land in is not bought but acquired through a request to

the chief and payment of a token in the form of kola, i.e. a little sum of money. In addition, the land

user is expected to make annual gifts to the chief from produce. A particular piece of land is acquired

on first-come, first-serve basis and there is little discrimination in the allocation of land. Due to

abundance of land, conflicts over acquisition are few. Conflicts arising due to land use are managed

first by clan heads, then by the village chief if the former fail to resolve the conflict. Conflicts that

cannot be solved by the village chief are referred to a paramount chief (Wungu chief) for resolution.

In case of competition for the same piece of land, the land may be divided among the competitors if

it is a big piece of land or given to the oldest if it is too small to be divided. If the contestants are a

man and a woman, the village chief states that it is a case to case decision but that usually the

woman is given preference over the man. Strangers (people not originally from a village) may

acquire through lease land from the chief for farming and other uses; but the chief must be pre-

informed of the intended use before the land is leased. Women do not own land (even clan lands)

but may acquire land through the process outlined above.

Land use decision:

In acquiring land, the potential user pre-informs the chief about what use he/she intends to put the

land to e.g. for farming or construction of houses. Once acquired, the acquirer has independent

usage rights to the land and may decide what crops to cultivate. Farmers’ decisions in this regard

depends on the type of the land and their resource-endowments. Female farmers may also decide

what crops to plant independently, with the consent and advice of their husbands. Previously,

married female farmers only cultivated vegetables, but can now cultivate any crop in the farm

141

systems here. Once acquired, there is security to land use, but this is enhanced by the annual gifts of

crop produce.

Input use:

Framers use external inputs, especially fertilizer, pesticides and herbicides/weedicides. Inputs are

purchased from nearby markets like Loagri or bigger markets such as Fumbisi, Walewale, or Tamale.

Sometimes input dealers sell items within the village. Inputs which are purchased in the bigger

markets are paid for in cash and there is no opportunity for credit purchases. If inputs are purchased

in the village from input dealers, farmers may have the possibility to get credit, to pay cash or kind,

at the time of purchase or later with or without a pre-determined level of interest. Other input

technology, as e.g. tractors to prepare land at the onset of the cropping season, farmers have to

search with difficulty (“run around for 3 days”). Ploughing costs approximately GHC 40 per acre.

Irrigation

Farming in the area is largely rain-fed, however farmers are aware of irrigation farming but only very

few farmers actually practice irrigation farming in the dry season. The few farmers that are known by

the population use self-acquired hand pumps and water from the river to cultivate beans and water

melons. Farmers think that irrigation farming is too expensive as they cannot afford the water

pumps required to lift water from the river onto their farms.

Land clearing-the bushfire problem

Bushfires are believed to be caused largely by the Fulani pastoralists in the area, who burn the grass

to cause its regrowth for the feeding their cattle. Indigenes also cause bushfires through harvesting

wild honey. Farmers are however aware of the consequences of bushfires including the destruction

of feed for livestock. To prevent their matured but unharvested crops from being burnt, farmers may

create fire belts around their farms which then get out of control and cause larger damage.

Credit access and use:

Financial institutions are usually not available in the villages, and credit sources are limited to inputs

dealers or other informal financial service providers (village money lenders, family, and friends).

Usually farmers can take credit with interest for farming which may be paid back with farm produce

after harvest. The interest rate is considerably high, for instance, a loan of GHC100 may be paid back

with one bag (100 -120kg) of beans which is worth GHS200. For instance a tractor service provider

receives a bag of maize (approx. GHC80) for ploughing every acre of land on credit. Credit of GHS100

142

for the production of cowpea/beans is equally paid back with a bag of beans worth GHC230 at the

time of harvest. So farmers face very high interest rates with often more than 50 percent.

Market access:

Farmers in the village can sell their produce on the village market, on local markets in Yagaba,

Fumbisi and bigger towns or at regional markets in Walewale or Tamale. In case of farmers selling in

district and regional markets, farmers have little access to price information concerning their

produce. In this case, farmers largely rely on middlemen (male and female traders from bigger

towns) and the prices are determined by middlemen. Another alternative sometimes used by

farmers are other sources of information as external linkages, personal contacts or telephones.

Immediately after harvest, prices of most commodities are low but gradually increase overtime.

Even though farmers know that they benefit from higher prices if they delay the sale of their

produce, they are unable to do so. Due to urgent need for cash farmers sell the harvest to repay

loans and meet other monetary needs of the household. Even though there is price bargaining,

buyers employ their market power to decide the prices of produce. Farmers however do not feel

that they are cheated because they realize the cost of transportation traders have to bear and the

need for them to make profit.

The IWAD project

For most community members in the FGD it was largely not sure how farmers may be selected into

the IWAD irrigation scheme. Farmers are however willing to be part of the project and see the

participation in the project’s demonstration fields and the fact that some of the community’s lands

have been acquired for the project as opportunities for their selection and participation. The

communication strategy of IWAD was usually via the village chief to maintain the hierarchy

structure.

Quantitative results of baseline survey

This section presents some results of the baseline survey to give an overview of the socio-economic

conditions of households and common farming practices. Also we will report on the quality of data

collected, first by looking on the number of non-response values and second, by comparing our

results with the latest Ghana Living Standard Survey (GLSS) of 2012 if possible.

Overall the non-response rate was minimal. We had in total four households which moved away

between the listing in February 2015 and the start of the baseline survey in March 2015. One issue

was that during the listing process, households which were in fact one household were listed as

143

separate households, e.g. the second wife. In this case we replaced the household with a household

that was also sampled randomly from the original household list. No household refused to answer

the questionnaire.

Village level comparison

The village questionnaire replicated most of the questions used in the GLSS 2012 and was adjusted

for some special issues on agriculture. The village questionnaire was administered to the village

head, usually the chief and the group of elders. The average population size is 1278 persons per

village; however, the range is large with some villages only having 5 households and others having

up to 550. The data we got from the Ghana Statistical Service turned out to be quite accurate but as

we will see later on, the sample is still balanced between the treatment and control group. About 90

percent of villages exist since more than 50 years. Only 5 villages were established between 10 and

50 years ago. Although we would expect that rural-urban migration is a threat to the existence of

these villages, 55 percent of villages experience an inflow of people rather than an outflow since

2005. However, 36 percent of villages experience an outflow while the remaining 8 percent see no

considerable change in population.

In terms of temporary migration, 90 percent of villages experience a periodic outflow of labor,

mainly to urban areas where people are engaged in self-employed activities. Young women between

15 and 30 often leave to the southern parts of Ghana for Kayayo. Women sell small goods on

markets or at the side of the street.

The most prevalent ethnic group in the sample are the Mamprusi (65 percent) followed by the Builsa

(12 percent), Gonja (8 percent) and other groups as the Fulani, Akan and Dagboma. Interestingly, 45

percent of villages are single ethnic group villages and in 55 percent of villages more than one ethnic

group exists. Besides farming as the main economic activity, the population is engaged in fishing (25

percent of villages), trading (45 percent of villages), handicraft (6 percent), mining (6 percent) and

sand winning and quarrying (2 percent). About 16 percent of villages only rely on farming as the

main economic activity and no other economic diversification exists.

More than 90 percent of villages indicate that a motorable road passes their village, however 78

percent of villages also report that the road is inaccessible during approximately 3 months of the

year in the rainy season. 15 percent of villages are connected to the electricity grid, but the majority

has no access to electricity or relies on a few solar panels (26 percent). 70 percent of villages have

access to a modern water source (borehole, public standpipe or protected well according to the

144

WHO (2008) definition) but still 30 percent rely on unimproved sources as unprotected wells, rivers

or ponds.

In terms of infrastructure, the villages of the sample are not well equipped. None of the

communities has a bank and the average distance to the next bank is 46 km. Only 20 percent have a

health center and the average distance to the next health center if none subsists is 10 km.

Agricultural extension offices, who provide advisory services on behalf of the Ministry of Agriculture,

are only available in 4 percent of villages. The major mean of public transport is the motorbike,

which in fact is a privately provided service. In 70 percent of villages a primary school subsists, 30

percent have a junior high school, only two villages have a senior high school (Wulugu and Yagaba).

The most grown crops in the area are maize, rice, millet and groundnuts which are harvested once a

year after the rainy season. In 40 percent of villages, there exists some kind of farmer cooperation.

In seven villages Masara N’arziki75 is active, which is one of the subsidiaries of Wienco Ltd. 82

percent of villages state that they have received less rains in the last 12 months than before which is

a threat to their survival.

The major development project in the region is the provision of electricity. The grid arrives from

Walewale step by step in the Overseas area.

Household characteristics

In this part we report the descriptive statistics of the socio-economic structure of the households

and farming practices in the survey area. A household is defined according to the definition of the

Ghana Statistical Service as a person or group of persons, related or unrelated, who live together in

the same housing unit, share a common cooking pot and water. For the purpose of this study we

further specified that the members of a household farm on the same fields, share the harvest and

revenues from selling the harvest.

In total 1400 households with 10436 individuals were interviewed in 49 communities. The sample is

balanced between treatment and control group in terms of the size of the household, the

characteristics of the household head (age, education and gender) and its composition of men,

women and children, see Table 2. The average household head is 43.95 years old, 19 percent have

any educational background and 3 percent are female. On average, households have 7.39 members

75 The program supports small and medium sized farmers and assists in good agricultural practices (see http://wienco.com/Site/subsidiaries.html ).

145

with a slight higher share of females of 3.73 compared to 3.66 males. The average household has

3.37 children below the age of 15 and 1.1 child below the age of five.

Table 26 Household characteristics

Sample Mean

se Mean Control Group

se Mean Treatment Group

se Difference

p-value

Household size 7.39 (0.222) 7.04 (0.217) 7.74 (0.378) 0.71 0.111

Age of household head 43.95 (0.570) 43.73 (0.630) 44.18 (0.941) 0.45 0.694

Household head any education

0.19 (0.019) 0.20 (0.026) 0.19 (0.028) -0.01 0.724

Female household head 0.03 (0.006) 0.03 (0.007) 0.03 (0.009) 0.00 0.992

Number of males 3.66 (0.107) 3.50 (0.121) 3.81 (0.175) 0.31 0.156

Number of females 3.73 (0.126) 3.52 (0.115) 3.93 (0.219) 0.41 0.107

No. children <5 years 1.10 (0.045) 1.06 (0.055) 1.14 (0.071) 0.08 0.366

No. children <15 years 3.37 (0.109) 3.27 (0.141) 3.47 (0.166) 0.20 0.367

No. Of students/pupils 1.79 (0.103) 1.70 (0.136) 1.89 (0.151) 0.19 0.346

Stunting children (<5years)

0.40 (0.021) 0.37 (0.025) 0.42 (0.032) 0.05 0.228

Wasting children (<5years)

0.32 (0.021) 0.31 (0.034) 0.32 (0.026) 0.01 0.829

Underweight children (<5years)

0.21 (0.020) 0.22 (0.032) 0.20 (0.024) -0.02 0.563

Home Ownership 0.94 (0.011) 0.93 (0.017) 0.94 (0.014) 0.02 0.425

Number of Rooms 4.28 (0.140) 4.08 (0.164) 4.49 (0.226) 0.42 0.143

Improved water source 0.65 (0.050) 0.60 (0.061) 0.71 (0.076) 0.11 0.265

Improved latrine 0.07 (0.013) 0.05 (0.017) 0.08 (0.018) 0.04 0.146

Observations 1400 703 697

Source: SK Basin baseline dataset 2015.

In terms of the nutrition related health status of children we observe that, on average, 40 percent of

children suffer from stunting (low height for age), 32 percent of children suffer from wasting (low

weight for age) and 21 percent of children are underweight (low weight for height).

Within the living conditions house ownership, number of rooms per household, use of improved

water source and improved latrines, households do not differ from each other in treatment and

control groups but show the expected pattern when compared to the national statistics of the

Ghana Statistical Service and other regions of rural West Africa. The household characteristics listed

in Table 2 all have very low numbers of missing values with less than 1 percent.

In terms of household expenditure, we also do not see a significant difference between the

treatment and control group, see Table 3. On average, households have a total annual expenditure

of GHC 5909. This is in line with the figures of the GLSS who reports an annual household

expenditure in the Northern Region of GHC 7153 in 2012 (GLSS, 2014). Our figure is about 17

percent lower, because we only consider rural areas while the GLSS also includes urban areas.

146

Expenditure on non-food items such as clothing, housing, education, health insurance and

ceremonies amounts to GHC 1438 per year. The expenditure on food items during the last 12

months was GHC 4752 showing that 72 percent of total expenditure goes to food and 28 percent are

spent on non-food items. On average women contribute 25 percent to overall household

expenditure while the contribution to food items is slightly higher with 30 percent.

Table 27 Household expenditure

Sample Mean

se Mean Control Group

se Mean Treatment

Group

se Difference p-value

Annual household expenditure

5,909.74 (244.876) 6,099.39 (311.214) 5,715.22 (370.329) -384.2 0.43

Food expenditure 4,752.33 (217.700) 4,845.42 (260.448) 4,659.09 (346.433) -186.3 0.67

Non-food expenditure

1,438.48 (67.007) 1,484.80 (93.677) 1,390.61 (93.933) -94.19 0.48

Share of food in expenditure

0.72 (0.008) 0.72 (0.013) 0.72 (0.011) -0.006 0.71

Share of woman's contribution in expenditure

0.25 (0.014) 0.24 (0.023) 0.27 (0.017) 0.032 0.27

Share of woman’s contribution to food expenditure

0.30 (0.019) 0.28 (0.030) 0.32 (0.024) 0.038 0.33

Source: SK Basin baseline dataset 2015.

Within the expenditure variables we have some missing values. 10 percent of households were not

able or did not want to report all necessary figures and therefore it is not possible to calculate

expenditure. Also we did not include households that were clear outliers, i.e. the top 1 percent of

households, due to errors during data collection or clear misreporting. However, the missing values

with 10 percent are still very low compared to other surveys where missing values sometimes

amount to 30 percent.

All households are self-employed farmers and earn the main income from farming. This is due to the

stratified sampling design where we excluded households that do not indicate that farming is a

major income source. Only 2% state a paid off-farm activity as main income activity apart from

farming as second activity, see Table 4. Slightly more households, 10 percent, have a paid secondary

activity besides farming as first income activity. If off-farm work is practiced, individuals are engaged

in trading, tailoring, transformation of agricultural products, production of nutrition (baker, butcher,

and miller) or an artisan. The average earnings are GHC 71.96 per month, with a lower value in the

control group. The differences are not statistically different between groups. In this set of variables

we also have very low numbers of missing with less than 1 percent of households not responding.

147

Table 28 Employment

Sample

Mean

se Mean Contro

l Group

se Mean Treatment Group

se Difference

p-valu

e

Earnings from paid labor by household in the last month

71.96 (11.832)

56.58 (16.539)

87.48 (15.862)

30.91 0.184

Paid main activity 0.02 (0.005) 0.03 (0.008) 0.02 (0.005) -0.00985 0.278

Paid secondary activity

0.10 (0.015) 0.08 (0.015) 0.12 (0.026) 0.0426 0.159

Source: SK Basin baseline dataset 2015.

Intra-household decision making

In the decision making module we included seven different intra-household decisions: decision on

household expenditure, land use, crops, sale of crops, purchasing assets, schooling of children, and

how the money of the breadwinner is used in general. Here we only report the decisions related to

agriculture, other results are available upon request.

The decision about how land is used, whether land should be rented, bought or sold, is mainly done

by the household head, both in the treatment and the control group, see Figure 4. Less often, the

decision is made by the household head and his wife(s) or the people in the household together.

Seldom, the decision is made by the oldest male child, the wife alone or as an individual decision by

the person responsible for the land.

Figure 12 Decision about land use

Source: SK Basin baseline dataset 2015.

The decision making process on which crops to grow looks similar to land use, but here in Figure 5

we see that the decision is less often made by the male household head alone, but rather it is a

common decision of husband and wife, especially in the control group.

148

Figure 13 Decision on crops to grow

Source: SK Basin baseline dataset 2015.

In contrast to crop cultivation, the sale of crops is clearly dominated by the household head,

especially in the control group, see Figure 6. Here, a common decision by husband and wife is more

often made in the control than the treatment group.

Figure 14 Sale of crops

Source: SK Basin baseline dataset 2015.

Farming Methods

In this section we report on the farming characteristics and farming methods which are important

outcomes for the impact measurement. The clear majority of 80 percent of the farmers receive their

seeds for planting from the previous harvest, 10 percent buy it on the market and the remaining 10

percent receive it from friends or family or a sales agent. There is no statistical difference between

the two groups in terms of seed purchase.

Table 5 reports the farming characteristics concerning chemical use, one important dimension that

should face an impact from the intervention. The average farmer cultivates 12 acres (4.8 hectare)

and experienced 2.9 self-reported agriculture shocks during the last year (drought, heavy or no

rainfall, different crop pests). About 81 percent of farmers use at least one type of chemical as input

for agriculture and there is a statistically significant difference between treatment and control

group. However, when differentiating between the three different chemicals, fertilizer, herbicides

149

and pesticides, we see a significant difference between the groups only for pesticides and

herbicides. The treatment group is more likely to use herbicides and pesticides but the share of using

fertilizer is similar. However, these results have to be treated with caution. Most farmers use some

form of local products which cannot be identified and the effect on soil and crops is questionable.

We received a list from IWAD with adequate, good quality agrochemicals which can be bought on

local markets. When comparing the list to the type of pesticides and herbicides used by farmers,

only 1 percent of farmers used a pesticide and only 8 percent use an herbicide that is

recommendable. When comparing usage of recommendable agrochemicals between groups the

difference becomes insignificant. The share of missing values in the whole farming sections is below

1 percent.

Table 29 Farming Characteristics

Sample Mean

se Mean Control Group

se Mean Treatm

ent Group

se Difference

p-value

Land cultivated in acres=0.4 hectares

12.19 (0.639) 11.91 (0.882) 12.47 (0.926) 0.57 0.66

Number of shocks in the last year

2.90 (0.112) 2.87 (0.111) 2.93 (0.196) 0.06 0.80

Use of any kind of chemical 0.81 (0.027) 0.74 (0.045) 0.88 (0.017) 0.15 0.00 Use of fertilizer 0.47 (0.032) 0.48 (0.051) 0.47 (0.038) -0.01 0.93

Use of pesticide 0.41 (0.043) 0.23 (0.039) 0.59 (0.047) 0.36 0.00

Use of herbicide 0.63 (0.032) 0.52 (0.039) 0.74 (0.032) 0.22 0.00

Use of recommendable pesticide

0.01 (0.003) 0.01 (0.003) 0.01 (0.004) 0.01 0.26

Use of recommendable herbicide

0.08 (0.016) 0.06 (0.017) 0.09 (0.026) 0.03 0.28

Source: SK Basin baseline dataset 2015.

Concerning land clearing activities after harvest, about 75 percent of the households indicated that

they have seen bushfires during the last year, equally distributed between treatment and control

group. About 64 percent of the sample regard “pile and burn” as the best method for land clearing

at the end of the harvest period. The share is higher for the control group with 70 percent

compared to the treatment group with 60 percent, the difference is statistically significant on a 10

percent level. This significant difference stems from the already treated FFS 2014 villages where 56

percent of farmers indicate that pile and burn is the adequate method. The share of the farmers in

the FFS 2015 and later treatment group is also 70 percent, similar to the control group. The results

are similar for mulching76 as the best method: about 10 percent of farmers in the control and 2015

treatment group think that mulching is the adequate method for land preparation, while 18 percent

76 IWAD is very active in promoting mulching as the best alternative after harvest to moist the soil and enrich the soil with nutrients.

150

in the 2014 treatment group indicate that mulching is best practice. So the FFS 2014 increased the

share of farmers regarding mulching as adequate by 8 percent.

Table 6 gives an overview of the major crops grown and harvested in the project area: maize, rice,

millet and groundnuts. Overall there is no considerable difference between the treatment and

control group concerning crops. Maize is cultivated by, on average, 87 percent of farmers as it is also

one of the major food items in the area. On average farmers harvest 1.09 tons of maize. The

productivity is 0.17 tons (equivalent to 170kg) per acre.

The share of farmers growing rice differs between treatment and control group but the productivity

of rice growers is equal between groups. The proportion of farmers growing rice is 22 percent in the

treatment group and 11 percent in the control group, the difference is significant on a 5 percent

level. On average farmers harvest 0.58 tones in total and 0.13 tones (equivalent to 130kg) per acre.

For millet we find the reverse: more farmers in the control group grow millet (23 percent) than in

the treatment group (11 percent). The same holds for groundnuts (42 vs. 19 percent). These are

crops IWAD wants to promote in the treatment area, so we will investigate whether there is catching

up. The productivity in tons per acre, however, is equal between treatment and control group, only

absolute values of tons of groundnut harvested are higher for the control groups because farmers

grow it on larger areas. We consider the differences between groups as minor and they pose no

threat to the impact measurement.

Table 30 Production of maize, rice, millet and groundnuts

Sample Mean

se Mean Control Group

se Mean Treatme

nt Group

se Difference

p-value

Proportion of farmers that cultivated maize

0.87 -0.021 0.85 -0.033 0.88 -0.026 0.0293 0.49

Tons of maize harvested

1.09 -0.081 1.17 -0.128 1.02 -0.098 -0.146 0.367

Tons of maize harvested per acre

0.17 -0.009 0.18 -0.014 0.16 -0.011 -0.0197 0.282

Proportion of farmers that cultivated rice

0.16 -0.027 0.11 -0.028 0.22 -0.047 0.108 0.0537

Tons of rice harvested 0.58 -0.078 0.51 -0.151 0.61 -0.084 0.0998 0.568

Tons of rice harvested per acre

0.13 -0.019 0.15 -0.047 0.12 -0.017 -0.0322 0.525

Proportion of farmers that cultivated millet

0.17 (0.033) 0.23 (0.053) 0.11 (0.033) -0.113 0.08

Tons of millet harvested

1.09 (0.166) 1.07 (0.215) 1.11 (0.245) 240 0.91

Tons of millet harvested per acre

0.33 (0.032) 0.34 (0.047) 0.31 (0.029) 240 0.65

151

Proportion of farmers that cultivated groundnut

0.31 (0.033) 0.42 (0.042) 0.19 (0.034) -0.227 0.00

Tons of groundnut harvested

1.74 (0.225) 2.03 (0.290) 1.11 (0.131) 429 0.01

Tons of groundnut harvested per acre

0.44 (0.034) 0.47 (0.041) 0.38 (0.051) 429 0.18

Source: SK Basin baseline dataset 2015.

Finally, we turn to the sales of crops. Table 7 reports figures of the self-reported market values of

production from farming. Farmers produce goods worth around GHC 3000 per year, with no

statistical difference between groups. The amount for the harvest sold and used for own

consumption is almost equal. The rest of the harvest is used to trade in kind goods which is hardly

impossible to measure quantitatively.

Table 31 Sale of harvest and home consumption

Sample Mean

se Mean Control Group

se Mean Treatment

Group

se Difference p-value

Market value of crops harvested

2,958.26 (228.440) 2,814.73 (340.773) 3,099.50 (298.779) 284.80 0.53

Market value of crops kept for home consumption

1,199.32 (96.455) 1,071.37 (74.988) 1,325.23 (174.127) 253.90 0.19

Market value of crops sold

1,204.69 (107.629) 1,157.37 (157.852) 1,251.25 (144.563) 93.89 0.66

Source: SK Basin baseline dataset 2015.

6 Risks for impact measurement Self-selection of participants in groups

We will face selection bias because farmers assign themselves to the treatment (training in FFS) out

of own motivation. This means, that only motivated farmers are participating. This is nothing we can

solve by any statistical strategy, but it will be important to be aware of the problem and control for

various socioeconomic characteristics of the participants. For the impact measurement, we will then

differentiate between the Intention-To-Treat (ITT) and Average-Treatment-effect on the Treated

(ATT). Therefore, we need to follow the assignment of farmers who receive inputs from IWAD or

work on the irrigation field during upcoming years.

During the farming season 2015, we already collected precise information on the farmers

participating in the FFS 2014 and receiving inputs from IWAD in 2015, and farmers receiving a plot of

land in the irrigation scheme. We matched the farmers receiving any treatment to the households in

152

the baseline survey to calculate the percentage of observations in our sample that is part of any of

the IWAD interventions.

Table 8 gives an overview of the FFS 2014 communities in column 1 and the respective estimated

number of households per village in column 2 (Ghana Statistical Service, 2015). Column 3 shows the

number of farmers that receive inputs for credit after following the FFS 2014 through all the sessions

and column 4 gives the share of households receiving IWAD treatment as percent of the whole

population. Column 5 lists the number of households per village interviewed in the baseline survey

(note that Kuuba as part of the irrigation treatment villages is very small and we interviewed all 29

households living in the village). Column 6 shows the number of households that match between the

farmers in the IWAD interventions and our baseline survey. Column 7 calculates the share of

matching farmers in our sample to compare the numbers to column 4. We see that our numbers are

higher than the real share of households participating in treatment. With the slight oversampling we

should be able to measure the interventions’ impact after the follow up survey in 2018.

The ATT effect should also be measurable because the research team decided to collect the

household data from all 281 farmers participating in the conservation agriculture intervention and

40 farmers participating in the irrigation scheme. For following the FFS some budget from the

baseline phase has been saved to allow monitoring uptake and match farmers from the FFS to our

baseline survey data regularly.

Table 32 Uptake of IWAD interventions

(1) (2) (3) (4) (5) (6) (7)

Treatment communities 2014

Population HH

Farmers participating conservation agriculture

Household share

HH baseline sample

Matching FFS=Interviewed

Share

(N) % (N)

Gbedembilisi 347 24 6.92 26 3 11.54

Kuuba 57 0 0 29 0 0

Loagri 3812 20 0.52 70 2 2.86

Gbima 559 17 3.04 70 10 14.29

Yagaba 3558 9 0.25 70 1 1.43

Jadema 1605 22 1.37 26 0 0

Kubori 1885 48 2.55 26 4 15.38

Kunkwak 1731 24 1.39 26 1 3.85

Prima 695 20 2.88 26 1 3.85

Sakpaba 837 20 2.39 26 2 7.69

Goriba 50 46 92 26 22 84.62

Kpasenkpe 2107 7 0.33 26 0 0

Wungu 7357 24 0.33 26 0 0

Total farmers 281 46

153

Irrigation Yagaba

3558 40 1.12 70 2 2.86

Parallel programs

The SUSTAINABLE MAIZE PROGRAMME IN NORTH GHANA is active in the same area and might cause

measurement problems because of spill-over of knowledge from the different projects to the

farmers in the region. This is a project by Wienco Ltd. and its affiliated company Masara N’arziki

financed by the Dutch Sustainable Entrepreneurship and Food Security Facility. We got access to the

community data where Masara is active to control other programs influencing similar outcome

measures. Masara N’arziki Farmers Association (MAFA) is a farmer cooperation program, initiated in

2005 by Wienco (Ghana) Ltd., and is active in six of the sample village.

Necessary information on communities and population size were collected at the district offices to

get complete lists of communities, population size, and information on the Millennium Villages (MV)

in the area (http://millenniumvillages.org/the-villages/sada-northern-ghana/ ). This makes it

possible to control for other programs in the area and learn how other project structures concerning

the institutional level affect program outcomes, as e.g. the MV approach. 9 villages are assigned to

the Millennium project, 5 in the control group and 4 in the treatment group. We took one extra MV

in the control group to increase statistical power.

Time horizon

As Waddington et al. (2014) point out, the very short time horizon of impact evaluations of FFS is a

threat to the impact measurement because behavioral change takes time. In the case of irrigation,

benefits in yields can be seen quite fast, as soon as a second harvest is possible. However, in case of

conservation agriculture a behavioral change of farmers is intended which will most likely take

several years. Also the improvement of the soil takes some times until yields can be improved.

Therefore, we postpone the endline survey to March 2018. A constant monitoring of the process on

the ground is necessary to decide on the right time for the endline survey.

154

References Akudugu, M. A., Garforth, C. J., & Dorward, P. (2013). Improvement in Crop Production in Ghana: Is it

due to Area Expansion or Increased Productivity? Developing Country Studies, 3(1), 176-186.

Alhassan, M., Loomis, J., Frasier, M., Davies, S., & Andales, A. (2013). Estimating Farmers’ Willingness

to Pay for Improved Irrigation: An Economic Study of the Bontanga Irrigation Scheme in Northern

Ghana. Journal of Agricultural Science, 5(4), p31.

Braimah, C. A., and S. Agodzo (2014). Challenges To Efficient And Effective Irrigation Water

Management In Ghana–The Case Of Bontanga Irrigation Project. IOSR Journal of Engineering, 4(4),

34-41.

David, S. and Asamoah, C. 2011, “Farmer Knowledge as an Early Indicator of IPM Adoption: A Case

Study from Cocoa Farmer Field Schools in Ghana”, Journal of Sustainable Development in Africa, vol.

14, no. 4, pp. 213–224.

Erbaugh, J.M., Donnermeyer, J., Amujal, M. and Kidoido, M. 2010. Assessing the Impact of Farmer

Field School Participation on IPM Adoption in Uganda, Journal of International Agricultural and

Extension Education, vol. 17, no. 3, pp. 5–17.

FAO (2004). http://www.fao.org/docrep/007/y5784e/y5784e02.htm#TopOfPage

FAO (2014). Ghana Irrigation Market Brief. FAO/IFC Cooperation Programme. Rome, Italy.

FAO 2015. What is Conservation Agriculture? Food and Agriculture Organization of the United

Nations. http://www.fao.org/ag/ca/1a.html

Ghana Statistical Service, 2014. 2010 Population and Housing Census.

http://www.statsghana.gov.gh/pop_stats.html

Giller, K. E., Witter, E., Corbeels, M. and P. Tittonell. 2009. Conservation agriculture and smallholder

farming in Africa: The heretics’ view. Field Crops Research 114 (1): 23–34.

doi:10.1016/j.fcr.2009.06.017.

Intergovernmental Panel on Climate Change (IPCC). (2007). Fourth Assessment Report: Climate

Change. Geneva. Retrieved from http://www.ipcc.ch/publications_and_data/ar4/syr/en/spms3.html

Kassam, A., Friedrich, T. and R. Derpsch. 2012. Overview of the Global Spread of Conservation

Agriculture. The journal of field actions, (Special Issue 6).

Kassam, A., Friedrich, T., Shaxson, F. and J. Pretty. 2009. The spread of Conservation Agriculture:

Justification, sustainability and uptake. International Journal of Agricultural Sustainability 7 (4): 292–

320.

Khandker, S., R. Koolwal, Gayatri B. Samad, Hussain A. 2010. Handbook on Impact Evaluation –

Quantitative Methods and Practices. The World Bank, Washington D.C.

Knowler, D., and B. Bradshaw. 2007. Farmers’ adoption of conservation agriculture: A review and

synthesis of recent research. Food Policy 32 (1): 25–48.

155

Mahama, C. 2009. Local government and traditional authorities in Ghana: Towards a more

productive relationship. Commonwealth Journal of Local Governance 4:8–25.

Namara, R. E., Horowitz, L., Nyamadi, B., & Barry, B. (2011). Irrigation Development in Ghana: Past

experiences, emerging opportunities, and future directions. Ghana Strategy Support Program (GSSP)

Working Paper, (0026).

ODI (1976). Irrigation management in developing countries: a suggested action programme.

Tonah, S. (2012). The Politicisation of a Chieftaincy Conflict: The Case of Dagbon, Northern Ghana.

Nordic Journal of African Studies, 21(1), 1-20.

Waddington, H, Snilstveit, B, Hombrados, J, Vojtkova, M, Phillips, D, Davies, P and White, H. 2014.

Farmer Field Schools for Improving Farming Practices and Farmer Outcomes: A Systematic Review

Campbell Systematic Reviews 2014(6), DOI: 10.4073/csr.2014.6

Wall, P.C. 2007. Tailoring Conservation Agriculture to the Needs of Small Farmers in Developing

Countries. Journal of Crop Improvement 19 (1-2): 137–55.

156

Annex Annex A1 List of evaluation indicators

Level Evaluation Question Indicator

Outcome On farmer level Do farmers complete farmer field schools? How many farmer start the training

and how many farmers receive inputs after the first year?

Do farmers adopt new seeds and planting methods? For each of the steps of the cultivation process the handling process is observed during:

1) Planting and Sowing season

2) Application of fertilizer

3) Application of herbicides

4) Application of pesticides

5) Harvest

Do farmers adopt good agricultural practices and other recommended behavior?

Good agricultural practices: minimum tillage, crop rotation, mulching, precision chemical application

Do women actively participate in agricultural activities, FFS and irrigation?

Share of farming women, share of women in FFS and share of women who receive inputs

Impacts

On farm level Do farmers have higher yields? Measure harvest output

Collect information on sales and market prices

Do farmers produce a higher variety of crops? Type of crops planted and harvested used for home consumption

Collect household data on types of crops planted, land use, crop management

Do farmer react better to crop pests? Collect data on shock coping strategies

Do farmers pay back loans? Collect information on credit, savings and remittances

Use information from IWAD about repayment behavior of farmers

Do FFS farmers gain better access to markets than other farmers? Elicit buyer structure from interviewed farms.

Collect information of regional market prices

Do farmers make use of the provided market strategies? Collect information on sales procedure

157

Do farmers which are members of the FFS get better prices on markets?

Ask for selling prices and cross check with price data from regional markets

Do farmers produce higher quality crops? Ask farmer about perceived quality of provided seeds and chemicals

Qualitative interviews with IWAD staff and Ghanaian agricultural extension workers

On household level Do households have higher expenditure? Collect information on household

food- and non-food expenditure

Do households accumulate more assets? Collect information on assets

Can households better deal with other shocks (health, death, house damage,…)?

Shock incidence and shock financial losses

Do families consume a higher variety of goods? Collect data on food consumption, diversity, food expenditure, food scarcity Collect data on food consumption, expenditure

Do households are less deprived in terms of food scarcity? Collect data on food consumption

Do child anthropometric measures improve? Measure height and weight of children<5

Does school enrolment and attendance change? Collect data on household level and flow up children's school attendance in irrigation villages (and maybe some FFS villages depending on budget)

Do women participate in intra-household decision making? Collect data on decision making in different dimensions:

1) expenditure

2) nutrition expenditure

3) types of crops planted

4) management of crops

5) selling of crops

6) education of children

Do household members find paid employed in the PPP consortia? Collect data on off-farm/ farm employment

Collect data on regional employees/contractors of the construction works and farming activities of IWAD

On institutional level

Does institutional arrangement have influence on farmers’ outcomes and behavior?

Institutional analysis

This is a publication of

Netherlands Enterprise Agency

Prinses Beatrixlaan 2

PO Box 93144 | 2509 AC The Hague

T +31 (0) 88 042 42 42

E klantcontact@rvo.nl

www.rvo.nl

This publication was commissioned by the ministry of Foreign Affairs.

© Netherlands Enterprise Agency | April 2020

Publication number: RVO-074-2020/RP-INT

NL Enterprise Agency is a department of the Dutch ministry of Economic

Affairs and Climate Policy that implements government policy for

Agricultural, sustainability, innovation, and international business and

cooperation. NL Enterprise Agency is the contact point for businesses,

educational institutions and government bodies for information and advice,

financing, networking and regulatory matters.

Netherlands Enterprise Agency is part of the ministry of Economic Affairs and

Climate Policy.